Reference

pcd!(wrbm::WeightNormRBM, data; kwargs...)

Train a weight-normalized RBM with Persistent Contrastive Divergence (PCD), following the same conventions as RestrictedBoltzmannMachines.pcd!, but optimizing the weight norms g and directions u instead of the weights w.

Returns (state, ps), the optimizer state and the optimized parameters.

∂free_energy(wrbm, v; wts, moments)

Gradient of free_energy(RBM(wrbm), v) with respect to the weight-normalized parameterization. Returns a NamedTuple with fields visible, hidden (gradients with respect to the layer parameter arrays layer.par), and g, u (gradients with respect to the weight norms and directions).

gu2w(g, u) -> w, un

Returns w, un (as a NamedTuple), from g, u, where:

\[\mathbf{w} = g \frac{\mathbf{u}}{\|\mathbf{u}\|}\]

and un are the norms $\|\mathbf{u}\|$.

w2gu(w, un) -> g, u

Returns g, u (as a NamedTuple), such that

\[\mathbf{w} = g \frac{\mathbf{u}}{\|\mathbf{u}\|}\]

where the norms $\|\mathbf{u}\|$ are given by un.

weight_norms(rbm)

Norms of weight patterns attached to each hidden unit.

∂wnorm(∂w, g, u)

Given the gradients ∂w of a function f(w) with respect to w, returns the gradients ∂g, ∂u of f with respect to the re-parameterization:

\[\mathbf{w} = g \frac{\mathbf{u}}{\|\mathbf{u}\|}\]