org.deeplearning4j.nn.layers.feedforward.rbm

Class RBM

java.lang.Object

org.deeplearning4j.nn.layers.BaseLayer<LayerConfT>

org.deeplearning4j.nn.layers.BasePretrainNetwork<RBM>

org.deeplearning4j.nn.layers.feedforward.rbm.RBM

All Implemented Interfaces:

Serializable, Cloneable, Layer, Model

public class RBM
extends BasePretrainNetwork<RBM>

Restricted Boltzmann Machine. Markov chain with gibbs sampling. Supports the following visible units: binary gaussian softmax linear Supports the following hidden units: rectified binary gaussian softmax linear Based on Hinton et al.'s work Great reference: http://www.iro.umontreal.ca/~lisa/publications2/index.php/publications/show/239

Author:

Adam Gibson

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.deeplearning4j.nn.api.Layer

Layer.TrainingMode, Layer.Type

Field Summary

Fields

Modifier and Type	Field and Description
protected org.nd4j.linalg.api.ndarray.INDArray	hiddenSigma
protected org.nd4j.linalg.api.ndarray.INDArray	sigma

Fields inherited from class org.deeplearning4j.nn.layers.BaseLayer

conf, dropoutApplied, dropoutMask, gradient, gradientsFlattened, gradientViews, index, input, iterationListeners, maskArray, optimizer, params, paramsFlattened, score, solver

Constructor Summary

Constructors

Constructor and Description
RBM(NeuralNetConfiguration conf)
RBM(NeuralNetConfiguration conf, org.nd4j.linalg.api.ndarray.INDArray input)

Method Summary

Modifier and Type	Method and Description
org.nd4j.linalg.api.ndarray.INDArray	activate(boolean training) Reconstructs the visible INPUT.
void	computeGradientAndScore() Update the score
void	contrastiveDivergence() Contrastive divergence revolves around the idea of approximating the log likelihood around x1(input) with repeated sampling.
void	fit(org.nd4j.linalg.api.ndarray.INDArray input) Note: k is the first input hidden params.
Pair<Pair<org.nd4j.linalg.api.ndarray.INDArray,org.nd4j.linalg.api.ndarray.INDArray>,Pair<org.nd4j.linalg.api.ndarray.INDArray,org.nd4j.linalg.api.ndarray.INDArray>>	gibbhVh(org.nd4j.linalg.api.ndarray.INDArray h) Gibbs sampling step: hidden ---> visible ---> hidden
void	iterate(org.nd4j.linalg.api.ndarray.INDArray input) iterate one iteration of the network
org.nd4j.linalg.api.ndarray.INDArray	propDown(org.nd4j.linalg.api.ndarray.INDArray h) Calculates the activation of the hidden: activation(h * W + vbias)
org.nd4j.linalg.api.ndarray.INDArray	propUp(org.nd4j.linalg.api.ndarray.INDArray v) Calculates the activation of the visible : sigmoid(v * W + hbias)
org.nd4j.linalg.api.ndarray.INDArray	propUp(org.nd4j.linalg.api.ndarray.INDArray v, boolean training) Calculates the activation of the visible : sigmoid(v * W + hbias)
Pair<org.nd4j.linalg.api.ndarray.INDArray,org.nd4j.linalg.api.ndarray.INDArray>	sampleHiddenGivenVisible(org.nd4j.linalg.api.ndarray.INDArray v) Binomial sampling of the hidden values given visible
Pair<org.nd4j.linalg.api.ndarray.INDArray,org.nd4j.linalg.api.ndarray.INDArray>	sampleVisibleGivenHidden(org.nd4j.linalg.api.ndarray.INDArray h) Guess the visible values given the hidden
Layer	transpose() Return a transposed copy of the weights/bias (this means reverse the number of inputs and outputs on the weights)

Methods inherited from class org.deeplearning4j.nn.layers.BasePretrainNetwork

createGradient, getCorruptedInput, numParams, numParams, params, params, setParams, setScoreWithZ

Methods inherited from class org.deeplearning4j.nn.layers.BaseLayer

accumulateScore, activate, activate, activate, activate, activate, activationMean, applyDropOutIfNecessary, applyLearningRateScoreDecay, backpropGradient, batchSize, calcGradient, calcL1, calcL2, clear, clone, conf, createGradient, derivativeActivation, error, fit, getIndex, getInput, getInputMiniBatchSize, getListeners, getMaskArray, getOptimizer, getParam, gradient, gradientAndScore, initParams, input, layerConf, merge, paramTable, preOutput, preOutput, preOutput, preOutput, score, setBackpropGradientsViewArray, setConf, setIndex, setInput, setInputMiniBatchSize, setListeners, setListeners, setMaskArray, setParam, setParams, setParamsViewArray, setParamTable, toString, type, update, update, validateInput

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

sigma

protected org.nd4j.linalg.api.ndarray.INDArray sigma

hiddenSigma

protected org.nd4j.linalg.api.ndarray.INDArray hiddenSigma

Constructor Detail

RBM

public RBM(NeuralNetConfiguration conf)

RBM

public RBM(NeuralNetConfiguration conf,
           org.nd4j.linalg.api.ndarray.INDArray input)

Method Detail

contrastiveDivergence

public void contrastiveDivergence()

Contrastive divergence revolves around the idea of approximating the log likelihood around x1(input) with repeated sampling. Given is an energy based model: the higher k is (the more we sample the model) the more we lower the energy (increase the likelihood of the model) and lower the likelihood (increase the energy) of the hidden samples. Other insights: CD - k involves keeping the first k samples of a gibbs sampling of the model.

computeGradientAndScore

public void computeGradientAndScore()

Description copied from interface: Model

Update the score

Specified by:

computeGradientAndScore in interface Model

Overrides:

computeGradientAndScore in class BaseLayer<RBM>

transpose

public Layer transpose()

Description copied from interface: Layer

Return a transposed copy of the weights/bias (this means reverse the number of inputs and outputs on the weights)

Specified by:

transpose in interface Layer

Overrides:

transpose in class BaseLayer<RBM>

Returns:

the transposed layer

sampleHiddenGivenVisible

public Pair<org.nd4j.linalg.api.ndarray.INDArray,org.nd4j.linalg.api.ndarray.INDArray> sampleHiddenGivenVisible(org.nd4j.linalg.api.ndarray.INDArray v)

Binomial sampling of the hidden values given visible

Specified by:

sampleHiddenGivenVisible in class BasePretrainNetwork<RBM>

Parameters:

v - the visible values

Returns:

a binomial distribution containing the expected values and the samples

gibbhVh

public Pair<Pair<org.nd4j.linalg.api.ndarray.INDArray,org.nd4j.linalg.api.ndarray.INDArray>,Pair<org.nd4j.linalg.api.ndarray.INDArray,org.nd4j.linalg.api.ndarray.INDArray>> gibbhVh(org.nd4j.linalg.api.ndarray.INDArray h)

Gibbs sampling step: hidden ---> visible ---> hidden

Parameters:

h - the hidden input

Returns:

the expected values and samples of both the visible samples given the hidden and the new hidden input and expected values

sampleVisibleGivenHidden

public Pair<org.nd4j.linalg.api.ndarray.INDArray,org.nd4j.linalg.api.ndarray.INDArray> sampleVisibleGivenHidden(org.nd4j.linalg.api.ndarray.INDArray h)

Guess the visible values given the hidden

Specified by:

sampleVisibleGivenHidden in class BasePretrainNetwork<RBM>

Parameters:

h - the hidden units

Returns:

a visible mean and sample relative to the hidden states passed in

propUp

public org.nd4j.linalg.api.ndarray.INDArray propUp(org.nd4j.linalg.api.ndarray.INDArray v,
                                                   boolean training)

Calculates the activation of the visible : sigmoid(v * W + hbias)

Parameters:

v - the visible layer

Returns:

the approximated activations of the visible layer

propUp

public org.nd4j.linalg.api.ndarray.INDArray propUp(org.nd4j.linalg.api.ndarray.INDArray v)

Calculates the activation of the visible : sigmoid(v * W + hbias)

Parameters:

v - the visible layer

Returns:

the approximated activations of the visible layer

propDown

public org.nd4j.linalg.api.ndarray.INDArray propDown(org.nd4j.linalg.api.ndarray.INDArray h)

Calculates the activation of the hidden: activation(h * W + vbias)

Parameters:

h - the hidden layer

Returns:

the approximated output of the hidden layer

activate

public org.nd4j.linalg.api.ndarray.INDArray activate(boolean training)

Reconstructs the visible INPUT. A reconstruction is a propdown of the reconstructed hidden input.

Specified by:

activate in interface Layer

Overrides:

activate in class BaseLayer<RBM>

Parameters:

training - true or false

Returns:

the reconstruction of the visible input

fit

public void fit(org.nd4j.linalg.api.ndarray.INDArray input)

Note: k is the first input hidden params.

Specified by:

fit in interface Model

Overrides:

fit in class BaseLayer<RBM>

Parameters:

input - the data to fit the model to

iterate

public void iterate(org.nd4j.linalg.api.ndarray.INDArray input)

Description copied from class: BaseLayer

iterate one iteration of the network

Specified by:

iterate in interface Model

Overrides:

iterate in class BaseLayer<RBM>

Parameters:

input - the input to iterate on