org.deeplearning4j.rbm

Class RBM

java.lang.Object

org.deeplearning4j.nn.BaseNeuralNetwork

org.deeplearning4j.rbm.RBM

All Implemented Interfaces:

Serializable, Cloneable, NeuralNetwork, Persistable, NeuralNetEpochListener

Direct Known Subclasses:

ConvolutionalRBM, CRBM, GaussianRectifiedLinearRBM

public class RBM
extends BaseNeuralNetwork

Restricted Boltzmann Machine. Markov chain with gibbs sampling. Based on Hinton et al.'s work Great reference: http://www.iro.umontreal.ca/~lisa/publications2/index.php/publications/show/239

Author:

Adam Gibson

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	RBM.Builder

Nested classes/interfaces inherited from interface org.deeplearning4j.nn.NeuralNetwork

NeuralNetwork.LossFunction, NeuralNetwork.OptimizationAlgorithm

Field Summary

Fields

Modifier and Type	Field and Description
protected NeuralNetworkOptimizer	optimizer

Fields inherited from class org.deeplearning4j.nn.BaseNeuralNetwork

applySparsity, dist, doMask, dropOut, fanIn, firstTimeThrough, gradientListeners, hBias, hBiasAdaGrad, input, l2, lossFunction, momentum, nHidden, normalizeByInputRows, nVisible, optimizationAlgo, renderWeightsEveryNumEpochs, rng, sparsity, useAdaGrad, useRegularization, vBias, vBiasAdaGrad, W, wAdaGrad

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	RBM()
protected	RBM(org.jblas.DoubleMatrix input, int nVisible, int n_hidden, org.jblas.DoubleMatrix W, org.jblas.DoubleMatrix hbias, org.jblas.DoubleMatrix vBias, org.apache.commons.math3.random.RandomGenerator rng, double fanIn, org.apache.commons.math3.distribution.RealDistribution dist)

Method Summary

Methods

Modifier and Type	Method and Description
void	contrastiveDivergence(double learningRate, int k, org.jblas.DoubleMatrix input) Contrastive divergence revolves around the idea of approximating the log likelihood around x1(input) with repeated sampling.
double	freeEnergy(org.jblas.DoubleMatrix visibleSample) Free energy for an RBM Lower energy models have higher probability of activations
NeuralNetworkGradient	getGradient(Object[] params)
Pair<Pair<org.jblas.DoubleMatrix,org.jblas.DoubleMatrix>,Pair<org.jblas.DoubleMatrix,org.jblas.DoubleMatrix>>	gibbhVh(org.jblas.DoubleMatrix h) Gibbs sampling step: hidden ---> visible ---> hidden
double	lossFunction(Object[] params) The loss function (cross entropy, reconstruction error,...)
org.jblas.DoubleMatrix	propDown(org.jblas.DoubleMatrix h) Calculates the activation of the hidden: sigmoid(h * W + vbias)
org.jblas.DoubleMatrix	propUp(org.jblas.DoubleMatrix v) Calculates the activation of the visible : sigmoid(v * W + hbias)
org.jblas.DoubleMatrix	reconstruct(org.jblas.DoubleMatrix v) Reconstructs the visible input.
Pair<org.jblas.DoubleMatrix,org.jblas.DoubleMatrix>	sampleHiddenGivenVisible(org.jblas.DoubleMatrix v) Binomial sampling of the hidden values given visible
Pair<org.jblas.DoubleMatrix,org.jblas.DoubleMatrix>	sampleVisibleGivenHidden(org.jblas.DoubleMatrix h) Guess the visible values given the hidden
void	train(org.jblas.DoubleMatrix input, double lr, Object[] params) Train one iteration of the network
void	trainTillConvergence(double learningRate, int k, org.jblas.DoubleMatrix input) Trains till global minimum is found.
void	trainTillConvergence(org.jblas.DoubleMatrix input, double lr, Object[] params) Note: k is the first input in params.

Methods inherited from class org.deeplearning4j.nn.BaseNeuralNetwork

applyDropOutIfNecessary, applySparsity, clone, dropOut, epochDone, fanIn, getAdaGrad, getDist, getGradientListeners, gethBias, gethBiasAdaGrad, getInput, getL2, getLossFunction, getMomentum, getnHidden, getnVisible, getOptimizationAlgorithm, getReConstructionCrossEntropy, getRenderEpochs, getRng, getSparsity, getvBias, getVBiasAdaGrad, getW, hBiasMean, initWeights, jostleWeighMatrix, l2RegularizedCoefficient, load, lossFunction, merge, negativeLogLikelihood, negativeLoglikelihood, normalizeByInputRows, resetAdaGrad, setAdaGrad, setDist, setDropOut, setFanIn, setGradientListeners, sethBias, setHbiasAdaGrad, setInput, setL2, setLossFunction, setMomentum, setnHidden, setnVisible, setOptimizationAlgorithm, setRenderEpochs, setRng, setSparsity, setvBias, setVBiasAdaGrad, setW, squaredLoss, transpose, triggerGradientEvents, update, updateGradientAccordingToParams, write

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

optimizer

protected NeuralNetworkOptimizer optimizer

Constructor Detail

RBM

protected RBM()

RBM

protected RBM(org.jblas.DoubleMatrix input,
   int nVisible,
   int n_hidden,
   org.jblas.DoubleMatrix W,
   org.jblas.DoubleMatrix hbias,
   org.jblas.DoubleMatrix vBias,
   org.apache.commons.math3.random.RandomGenerator rng,
   double fanIn,
   org.apache.commons.math3.distribution.RealDistribution dist)

Method Detail

trainTillConvergence

public void trainTillConvergence(double learningRate,
                        int k,
                        org.jblas.DoubleMatrix input)

Trains till global minimum is found.

Parameters:

learningRate -

k -

input -

contrastiveDivergence

public void contrastiveDivergence(double learningRate,
                         int k,
                         org.jblas.DoubleMatrix input)

Contrastive divergence revolves around the idea of approximating the log likelihood around x1(input) with repeated sampling. Given is an energy based model: the higher k is (the more we sample the model) the more we lower the energy (increase the likelihood of the model) and lower the likelihood (increase the energy) of the hidden samples. Other insights: CD - k involves keeping the first k samples of a gibbs sampling of the model.

Parameters:

learningRate - the learning rate to scale by

k - the number of iterations to do

input - the input to sample from

getGradient

public NeuralNetworkGradient getGradient(Object[] params)

freeEnergy

public double freeEnergy(org.jblas.DoubleMatrix visibleSample)

Free energy for an RBM Lower energy models have higher probability of activations

Parameters:

visibleSample - the sample to test on

Returns:

the free engery for this sample

sampleHiddenGivenVisible

public Pair<org.jblas.DoubleMatrix,org.jblas.DoubleMatrix> sampleHiddenGivenVisible(org.jblas.DoubleMatrix v)

Binomial sampling of the hidden values given visible

Parameters:

v - the visible values

Returns:

a binomial distribution containing the expected values and the samples

gibbhVh

public Pair<Pair<org.jblas.DoubleMatrix,org.jblas.DoubleMatrix>,Pair<org.jblas.DoubleMatrix,org.jblas.DoubleMatrix>> gibbhVh(org.jblas.DoubleMatrix h)

Gibbs sampling step: hidden ---> visible ---> hidden

Parameters:

h - the hidden input

Returns:

the expected values and samples of both the visible samples given the hidden and the new hidden input and expected values

sampleVisibleGivenHidden

public Pair<org.jblas.DoubleMatrix,org.jblas.DoubleMatrix> sampleVisibleGivenHidden(org.jblas.DoubleMatrix h)

Guess the visible values given the hidden

Parameters:

h -

Returns:

propUp

public org.jblas.DoubleMatrix propUp(org.jblas.DoubleMatrix v)

Calculates the activation of the visible : sigmoid(v * W + hbias)

Parameters:

v - the visible layer

Returns:

the approximated activations of the visible layer

propDown

public org.jblas.DoubleMatrix propDown(org.jblas.DoubleMatrix h)

Calculates the activation of the hidden: sigmoid(h * W + vbias)

Parameters:

h - the hidden layer

Returns:

the approximated output of the hidden layer

reconstruct

public org.jblas.DoubleMatrix reconstruct(org.jblas.DoubleMatrix v)

Reconstructs the visible input. A reconstruction is a propdown of the reconstructed hidden input.

Specified by:

reconstruct in class BaseNeuralNetwork

Parameters:

v - the visible input

Returns:

the reconstruction of the visible input

trainTillConvergence

public void trainTillConvergence(org.jblas.DoubleMatrix input,
                        double lr,
                        Object[] params)

Note: k is the first input in params.

Parameters:

input - the input to train on

lr - the learning rate to use

params - the params (k,corruption level, max epochs,...)

lossFunction

public double lossFunction(Object[] params)

Description copied from class: BaseNeuralNetwork

The loss function (cross entropy, reconstruction error,...)

Specified by:

lossFunction in class BaseNeuralNetwork

Returns:

the loss function

train

public void train(org.jblas.DoubleMatrix input,
         double lr,
         Object[] params)

Description copied from class: BaseNeuralNetwork

Train one iteration of the network

Specified by:

train in interface NeuralNetwork

Specified by:

train in class BaseNeuralNetwork

Parameters:

input - the input to train on

lr - the learning rate to train at

params - the extra params (k, corruption level,...)