org.deeplearning4j.rbm

Class RBM

java.lang.Object

org.deeplearning4j.nn.BaseNeuralNetwork

org.deeplearning4j.rbm.RBM

All Implemented Interfaces:

Serializable, Cloneable, NeuralNetwork, Persistable

Direct Known Subclasses:

CRBM

public class RBM
extends BaseNeuralNetwork

Restricted Boltzmann Machine. Markov chain with gibbs sampling. Based on Hinton et al.'s work Great reference: http://www.iro.umontreal.ca/~lisa/publications2/index.php/publications/show/239

Author:

Adam Gibson

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	RBM.Builder

Field Summary

Fields

Modifier and Type	Field and Description
protected NeuralNetworkOptimizer	optimizer

Fields inherited from class org.deeplearning4j.nn.BaseNeuralNetwork

dist, fanIn, hBias, input, l2, momentum, nHidden, nVisible, renderWeightsEveryNumEpochs, rng, sparsity, useRegularization, vBias, W

Constructor Summary

Constructors

Constructor and Description
RBM()
RBM(org.jblas.DoubleMatrix input, int n_visible, int n_hidden, org.jblas.DoubleMatrix W, org.jblas.DoubleMatrix hbias, org.jblas.DoubleMatrix vbias, org.apache.commons.math3.random.RandomGenerator rng, double fanIn, org.apache.commons.math3.distribution.RealDistribution dist)
RBM(int nVisible, int nHidden, org.jblas.DoubleMatrix W, org.jblas.DoubleMatrix hbias, org.jblas.DoubleMatrix vbias, org.apache.commons.math3.random.RandomGenerator rng, double fanIn, org.apache.commons.math3.distribution.RealDistribution dist)

Method Summary

Methods

Modifier and Type	Method and Description
void	contrastiveDivergence(double learningRate, int k, org.jblas.DoubleMatrix input) Contrastive divergence revolves around the idea of approximating the log likelihood around x1(input) with repeated sampling.
NeuralNetworkGradient	getGradient(Object[] params)
Pair<Pair<org.jblas.DoubleMatrix,org.jblas.DoubleMatrix>,Pair<org.jblas.DoubleMatrix,org.jblas.DoubleMatrix>>	gibbhVh(org.jblas.DoubleMatrix h) Gibbs sampling step: hidden ---> visible ---> hidden
double	lossFunction(Object[] params) The loss function (cross entropy, reconstruction error,...)
org.jblas.DoubleMatrix	propDown(org.jblas.DoubleMatrix h) Propagates hidden down to visible
org.jblas.DoubleMatrix	propUp(org.jblas.DoubleMatrix v)
org.jblas.DoubleMatrix	reconstruct(org.jblas.DoubleMatrix v) Reconstructs the visible input.
Pair<org.jblas.DoubleMatrix,org.jblas.DoubleMatrix>	sampleHiddenGivenVisible(org.jblas.DoubleMatrix v) Binomial sampling of the hidden values given visible
Pair<org.jblas.DoubleMatrix,org.jblas.DoubleMatrix>	sampleVGivenH(org.jblas.DoubleMatrix h) Guess the visible values given the hidden
void	train(org.jblas.DoubleMatrix input, double lr, Object[] params) Train one iteration of the network
void	trainTillConvergence(double learningRate, int k, org.jblas.DoubleMatrix input) Trains till global minimum is found.
void	trainTillConvergence(org.jblas.DoubleMatrix input, double lr, Object[] params) Note: k is the first input in params.

Methods inherited from class org.deeplearning4j.nn.BaseNeuralNetwork

clone, fanIn, getDist, gethBias, getInput, getL2, getMomentum, getnHidden, getnVisible, getReConstructionCrossEntropy, getRenderEpochs, getRng, getSparsity, getvBias, getW, initWeights, jostleWeighMatrix, l2RegularizedCoefficient, load, lossFunction, merge, setDist, setFanIn, sethBias, setInput, setL2, setMomentum, setnHidden, setnVisible, setRenderEpochs, setRng, setSparsity, setvBias, setW, squaredLoss, transpose, update, write

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

optimizer

protected NeuralNetworkOptimizer optimizer

Constructor Detail

RBM

public RBM()

RBM

public RBM(int nVisible,
   int nHidden,
   org.jblas.DoubleMatrix W,
   org.jblas.DoubleMatrix hbias,
   org.jblas.DoubleMatrix vbias,
   org.apache.commons.math3.random.RandomGenerator rng,
   double fanIn,
   org.apache.commons.math3.distribution.RealDistribution dist)

RBM

public RBM(org.jblas.DoubleMatrix input,
   int n_visible,
   int n_hidden,
   org.jblas.DoubleMatrix W,
   org.jblas.DoubleMatrix hbias,
   org.jblas.DoubleMatrix vbias,
   org.apache.commons.math3.random.RandomGenerator rng,
   double fanIn,
   org.apache.commons.math3.distribution.RealDistribution dist)

Method Detail

trainTillConvergence

public void trainTillConvergence(double learningRate,
                        int k,
                        org.jblas.DoubleMatrix input)

Trains till global minimum is found.

Parameters:

learningRate -

k -

input -

contrastiveDivergence

public void contrastiveDivergence(double learningRate,
                         int k,
                         org.jblas.DoubleMatrix input)

Contrastive divergence revolves around the idea of approximating the log likelihood around x1(input) with repeated sampling. Given is an energy based model: the higher k is (the more we sample the model) the more we lower the energy (increase the likelihood of the model) and lower the likelihood (increase the energy) of the hidden samples. Other insights: CD - k involves keeping the first k samples of a gibbs sampling of the model.

Parameters:

learningRate - the learning rate to scale by

k - the number of iterations to do

input - the input to sample from

getGradient

public NeuralNetworkGradient getGradient(Object[] params)

sampleHiddenGivenVisible

public Pair<org.jblas.DoubleMatrix,org.jblas.DoubleMatrix> sampleHiddenGivenVisible(org.jblas.DoubleMatrix v)

Binomial sampling of the hidden values given visible

Parameters:

v - the visible values

Returns:

a binomial distribution containing the expected values and the samples

gibbhVh

public Pair<Pair<org.jblas.DoubleMatrix,org.jblas.DoubleMatrix>,Pair<org.jblas.DoubleMatrix,org.jblas.DoubleMatrix>> gibbhVh(org.jblas.DoubleMatrix h)

Gibbs sampling step: hidden ---> visible ---> hidden

Parameters:

h - the hidden input

Returns:

the expected values and samples of both the visible samples given the hidden and the new hidden input and expected values

sampleVGivenH

public Pair<org.jblas.DoubleMatrix,org.jblas.DoubleMatrix> sampleVGivenH(org.jblas.DoubleMatrix h)

Guess the visible values given the hidden

Parameters:

h -

Returns:

propUp

public org.jblas.DoubleMatrix propUp(org.jblas.DoubleMatrix v)

propDown

public org.jblas.DoubleMatrix propDown(org.jblas.DoubleMatrix h)

Propagates hidden down to visible

Parameters:

h - the hidden layer

Returns:

the approximated output of the hidden layer

reconstruct

public org.jblas.DoubleMatrix reconstruct(org.jblas.DoubleMatrix v)

Reconstructs the visible input. A reconstruction is a propdown of the reconstructed hidden input.

Specified by:

reconstruct in class BaseNeuralNetwork

Parameters:

v - the visible input

Returns:

the reconstruction of the visible input

trainTillConvergence

public void trainTillConvergence(org.jblas.DoubleMatrix input,
                        double lr,
                        Object[] params)

Note: k is the first input in params.

lossFunction

public double lossFunction(Object[] params)

Description copied from class: BaseNeuralNetwork

The loss function (cross entropy, reconstruction error,...)

Specified by:

lossFunction in class BaseNeuralNetwork

Returns:

the loss function

train

public void train(org.jblas.DoubleMatrix input,
         double lr,
         Object[] params)

Description copied from class: BaseNeuralNetwork

Train one iteration of the network

Specified by:

train in interface NeuralNetwork

Specified by:

train in class BaseNeuralNetwork

Parameters:

input - the input to train on

lr - the learning rate to train at

params - the extra params (k, corruption level,...)