org.deeplearning4j.da

Class DenoisingAutoEncoder

java.lang.Object

org.deeplearning4j.nn.BaseNeuralNetwork

org.deeplearning4j.da.DenoisingAutoEncoder

All Implemented Interfaces:

Serializable, Cloneable, NeuralNetwork, Persistable, NeuralNetEpochListener

public class DenoisingAutoEncoder
extends BaseNeuralNetwork
implements Serializable

Denoising Autoencoder. Add Gaussian noise to input and learn a reconstruction function.

Author:

Adam Gibson

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DenoisingAutoEncoder.Builder

Nested classes/interfaces inherited from interface org.deeplearning4j.nn.NeuralNetwork

NeuralNetwork.LossFunction, NeuralNetwork.OptimizationAlgorithm

Field Summary

Fields inherited from class org.deeplearning4j.nn.BaseNeuralNetwork

applySparsity, dist, doMask, dropOut, fanIn, firstTimeThrough, gradientListeners, hBias, hBiasAdaGrad, input, l2, lossFunction, momentum, nHidden, normalizeByInputRows, nVisible, optimizationAlgo, optimizer, renderWeightsEveryNumEpochs, rng, sparsity, useAdaGrad, useRegularization, vBias, vBiasAdaGrad, W, wAdaGrad

Constructor Summary

Constructors

Constructor and Description
DenoisingAutoEncoder()
DenoisingAutoEncoder(org.jblas.DoubleMatrix input, int nVisible, int nHidden, org.jblas.DoubleMatrix W, org.jblas.DoubleMatrix hbias, org.jblas.DoubleMatrix vbias, org.apache.commons.math3.random.RandomGenerator rng, double fanIn, org.apache.commons.math3.distribution.RealDistribution dist)

Method Summary

Methods

Modifier and Type	Method and Description
org.jblas.DoubleMatrix	getCorruptedInput(org.jblas.DoubleMatrix x, double corruptionLevel) Corrupts the given input by doing a binomial sampling given the corruption level
NeuralNetworkGradient	getGradient(Object[] params)
org.jblas.DoubleMatrix	getHiddenValues(org.jblas.DoubleMatrix x)
org.jblas.DoubleMatrix	getReconstructedInput(org.jblas.DoubleMatrix y)
double	lossFunction(Object[] params) The loss function (cross entropy, reconstruction error,...)
org.jblas.DoubleMatrix	reconstruct(org.jblas.DoubleMatrix x) All neural networks are based on this idea of minimizing reconstruction error.
Pair<org.jblas.DoubleMatrix,org.jblas.DoubleMatrix>	sampleHiddenGivenVisible(org.jblas.DoubleMatrix v) Sample hidden mean and sample given visible
Pair<org.jblas.DoubleMatrix,org.jblas.DoubleMatrix>	sampleVisibleGivenHidden(org.jblas.DoubleMatrix h) Sample visible mean and sample given hidden
void	train(org.jblas.DoubleMatrix x, double lr, double corruptionLevel) Perform one iteration of training
void	train(org.jblas.DoubleMatrix input, double lr, Object[] params) Train one iteration of the network
void	trainTillConvergence(org.jblas.DoubleMatrix x, double lr, double corruptionLevel) Run a network optimizer
void	trainTillConvergence(org.jblas.DoubleMatrix input, double lr, Object[] params) Trains via an optimization algorithm such as SGD or Conjugate Gradient

Methods inherited from class org.deeplearning4j.nn.BaseNeuralNetwork

applyDropOutIfNecessary, applySparsity, clone, dropOut, epochDone, fanIn, getAdaGrad, getDist, getGradientListeners, gethBias, gethBiasAdaGrad, getInput, getL2, getLossFunction, getMomentum, getnHidden, getnVisible, getOptimizationAlgorithm, getReConstructionCrossEntropy, getRenderEpochs, getRng, getSparsity, getvBias, getVBiasAdaGrad, getW, hBiasMean, initWeights, jostleWeighMatrix, l2RegularizedCoefficient, load, lossFunction, merge, negativeLogLikelihood, negativeLoglikelihood, normalizeByInputRows, resetAdaGrad, setAdaGrad, setDist, setDropOut, setFanIn, setGradientListeners, sethBias, setHbiasAdaGrad, setInput, setL2, setLossFunction, setMomentum, setnHidden, setnVisible, setOptimizationAlgorithm, setRenderEpochs, setRng, setSparsity, setvBias, setVBiasAdaGrad, setW, squaredLoss, transpose, triggerGradientEvents, update, updateGradientAccordingToParams, write

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DenoisingAutoEncoder

public DenoisingAutoEncoder()

DenoisingAutoEncoder

public DenoisingAutoEncoder(org.jblas.DoubleMatrix input,
                    int nVisible,
                    int nHidden,
                    org.jblas.DoubleMatrix W,
                    org.jblas.DoubleMatrix hbias,
                    org.jblas.DoubleMatrix vbias,
                    org.apache.commons.math3.random.RandomGenerator rng,
                    double fanIn,
                    org.apache.commons.math3.distribution.RealDistribution dist)

Method Detail

getCorruptedInput

public org.jblas.DoubleMatrix getCorruptedInput(org.jblas.DoubleMatrix x,
                                       double corruptionLevel)

Corrupts the given input by doing a binomial sampling given the corruption level

Parameters:

x - the input to corrupt

corruptionLevel - the corruption value

Returns:

the binomial sampled corrupted input

sampleHiddenGivenVisible

public Pair<org.jblas.DoubleMatrix,org.jblas.DoubleMatrix> sampleHiddenGivenVisible(org.jblas.DoubleMatrix v)

Description copied from interface: NeuralNetwork

Sample hidden mean and sample given visible

Specified by:

sampleHiddenGivenVisible in interface NeuralNetwork

Parameters:

v - the the visible input

Returns:

a pair with mean, sample

sampleVisibleGivenHidden

public Pair<org.jblas.DoubleMatrix,org.jblas.DoubleMatrix> sampleVisibleGivenHidden(org.jblas.DoubleMatrix h)

Description copied from interface: NeuralNetwork

Sample visible mean and sample given hidden

Specified by:

sampleVisibleGivenHidden in interface NeuralNetwork

Parameters:

h - the the hidden input

Returns:

a pair with mean, sample

getHiddenValues

public org.jblas.DoubleMatrix getHiddenValues(org.jblas.DoubleMatrix x)

getReconstructedInput

public org.jblas.DoubleMatrix getReconstructedInput(org.jblas.DoubleMatrix y)

trainTillConvergence

public void trainTillConvergence(org.jblas.DoubleMatrix x,
                        double lr,
                        double corruptionLevel)

Run a network optimizer

Parameters:

x - the input

lr - the learning rate

corruptionLevel - the corruption level

train

public void train(org.jblas.DoubleMatrix x,
         double lr,
         double corruptionLevel)

Perform one iteration of training

Parameters:

x - the input

lr - the learning rate

corruptionLevel - the corruption level to train with

reconstruct

public org.jblas.DoubleMatrix reconstruct(org.jblas.DoubleMatrix x)

Description copied from class: BaseNeuralNetwork

All neural networks are based on this idea of minimizing reconstruction error. Both RBMs and Denoising AutoEncoders have a component for reconstructing, ala different implementations.

Specified by:

reconstruct in class BaseNeuralNetwork

Parameters:

x - the input to reconstruct

Returns:

the reconstructed input

trainTillConvergence

public void trainTillConvergence(org.jblas.DoubleMatrix input,
                        double lr,
                        Object[] params)

Description copied from interface: NeuralNetwork

Trains via an optimization algorithm such as SGD or Conjugate Gradient

Specified by:

trainTillConvergence in interface NeuralNetwork

Parameters:

input - the input to train on

lr - the learning rate to use

params - the params (k,corruption level, max epochs,...)

lossFunction

public double lossFunction(Object[] params)

Description copied from class: BaseNeuralNetwork

The loss function (cross entropy, reconstruction error,...)

Specified by:

lossFunction in class BaseNeuralNetwork

Returns:

the loss function

train

public void train(org.jblas.DoubleMatrix input,
         double lr,
         Object[] params)

Description copied from class: BaseNeuralNetwork

Train one iteration of the network

Specified by:

train in interface NeuralNetwork

Specified by:

train in class BaseNeuralNetwork

Parameters:

input - the input to train on

lr - the learning rate to train at

params - the extra params (k, corruption level,...)

getGradient

public NeuralNetworkGradient getGradient(Object[] params)

Specified by:

getGradient in interface NeuralNetwork