org.deeplearning4j.nn

Class BaseNeuralNetwork

java.lang.Object

org.deeplearning4j.nn.BaseNeuralNetwork

All Implemented Interfaces:

Serializable, Cloneable, NeuralNetwork, Persistable

Direct Known Subclasses:

DenoisingAutoEncoder, RBM

public abstract class BaseNeuralNetwork
extends Object
implements NeuralNetwork, Persistable

Baseline class for any Neural Network used as a layer in a deep network such as an DBN

Author:

Adam Gibson

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BaseNeuralNetwork.Builder<E extends BaseNeuralNetwork>

Field Summary

Fields

Modifier and Type	Field and Description
org.apache.commons.math3.distribution.RealDistribution	dist
double	fanIn
org.jblas.DoubleMatrix	hBias
org.jblas.DoubleMatrix	input
double	l2
double	momentum
int	nHidden Number of hidden units One tip with this is usually having more hidden units than inputs (read: input rows here) will typically cause terrible overfitting.
int	nVisible
NeuralNetworkOptimizer	optimizer
int	renderWeightsEveryNumEpochs
org.apache.commons.math3.random.RandomGenerator	rng
double	sparsity
boolean	useRegularization
org.jblas.DoubleMatrix	vBias
org.jblas.DoubleMatrix	W

Constructor Summary

Constructors

Constructor and Description
BaseNeuralNetwork()
BaseNeuralNetwork(org.jblas.DoubleMatrix input, int nVisible, int nHidden, org.jblas.DoubleMatrix W, org.jblas.DoubleMatrix hbias, org.jblas.DoubleMatrix vbias, org.apache.commons.math3.random.RandomGenerator rng, double fanIn, org.apache.commons.math3.distribution.RealDistribution dist)
BaseNeuralNetwork(int nVisible, int nHidden, org.jblas.DoubleMatrix W, org.jblas.DoubleMatrix hbias, org.jblas.DoubleMatrix vbias, org.apache.commons.math3.random.RandomGenerator rng, double fanIn, org.apache.commons.math3.distribution.RealDistribution dist)

Method Summary

Methods

Modifier and Type	Method and Description
NeuralNetwork	clone()
double	fanIn()
org.apache.commons.math3.distribution.RealDistribution	getDist()
org.jblas.DoubleMatrix	gethBias()
org.jblas.DoubleMatrix	getInput()
double	getL2()
double	getMomentum()
int	getnHidden()
int	getnVisible()
double	getReConstructionCrossEntropy() Reconstruction entropy.
int	getRenderEpochs()
org.apache.commons.math3.random.RandomGenerator	getRng()
double	getSparsity()
org.jblas.DoubleMatrix	getvBias()
org.jblas.DoubleMatrix	getW()
protected void	initWeights() Initialize weights.
void	jostleWeighMatrix()
double	l2RegularizedCoefficient()
void	load(InputStream is) Load (using ObjectInputStream
double	lossFunction()
abstract double	lossFunction(Object[] params) The loss function (cross entropy, reconstruction error,...)
void	merge(NeuralNetwork network, int batchSize) Performs a network merge in the form of a += b - a / n where a is a matrix here b is a matrix on the incoming network and n is the batch size
abstract org.jblas.DoubleMatrix	reconstruct(org.jblas.DoubleMatrix x) All neural networks are based on this idea of minimizing reconstruction error.
void	setDist(org.apache.commons.math3.distribution.RealDistribution dist)
void	setFanIn(double fanIn)
void	sethBias(org.jblas.DoubleMatrix hBias)
void	setInput(org.jblas.DoubleMatrix input)
void	setL2(double l2)
void	setMomentum(double momentum)
void	setnHidden(int nHidden)
void	setnVisible(int nVisible)
void	setRenderEpochs(int renderEpochs)
void	setRng(org.apache.commons.math3.random.RandomGenerator rng)
void	setSparsity(double sparsity)
void	setvBias(org.jblas.DoubleMatrix vBias)
void	setW(org.jblas.DoubleMatrix w)
double	squaredLoss()
abstract void	train(org.jblas.DoubleMatrix input, double lr, Object[] params) Train one iteration of the network
NeuralNetwork	transpose()
void	update(BaseNeuralNetwork n) Copies params from the passed in network to this one
void	write(OutputStream os) Write this to an object output stream

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.deeplearning4j.nn.NeuralNetwork

getGradient, trainTillConvergence

Field Detail

nVisible

public int nVisible

nHidden

public int nHidden

Number of hidden units One tip with this is usually having more hidden units than inputs (read: input rows here) will typically cause terrible overfitting. Another rule worthy of note: more training data typically results in more redundant data. It is usually a better idea to use a smaller number of hidden units.

W

public org.jblas.DoubleMatrix W

hBias

public org.jblas.DoubleMatrix hBias

vBias

public org.jblas.DoubleMatrix vBias

rng

public org.apache.commons.math3.random.RandomGenerator rng

input

public org.jblas.DoubleMatrix input

sparsity

public double sparsity

momentum

public double momentum

dist

public transient org.apache.commons.math3.distribution.RealDistribution dist

l2

public double l2

optimizer

public transient NeuralNetworkOptimizer optimizer

renderWeightsEveryNumEpochs

public int renderWeightsEveryNumEpochs

fanIn

public double fanIn

useRegularization

public boolean useRegularization

Constructor Detail

BaseNeuralNetwork

public BaseNeuralNetwork()

BaseNeuralNetwork

public BaseNeuralNetwork(int nVisible,
                 int nHidden,
                 org.jblas.DoubleMatrix W,
                 org.jblas.DoubleMatrix hbias,
                 org.jblas.DoubleMatrix vbias,
                 org.apache.commons.math3.random.RandomGenerator rng,
                 double fanIn,
                 org.apache.commons.math3.distribution.RealDistribution dist)

Parameters:

nVisible - the number of outbound nodes

nHidden - the number of nodes in the hidden layer

W - the weights for this vector, maybe null, if so this will create a matrix with nHidden x nVisible dimensions.

hBias - the hidden bias

vBias - the visible bias (usually b for the output layer)

rng - the rng, if not a seed of 1234 is used.

BaseNeuralNetwork

public BaseNeuralNetwork(org.jblas.DoubleMatrix input,
                 int nVisible,
                 int nHidden,
                 org.jblas.DoubleMatrix W,
                 org.jblas.DoubleMatrix hbias,
                 org.jblas.DoubleMatrix vbias,
                 org.apache.commons.math3.random.RandomGenerator rng,
                 double fanIn,
                 org.apache.commons.math3.distribution.RealDistribution dist)

Parameters:

input - the input examples

nVisible - the number of outbound nodes

nHidden - the number of nodes in the hidden layer

W - the weights for this vector, maybe null, if so this will create a matrix with nHidden x nVisible dimensions.

hBias - the hidden bias

vBias - the visible bias (usually b for the output layer)

rng - the rng, if not a seed of 1234 is used.

Method Detail

l2RegularizedCoefficient

public double l2RegularizedCoefficient()

Specified by:

l2RegularizedCoefficient in interface NeuralNetwork

initWeights

protected void initWeights()

Initialize weights. This includes steps for doing a random initialization of W as well as the vbias and hbias

setRenderEpochs

public void setRenderEpochs(int renderEpochs)

Specified by:

setRenderEpochs in interface NeuralNetwork

getRenderEpochs

public int getRenderEpochs()

Specified by:

getRenderEpochs in interface NeuralNetwork

fanIn

public double fanIn()

Specified by:

fanIn in interface NeuralNetwork

setFanIn

public void setFanIn(double fanIn)

Specified by:

setFanIn in interface NeuralNetwork

jostleWeighMatrix

public void jostleWeighMatrix()

transpose

public NeuralNetwork transpose()

Specified by:

transpose in interface NeuralNetwork

clone

public NeuralNetwork clone()

Specified by:

clone in interface NeuralNetwork

Overrides:

clone in class Object

getDist

public org.apache.commons.math3.distribution.RealDistribution getDist()

Specified by:

getDist in interface NeuralNetwork

setDist

public void setDist(org.apache.commons.math3.distribution.RealDistribution dist)

Specified by:

setDist in interface NeuralNetwork

merge

public void merge(NeuralNetwork network,
         int batchSize)

Description copied from interface: NeuralNetwork

Performs a network merge in the form of a += b - a / n where a is a matrix here b is a matrix on the incoming network and n is the batch size

Specified by:

merge in interface NeuralNetwork

Parameters:

network - the network to merge with

batchSize - the batch size (number of training examples) to average by

update

public void update(BaseNeuralNetwork n)

Copies params from the passed in network to this one

Parameters:

n - the network to copy

load

public void load(InputStream is)

Load (using ObjectInputStream

Specified by:

load in interface Persistable

Parameters:

is - the input stream to load from (usually a file)

getReConstructionCrossEntropy

public double getReConstructionCrossEntropy()

Reconstruction entropy. This compares the similarity of two probability distributions, in this case that would be the input and the reconstructed input with gaussian noise. This will account for either regularization or none depending on the configuration.

Specified by:

getReConstructionCrossEntropy in interface NeuralNetwork

Returns:

reconstruction error

getnVisible

public int getnVisible()

Specified by:

getnVisible in interface NeuralNetwork

setnVisible

public void setnVisible(int nVisible)

Specified by:

setnVisible in interface NeuralNetwork

getnHidden

public int getnHidden()

Specified by:

getnHidden in interface NeuralNetwork

setnHidden

public void setnHidden(int nHidden)

Specified by:

setnHidden in interface NeuralNetwork

getW

public org.jblas.DoubleMatrix getW()

Specified by:

getW in interface NeuralNetwork

setW

public void setW(org.jblas.DoubleMatrix w)

Specified by:

setW in interface NeuralNetwork

gethBias

public org.jblas.DoubleMatrix gethBias()

Specified by:

gethBias in interface NeuralNetwork

sethBias

public void sethBias(org.jblas.DoubleMatrix hBias)

Specified by:

sethBias in interface NeuralNetwork

getvBias

public org.jblas.DoubleMatrix getvBias()

Specified by:

getvBias in interface NeuralNetwork

setvBias

public void setvBias(org.jblas.DoubleMatrix vBias)

Specified by:

setvBias in interface NeuralNetwork

getRng

public org.apache.commons.math3.random.RandomGenerator getRng()

Specified by:

getRng in interface NeuralNetwork

setRng

public void setRng(org.apache.commons.math3.random.RandomGenerator rng)

Specified by:

setRng in interface NeuralNetwork

getInput

public org.jblas.DoubleMatrix getInput()

Specified by:

getInput in interface NeuralNetwork

setInput

public void setInput(org.jblas.DoubleMatrix input)

Specified by:

setInput in interface NeuralNetwork

getSparsity

public double getSparsity()

Specified by:

getSparsity in interface NeuralNetwork

setSparsity

public void setSparsity(double sparsity)

Specified by:

setSparsity in interface NeuralNetwork

getMomentum

public double getMomentum()

Specified by:

getMomentum in interface NeuralNetwork

setMomentum

public void setMomentum(double momentum)

Specified by:

setMomentum in interface NeuralNetwork

getL2

public double getL2()

Specified by:

getL2 in interface NeuralNetwork

setL2

public void setL2(double l2)

Specified by:

setL2 in interface NeuralNetwork

write

public void write(OutputStream os)

Write this to an object output stream

Specified by:

write in interface Persistable

Parameters:

os - the output stream to write to

reconstruct

public abstract org.jblas.DoubleMatrix reconstruct(org.jblas.DoubleMatrix x)

All neural networks are based on this idea of minimizing reconstruction error. Both RBMs and Denoising AutoEncoders have a component for reconstructing, ala different implementations.

Parameters:

x - the input to reconstruct

Returns:

the reconstructed input

lossFunction

public abstract double lossFunction(Object[] params)

The loss function (cross entropy, reconstruction error,...)

Returns:

the loss function

lossFunction

public double lossFunction()

train

public abstract void train(org.jblas.DoubleMatrix input,
         double lr,
         Object[] params)

Train one iteration of the network

Specified by:

train in interface NeuralNetwork

Parameters:

input - the input to train on

lr - the learning rate to train at

params - the extra params (k, corruption level,...)

squaredLoss

public double squaredLoss()

Specified by:

squaredLoss in interface NeuralNetwork