org.deeplearning4j.nn

Class BaseNeuralNetwork

java.lang.Object

org.deeplearning4j.nn.BaseNeuralNetwork

All Implemented Interfaces:

Serializable, Cloneable, Model, NeuralNetwork, Persistable, IterationListener

Direct Known Subclasses:

AutoEncoder, DenoisingAutoEncoder, RBM

public abstract class BaseNeuralNetwork
extends Object
implements NeuralNetwork, Persistable

Baseline class for any Neural Network used as a layer in a deep network such as an DBN

Author:

Adam Gibson

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BaseNeuralNetwork.Builder<E extends BaseNeuralNetwork>

Nested classes/interfaces inherited from interface org.deeplearning4j.nn.api.NeuralNetwork

NeuralNetwork.OptimizationAlgorithm

Field Summary

Fields

Modifier and Type	Field and Description
protected NeuralNetConfiguration	conf
protected org.nd4j.linalg.api.ndarray.INDArray	doMask
protected org.nd4j.linalg.api.ndarray.INDArray	hBias
protected org.nd4j.linalg.learning.AdaGrad	hBiasAdaGrad
protected org.nd4j.linalg.api.ndarray.INDArray	hBiasGradient
protected org.nd4j.linalg.api.ndarray.INDArray	input
protected int	lastMiniBatchSize
protected NeuralNetworkOptimizer	optimizer
protected org.nd4j.linalg.api.ndarray.INDArray	vBias
protected org.nd4j.linalg.learning.AdaGrad	vBiasAdaGrad
protected org.nd4j.linalg.api.ndarray.INDArray	vBiasGradient
protected org.nd4j.linalg.api.ndarray.INDArray	W
protected org.nd4j.linalg.learning.AdaGrad	wAdaGrad
protected org.nd4j.linalg.api.ndarray.INDArray	wGradient

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	BaseNeuralNetwork()
	BaseNeuralNetwork(org.nd4j.linalg.api.ndarray.INDArray input, org.nd4j.linalg.api.ndarray.INDArray W, org.nd4j.linalg.api.ndarray.INDArray hbias, org.nd4j.linalg.api.ndarray.INDArray vbias, NeuralNetConfiguration conf)

Method Summary

Modifier and Type	Method and Description
protected void	applyDropOutIfNecessary(org.nd4j.linalg.api.ndarray.INDArray input)
protected void	applySparsity(org.nd4j.linalg.api.ndarray.INDArray hBiasGradient) Applies sparsity to the passed in hbias gradient
void	backProp(double lr, int iterations, Object[] extraParams) Backprop with the output being the reconstruction
void	clearInput() Clears the input from the neural net
NeuralNetwork	clone()
NeuralNetConfiguration	conf()
void	fit(org.nd4j.linalg.api.ndarray.INDArray data) Fit the model to the given data
org.nd4j.linalg.learning.AdaGrad	getAdaGrad()
org.nd4j.linalg.api.ndarray.INDArray	gethBias()
org.nd4j.linalg.learning.AdaGrad	gethBiasAdaGrad()
org.nd4j.linalg.api.ndarray.INDArray	getInput()
org.nd4j.linalg.api.ndarray.INDArray	getvBias()
org.nd4j.linalg.learning.AdaGrad	getVBiasAdaGrad()
org.nd4j.linalg.api.ndarray.INDArray	getW()
org.nd4j.linalg.api.ndarray.INDArray	hBiasMean()
protected void	initWeights() Initialize weights.
void	iterationDone(int iteration) Event listener for each iteration
double	l2RegularizedCoefficient()
void	load(InputStream is) Load (using ObjectInputStream
void	merge(NeuralNetwork network, int batchSize) Performs a network merge in the form of a += b - a / n where a is a matrix here b is a matrix on the incoming network and n is the batch size
int	numParams() The number of parameters for the model
org.nd4j.linalg.api.ndarray.INDArray	params() Returns the parameters of the neural network
protected org.nd4j.linalg.api.ndarray.INDArray	preProcessInput(org.nd4j.linalg.api.ndarray.INDArray input)
double	score()
void	setAdaGrad(org.nd4j.linalg.learning.AdaGrad adaGrad)
void	setConf(NeuralNetConfiguration conf)
void	sethBias(org.nd4j.linalg.api.ndarray.INDArray hBias)
void	setHbiasAdaGrad(org.nd4j.linalg.learning.AdaGrad adaGrad)
void	setInput(org.nd4j.linalg.api.ndarray.INDArray input)
void	setParams(org.nd4j.linalg.api.ndarray.INDArray params) Set the parameters for this model.
void	setvBias(org.nd4j.linalg.api.ndarray.INDArray vBias)
void	setVBiasAdaGrad(org.nd4j.linalg.learning.AdaGrad adaGrad)
void	setW(org.nd4j.linalg.api.ndarray.INDArray w)
abstract org.nd4j.linalg.api.ndarray.INDArray	transform(org.nd4j.linalg.api.ndarray.INDArray x) All neural networks are based on this idea of minimizing reconstruction error.
NeuralNetwork	transpose()
void	update(BaseNeuralNetwork n) Copies params from the passed in network to this one
protected void	updateGradientAccordingToParams(NeuralNetworkGradient gradient, int iteration, double learningRate) Update the gradient according to the configuration such as adagrad, momentum, and sparsity
void	write(OutputStream os) Write this to an object output stream

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.deeplearning4j.nn.api.NeuralNetwork

getGradient, hiddenActivation, sampleHiddenGivenVisible, sampleVisibleGivenHidden

Methods inherited from interface org.deeplearning4j.nn.api.Model

fit, iterate

Field Detail

W

protected org.nd4j.linalg.api.ndarray.INDArray W

hBias

protected org.nd4j.linalg.api.ndarray.INDArray hBias

vBias

protected org.nd4j.linalg.api.ndarray.INDArray vBias

input

protected org.nd4j.linalg.api.ndarray.INDArray input

optimizer

protected transient NeuralNetworkOptimizer optimizer

doMask

protected org.nd4j.linalg.api.ndarray.INDArray doMask

wGradient

protected org.nd4j.linalg.api.ndarray.INDArray wGradient

vBiasGradient

protected org.nd4j.linalg.api.ndarray.INDArray vBiasGradient

hBiasGradient

protected org.nd4j.linalg.api.ndarray.INDArray hBiasGradient

lastMiniBatchSize

protected int lastMiniBatchSize

wAdaGrad

protected org.nd4j.linalg.learning.AdaGrad wAdaGrad

hBiasAdaGrad

protected org.nd4j.linalg.learning.AdaGrad hBiasAdaGrad

vBiasAdaGrad

protected org.nd4j.linalg.learning.AdaGrad vBiasAdaGrad

conf

protected NeuralNetConfiguration conf

Constructor Detail

BaseNeuralNetwork

protected BaseNeuralNetwork()

BaseNeuralNetwork

public BaseNeuralNetwork(org.nd4j.linalg.api.ndarray.INDArray input,
                         org.nd4j.linalg.api.ndarray.INDArray W,
                         org.nd4j.linalg.api.ndarray.INDArray hbias,
                         org.nd4j.linalg.api.ndarray.INDArray vbias,
                         NeuralNetConfiguration conf)

Method Detail

params

public org.nd4j.linalg.api.ndarray.INDArray params()

Returns the parameters of the neural network

Specified by:

params in interface Model

Returns:

the parameters of the neural network

l2RegularizedCoefficient

public double l2RegularizedCoefficient()

initWeights

protected void initWeights()

Initialize weights. This includes steps for doing a random initialization of W as well as the vbias and hbias

numParams

public int numParams()

The number of parameters for the model

Specified by:

numParams in interface Model

Returns:

the number of parameters for the model

setParams

public void setParams(org.nd4j.linalg.api.ndarray.INDArray params)

Set the parameters for this model. This expects a linear ndarray which then be unpacked internally relative to the expected ordering of the model

Specified by:

setParams in interface Model

Parameters:

params - the parameters for the model

backProp

public void backProp(double lr,
                     int iterations,
                     Object[] extraParams)

Backprop with the output being the reconstruction

Specified by:

backProp in interface NeuralNetwork

Parameters:

lr - the learning rate to use

iterations - the max number of epochs to run

extraParams - implementation specific params

fit

public void fit(org.nd4j.linalg.api.ndarray.INDArray data)

Fit the model to the given data

Specified by:

fit in interface Model

Parameters:

data - the data to fit the model to

applySparsity

protected void applySparsity(org.nd4j.linalg.api.ndarray.INDArray hBiasGradient)

Applies sparsity to the passed in hbias gradient

Parameters:

hBiasGradient - the hbias gradient to apply to

updateGradientAccordingToParams

protected void updateGradientAccordingToParams(NeuralNetworkGradient gradient,
                                               int iteration,
                                               double learningRate)

Update the gradient according to the configuration such as adagrad, momentum, and sparsity

Parameters:

gradient - the gradient to modify

iteration - the current iteration

learningRate - the learning rate for the current iteration

score

public double score()

Specified by:

score in interface Model

clearInput

public void clearInput()

Clears the input from the neural net

Specified by:

clearInput in interface NeuralNetwork

getAdaGrad

public org.nd4j.linalg.learning.AdaGrad getAdaGrad()

Specified by:

getAdaGrad in interface NeuralNetwork

setAdaGrad

public void setAdaGrad(org.nd4j.linalg.learning.AdaGrad adaGrad)

Specified by:

setAdaGrad in interface NeuralNetwork

transpose

public NeuralNetwork transpose()

Specified by:

transpose in interface NeuralNetwork

conf

public NeuralNetConfiguration conf()

Specified by:

conf in interface NeuralNetwork

setConf

public void setConf(NeuralNetConfiguration conf)

Specified by:

setConf in interface NeuralNetwork

clone

public NeuralNetwork clone()

Specified by:

clone in interface NeuralNetwork

Overrides:

clone in class Object

merge

public void merge(NeuralNetwork network,
                  int batchSize)

Description copied from interface: NeuralNetwork

Performs a network merge in the form of a += b - a / n where a is a matrix here b is a matrix on the incoming network and n is the batch size

Specified by:

merge in interface NeuralNetwork

Parameters:

network - the network to merge with

batchSize - the batch size (number of training examples) to average by

update

public void update(BaseNeuralNetwork n)

Copies params from the passed in network to this one

Parameters:

n - the network to copy

load

public void load(InputStream is)

Load (using ObjectInputStream

Specified by:

load in interface Persistable

Parameters:

is - the input stream to load from (usually a file)

getW

public org.nd4j.linalg.api.ndarray.INDArray getW()

Specified by:

getW in interface NeuralNetwork

setW

public void setW(org.nd4j.linalg.api.ndarray.INDArray w)

Specified by:

setW in interface NeuralNetwork

gethBias

public org.nd4j.linalg.api.ndarray.INDArray gethBias()

Specified by:

gethBias in interface NeuralNetwork

sethBias

public void sethBias(org.nd4j.linalg.api.ndarray.INDArray hBias)

Specified by:

sethBias in interface NeuralNetwork

getvBias

public org.nd4j.linalg.api.ndarray.INDArray getvBias()

Specified by:

getvBias in interface NeuralNetwork

setvBias

public void setvBias(org.nd4j.linalg.api.ndarray.INDArray vBias)

Specified by:

setvBias in interface NeuralNetwork

getInput

public org.nd4j.linalg.api.ndarray.INDArray getInput()

Specified by:

getInput in interface NeuralNetwork

setInput

public void setInput(org.nd4j.linalg.api.ndarray.INDArray input)

Specified by:

setInput in interface NeuralNetwork

gethBiasAdaGrad

public org.nd4j.linalg.learning.AdaGrad gethBiasAdaGrad()

Specified by:

gethBiasAdaGrad in interface NeuralNetwork

setHbiasAdaGrad

public void setHbiasAdaGrad(org.nd4j.linalg.learning.AdaGrad adaGrad)

Specified by:

setHbiasAdaGrad in interface NeuralNetwork

getVBiasAdaGrad

public org.nd4j.linalg.learning.AdaGrad getVBiasAdaGrad()

Specified by:

getVBiasAdaGrad in interface NeuralNetwork

setVBiasAdaGrad

public void setVBiasAdaGrad(org.nd4j.linalg.learning.AdaGrad adaGrad)

Specified by:

setVBiasAdaGrad in interface NeuralNetwork

write

public void write(OutputStream os)

Write this to an object output stream

Specified by:

write in interface Persistable

Parameters:

os - the output stream to write to

transform

public abstract org.nd4j.linalg.api.ndarray.INDArray transform(org.nd4j.linalg.api.ndarray.INDArray x)

All neural networks are based on this idea of minimizing reconstruction error. Both RBMs and Denoising AutoEncoders have a component for reconstructing, ala different implementations.

Specified by:

transform in interface Model

Parameters:

x - the input to transform

Returns:

the reconstructed input

applyDropOutIfNecessary

protected void applyDropOutIfNecessary(org.nd4j.linalg.api.ndarray.INDArray input)

hBiasMean

public org.nd4j.linalg.api.ndarray.INDArray hBiasMean()

Specified by:

hBiasMean in interface NeuralNetwork

preProcessInput

protected org.nd4j.linalg.api.ndarray.INDArray preProcessInput(org.nd4j.linalg.api.ndarray.INDArray input)

iterationDone

public void iterationDone(int iteration)

Description copied from interface: IterationListener

Event listener for each iteration

Specified by:

iterationDone in interface NeuralNetwork

Specified by:

iterationDone in interface IterationListener