org.deeplearning4j.nn.layers

Class BaseLayer

java.lang.Object

org.deeplearning4j.nn.layers.BaseLayer

All Implemented Interfaces:

Serializable, Cloneable, Layer, Model

Direct Known Subclasses:

BasePretrainNetwork, ConvolutionDownSampleLayer, LSTM, OutputLayer, RecursiveAutoEncoder

public abstract class BaseLayer
extends Object
implements Layer

A layer with a bias and activation function

Author:

Adam Gibson

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.deeplearning4j.nn.api.Layer

Layer.Type

Field Summary

Fields

Modifier and Type	Field and Description
protected NeuralNetConfiguration	conf
protected org.nd4j.linalg.api.ndarray.INDArray	dropoutMask
protected org.nd4j.linalg.api.ndarray.INDArray	input
protected Collection<IterationListener>	iterationListeners
protected ConvexOptimizer	optimizer
protected ParamInitializer	paramInitializer
protected Map<String,org.nd4j.linalg.api.ndarray.INDArray>	params
protected double	score

Constructor Summary

Constructors

Constructor and Description
BaseLayer(NeuralNetConfiguration conf)
BaseLayer(NeuralNetConfiguration conf, org.nd4j.linalg.api.ndarray.INDArray input)

Method Summary

Modifier and Type	Method and Description
void	accumulateScore(double accum) Sets a rolling tally for the score.
org.nd4j.linalg.api.ndarray.INDArray	activate() Trigger an activation with the last specified input
org.nd4j.linalg.api.ndarray.INDArray	activate(org.nd4j.linalg.api.ndarray.INDArray input) Initialize the layer with the given input and return the activation for this layer given this input
org.nd4j.linalg.api.ndarray.INDArray	activationMean() Calculate the mean representation for the activation for this layer
protected void	applyDropOutIfNecessary(org.nd4j.linalg.api.ndarray.INDArray input)
Pair<Gradient,Gradient>	backWard(Gradient ixes, Gradient deltas, org.nd4j.linalg.api.ndarray.INDArray activation, String previousActivation) Propagate errors backwards for a particular layer.
Gradient	backwardGradient(org.nd4j.linalg.api.ndarray.INDArray activation, Gradient errorSignal) Calculate the gradient relative to the error in the next layer
int	batchSize() The current inputs batch size
Gradient	calcGradient(Gradient layerError, org.nd4j.linalg.api.ndarray.INDArray activation) Calculate the gradient
void	clear() Clear input
Layer	clone() Clone the layer
NeuralNetConfiguration	conf() The configuration for the neural network
protected Gradient	createGradient(org.nd4j.linalg.api.ndarray.INDArray... gradients) Create a gradient list based on the passed in parameters.
org.nd4j.linalg.api.ndarray.INDArray	derivativeActivation(org.nd4j.linalg.api.ndarray.INDArray input) Take the derivative of the given input based on the activation
Gradient	error(org.nd4j.linalg.api.ndarray.INDArray errorSignal) Calculate error with respect to the current layer.
Gradient	errorSignal(Gradient error, org.nd4j.linalg.api.ndarray.INDArray input) Error signal for this layer Using the amount of error caused by this layer calculate the error signal used as input in to the next layer.
void	fit() All models have a fit method
void	fit(org.nd4j.linalg.api.ndarray.INDArray input) Fit the model to the given data
Collection<IterationListener>	getIterationListeners() Get the iteration listeners for this layer.
ConvexOptimizer	getOptimizer() Returns this models optimizer
org.nd4j.linalg.api.ndarray.INDArray	getParam(String param) Get the parameter
Pair<Gradient,Double>	gradientAndScore() Get the gradient and score
void	initParams() Initialize the parameters
org.nd4j.linalg.api.ndarray.INDArray	input() The input/feature matrix for the model
void	iterate(org.nd4j.linalg.api.ndarray.INDArray input) iterate one iteration of the network
void	merge(Layer l, int batchSize) Averages the given logistic regression from a mini batch in to this one
int	numParams() The number of parameters for the model
org.nd4j.linalg.api.ndarray.INDArray	params() Returns the parameters of the neural network
Map<String,org.nd4j.linalg.api.ndarray.INDArray>	paramTable() The param table
org.nd4j.linalg.api.ndarray.INDArray	preOutput(org.nd4j.linalg.api.ndarray.INDArray x) Classify input
double	score() The score for the model
void	setConf(NeuralNetConfiguration conf) Setter for the configuration
void	setIterationListeners(Collection<IterationListener> listeners) Set the iteration listeners for this layer.
void	setParam(String key, org.nd4j.linalg.api.ndarray.INDArray val) Set the parameter with a new ndarray
void	setParams(org.nd4j.linalg.api.ndarray.INDArray params) Set the parameters for this model.
void	setParamTable(Map<String,org.nd4j.linalg.api.ndarray.INDArray> paramTable) Setter for the param table
void	setScore() Update the score
Layer	transpose() Return a transposed copy of the weights/bias (this means reverse the number of inputs and outputs on the weights)
Layer.Type	type() Returns the layer type
void	update(Gradient gradient) Perform one update applying the gradient
void	validateInput() Validate the input

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.deeplearning4j.nn.api.Model

gradient, transform

Field Detail

input

protected org.nd4j.linalg.api.ndarray.INDArray input

params

protected Map<String,org.nd4j.linalg.api.ndarray.INDArray> params

conf

protected NeuralNetConfiguration conf

dropoutMask

protected org.nd4j.linalg.api.ndarray.INDArray dropoutMask

paramInitializer

protected ParamInitializer paramInitializer

score

protected double score

optimizer

protected ConvexOptimizer optimizer

iterationListeners

protected Collection<IterationListener> iterationListeners

Constructor Detail

BaseLayer

public BaseLayer(NeuralNetConfiguration conf)

BaseLayer

public BaseLayer(NeuralNetConfiguration conf,
                 org.nd4j.linalg.api.ndarray.INDArray input)

Method Detail

getIterationListeners

public Collection<IterationListener> getIterationListeners()

Description copied from interface: Layer

Get the iteration listeners for this layer.

Specified by:

getIterationListeners in interface Layer

setIterationListeners

public void setIterationListeners(Collection<IterationListener> listeners)

Description copied from interface: Layer

Set the iteration listeners for this layer.

Specified by:

setIterationListeners in interface Layer

error

public Gradient error(org.nd4j.linalg.api.ndarray.INDArray errorSignal)

Description copied from interface: Layer

Calculate error with respect to the current layer. This gradient will contain the error signal

Specified by:

error in interface Layer

Parameters:

errorSignal - the gradient for the forward layer If this is the final layer, it will start with the error from the output. This is on the user to initialize.

Returns:

the gradient wrt the parameters on the current layer

derivativeActivation

public org.nd4j.linalg.api.ndarray.INDArray derivativeActivation(org.nd4j.linalg.api.ndarray.INDArray input)

Description copied from interface: Layer

Take the derivative of the given input based on the activation

Specified by:

derivativeActivation in interface Layer

Parameters:

input - the input to take the derivative of

Returns:

the derivative of the action

errorSignal

public Gradient errorSignal(Gradient error,
                            org.nd4j.linalg.api.ndarray.INDArray input)

Description copied from interface: Layer

Error signal for this layer Using the amount of error caused by this layer calculate the error signal used as input in to the next layer. This is used for actually calculating the gradient of the layer

Specified by:

errorSignal in interface Layer

Returns:

calcGradient

public Gradient calcGradient(Gradient layerError,
                             org.nd4j.linalg.api.ndarray.INDArray activation)

Description copied from interface: Layer

Calculate the gradient

Specified by:

calcGradient in interface Layer

Parameters:

layerError - the layer error

Returns:

the gradient

backwardGradient

public Gradient backwardGradient(org.nd4j.linalg.api.ndarray.INDArray activation,
                                 Gradient errorSignal)

Description copied from interface: Layer

Calculate the gradient relative to the error in the next layer

Specified by:

backwardGradient in interface Layer

Parameters:

activation - the activation from the network

errorSignal - the error signal caused by this network.

Returns:

fit

public void fit()

Description copied from interface: Model

All models have a fit method

Specified by:

fit in interface Model

setScore

public void setScore()

Description copied from interface: Model

Update the score

Specified by:

setScore in interface Model

iterate

public void iterate(org.nd4j.linalg.api.ndarray.INDArray input)

iterate one iteration of the network

Specified by:

iterate in interface Model

Parameters:

input - the input to iterate on

update

public void update(Gradient gradient)

Description copied from interface: Model

Perform one update applying the gradient

Specified by:

update in interface Model

Parameters:

gradient - the gradient to apply

getOptimizer

public ConvexOptimizer getOptimizer()

Description copied from interface: Model

Returns this models optimizer

Specified by:

getOptimizer in interface Model

Returns:

this models optimizer

setConf

public void setConf(NeuralNetConfiguration conf)

Description copied from interface: Model

Setter for the configuration

Specified by:

setConf in interface Model

setParam

public void setParam(String key,
                     org.nd4j.linalg.api.ndarray.INDArray val)

Description copied from interface: Model

Set the parameter with a new ndarray

Specified by:

setParam in interface Model

Parameters:

key - the key to se t

val - the new ndarray

params

public org.nd4j.linalg.api.ndarray.INDArray params()

Returns the parameters of the neural network

Specified by:

params in interface Model

Returns:

the parameters of the neural network

setParams

public void setParams(org.nd4j.linalg.api.ndarray.INDArray params)

Description copied from interface: Model

Set the parameters for this model. This expects a linear ndarray which then be unpacked internally relative to the expected ordering of the model

Specified by:

setParams in interface Model

Parameters:

params - the parameters for the model

initParams

public void initParams()

Description copied from interface: Model

Initialize the parameters

Specified by:

initParams in interface Model

paramTable

public Map<String,org.nd4j.linalg.api.ndarray.INDArray> paramTable()

Description copied from interface: Model

The param table

Specified by:

paramTable in interface Model

Returns:

setParamTable

public void setParamTable(Map<String,org.nd4j.linalg.api.ndarray.INDArray> paramTable)

Description copied from interface: Model

Setter for the param table

Specified by:

setParamTable in interface Model

getParam

public org.nd4j.linalg.api.ndarray.INDArray getParam(String param)

Description copied from interface: Model

Get the parameter

Specified by:

getParam in interface Model

Parameters:

param - the key of the parameter

Returns:

the parameter vector/matrix with that particular key

preOutput

public org.nd4j.linalg.api.ndarray.INDArray preOutput(org.nd4j.linalg.api.ndarray.INDArray x)

Classify input

Specified by:

preOutput in interface Layer

Parameters:

x - the input (can either be a matrix or vector) If it's a matrix, each row is considered an example and associated rows are classified accordingly. Each row will be the likelihood of a label given that example

Returns:

a probability distribution for each row

batchSize

public int batchSize()

Description copied from interface: Model

The current inputs batch size

Specified by:

batchSize in interface Model

Returns:

the current inputs batch size

activate

public org.nd4j.linalg.api.ndarray.INDArray activate()

Description copied from interface: Layer

Trigger an activation with the last specified input

Specified by:

activate in interface Layer

Returns:

the activation of the last specified input

activate

public org.nd4j.linalg.api.ndarray.INDArray activate(org.nd4j.linalg.api.ndarray.INDArray input)

Description copied from interface: Layer

Initialize the layer with the given input and return the activation for this layer given this input

Specified by:

activate in interface Layer

Parameters:

input - the input to use

Returns:

activationMean

public org.nd4j.linalg.api.ndarray.INDArray activationMean()

Description copied from interface: Layer

Calculate the mean representation for the activation for this layer

Specified by:

activationMean in interface Layer

Returns:

the activation mean for this layer

conf

public NeuralNetConfiguration conf()

Description copied from interface: Model

The configuration for the neural network

Specified by:

conf in interface Model

Returns:

the configuration for the neural network

clear

public void clear()

Description copied from interface: Model

Clear input

Specified by:

clear in interface Model

applyDropOutIfNecessary

protected void applyDropOutIfNecessary(org.nd4j.linalg.api.ndarray.INDArray input)

merge

public void merge(Layer l,
                  int batchSize)

Averages the given logistic regression from a mini batch in to this one

Specified by:

merge in interface Layer

Parameters:

l - the logistic regression to average in to this one

batchSize - the batch size

clone

public Layer clone()

Description copied from interface: Layer

Clone the layer

Specified by:

clone in interface Layer

Overrides:

clone in class Object

Returns:

type

public Layer.Type type()

Description copied from interface: Layer

Returns the layer type

Specified by:

type in interface Layer

Returns:

numParams

public int numParams()

The number of parameters for the model

Specified by:

numParams in interface Model

Returns:

the number of parameters for the model

fit

public void fit(org.nd4j.linalg.api.ndarray.INDArray input)

Description copied from interface: Model

Fit the model to the given data

Specified by:

fit in interface Model

Parameters:

input - the data to fit the model to

gradientAndScore

public Pair<Gradient,Double> gradientAndScore()

Description copied from interface: Model

Get the gradient and score

Specified by:

gradientAndScore in interface Model

Returns:

the gradient and score

score

public double score()

Description copied from interface: Model

The score for the model

Specified by:

score in interface Model

Returns:

the score for the model

input

public org.nd4j.linalg.api.ndarray.INDArray input()

Description copied from interface: Model

The input/feature matrix for the model

Specified by:

input in interface Model

Returns:

the input/feature matrix for the model

validateInput

public void validateInput()

Description copied from interface: Model

Validate the input

Specified by:

validateInput in interface Model

createGradient

protected Gradient createGradient(org.nd4j.linalg.api.ndarray.INDArray... gradients)

Create a gradient list based on the passed in parameters. Will throw an IllegalArgumentException if the number of gradient matrices isn't equal to the number of keys in the parameter list

Parameters:

gradients - the gradients to create from

Returns:

the create based on the passed in ndarrays

transpose

public Layer transpose()

Description copied from interface: Layer

Return a transposed copy of the weights/bias (this means reverse the number of inputs and outputs on the weights)

Specified by:

transpose in interface Layer

Returns:

the transposed layer

backWard

public Pair<Gradient,Gradient> backWard(Gradient ixes,
                                        Gradient deltas,
                                        org.nd4j.linalg.api.ndarray.INDArray activation,
                                        String previousActivation)

Description copied from interface: Layer

Propagate errors backwards for a particular layer. This calculates the node error for a given true output.

Specified by:

backWard in interface Layer

Parameters:

ixes - the errors to propagate

deltas - the previous deltas

activation - the activation from the previous layer

previousActivation - the previous layer's activation

accumulateScore

public void accumulateScore(double accum)

Description copied from interface: Model

Sets a rolling tally for the score. This is useful for mini batch learning when you are accumulating error across a dataset.

Specified by:

accumulateScore in interface Model

Parameters:

accum - the amount to accum