org.deeplearning4j.nn.api

Interface Layer

All Superinterfaces:

Cloneable, Model, Serializable, Trainable

All Known Subinterfaces:

IOutputLayer, RecurrentLayer

All Known Implementing Classes:

AbstractLayer, ActivationLayer, AutoEncoder, BaseLayer, BaseOutputLayer, BasePretrainNetwork, BaseRecurrentLayer, BaseWrapperLayer, BatchNormalization, BidirectionalLayer, CenterLossOutputLayer, Cnn3DLossLayer, CnnLossLayer, Convolution1DLayer, Convolution3DLayer, ConvolutionLayer, Cropping1DLayer, Cropping2DLayer, Cropping3DLayer, Deconvolution2DLayer, Deconvolution3DLayer, DenseLayer, DepthwiseConvolution2DLayer, DropoutLayer, ElementWiseMultiplicationLayer, EmbeddingLayer, EmbeddingSequenceLayer, FrozenLayer, FrozenLayerWithBackprop, GlobalPoolingLayer, GravesBidirectionalLSTM, GravesLSTM, LastTimeStepLayer, LocalResponseNormalization, LossLayer, LSTM, MaskLayer, MaskZeroLayer, MultiLayerNetwork, OCNNOutputLayer, OutputLayer, PReLU, RepeatVector, RnnLossLayer, RnnOutputLayer, SameDiffLayer, SameDiffOutputLayer, SeparableConvolution2DLayer, SimpleRnn, SpaceToBatch, SpaceToDepth, Subsampling1DLayer, Subsampling3DLayer, SubsamplingLayer, TimeDistributedLayer, Upsampling1D, Upsampling2D, Upsampling3D, VariationalAutoencoder, Yolo2OutputLayer, ZeroPadding1DLayer, ZeroPadding3DLayer, ZeroPaddingLayer

public interface Layer
extends Serializable, Cloneable, Model, Trainable

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	Layer.TrainingMode
static class	Layer.Type

Method Summary

Modifier and Type	Method and Description
INDArray	activate(boolean training, LayerWorkspaceMgr workspaceMgr) Perform forward pass and return the activations array with the last set input
INDArray	activate(INDArray input, boolean training, LayerWorkspaceMgr mgr) Perform forward pass and return the activations array with the specified input
void	allowInputModification(boolean allow) A performance optimization: mark whether the layer is allowed to modify its input array in-place.
Pair<Gradient,INDArray>	backpropGradient(INDArray epsilon, LayerWorkspaceMgr workspaceMgr) Calculate the gradient relative to the error in the next layer
double	calcRegularizationScore(boolean backpropOnlyParams) Calculate the regularization component of the score, for the parameters in this layer For example, the L1, L2 and/or weight decay components of the loss function
void	clearNoiseWeightParams()
Pair<INDArray,MaskState>	feedForwardMaskArray(INDArray maskArray, MaskState currentMaskState, int minibatchSize) Feed forward the input mask array, setting in the layer as appropriate.
int	getEpochCount()
LayerHelper	getHelper()
int	getIndex() Get the layer index.
int	getInputMiniBatchSize() Get current/last input mini-batch size, as set by setInputMiniBatchSize(int)
int	getIterationCount()
Collection<TrainingListener>	getListeners() Get the iteration listeners for this layer.
INDArray	getMaskArray()
boolean	isPretrainLayer() Returns true if the layer can be trained in an unsupervised/pretrain manner (AE, VAE, etc)
void	setCacheMode(CacheMode mode) This method sets given CacheMode for current layer
void	setEpochCount(int epochCount) Set the current epoch count (number of epochs passed ) for the layer/network
void	setIndex(int index) Set the layer index.
void	setInput(INDArray input, LayerWorkspaceMgr workspaceMgr) Set the layer input.
void	setInputMiniBatchSize(int size) Set current/last input mini-batch size. Used for score and gradient calculations.
void	setIterationCount(int iterationCount) Set the current iteration count (number of parameter updates) for the layer/network
void	setListeners(Collection<TrainingListener> listeners) Set the TrainingListeners for this model.
void	setListeners(TrainingListener... listeners) Set the TrainingListeners for this model.
void	setMaskArray(INDArray maskArray) Set the mask array.
Layer.Type	type() Returns the layer type

Methods inherited from interface org.deeplearning4j.nn.api.Model

addListeners, applyConstraints, batchSize, clear, close, computeGradientAndScore, conf, fit, fit, getGradientsViewArray, getOptimizer, getParam, gradient, gradientAndScore, init, input, numParams, numParams, params, paramTable, paramTable, score, setBackpropGradientsViewArray, setConf, setParam, setParams, setParamsViewArray, setParamTable, update, update

Methods inherited from interface org.deeplearning4j.nn.api.Trainable

getConfig, getGradientsViewArray, numParams, params, paramTable, updaterDivideByMinibatch

Method Detail

setCacheMode

void setCacheMode(CacheMode mode)

This method sets given CacheMode for current layer

Parameters:

mode -

calcRegularizationScore

double calcRegularizationScore(boolean backpropOnlyParams)

Calculate the regularization component of the score, for the parameters in this layer
For example, the L1, L2 and/or weight decay components of the loss function

Parameters:

backpropOnlyParams - If true: calculate regularization score based on backprop params only. If false: calculate based on all params (including pretrain params, if any)

Returns:

the regularization score of

type

Layer.Type type()

Returns the layer type

Returns:

backpropGradient

Pair<Gradient,INDArray> backpropGradient(INDArray epsilon,
                                         LayerWorkspaceMgr workspaceMgr)

Calculate the gradient relative to the error in the next layer

Parameters:

epsilon - w^(L+1)*delta^(L+1). Or, equiv: dC/da, i.e., (dC/dz)*(dz/da) = dC/da, where C is cost function a=sigma(z) is activation.

workspaceMgr - Workspace manager

Returns:

Pair where Gradient is gradient for this layer, INDArray is epsilon (activation gradient) needed by next layer, but before element-wise multiply by sigmaPrime(z). So for standard feed-forward layer, if this layer is L, then return.getSecond() == dL/dIn = (w^(L)*(delta^(L))^T)^T. Note that the returned array should be placed in the ArrayType.ACTIVATION_GRAD workspace via the workspace manager

activate

INDArray activate(boolean training,
                  LayerWorkspaceMgr workspaceMgr)

Perform forward pass and return the activations array with the last set input

Parameters:

training - training or test mode

workspaceMgr - Workspace manager

Returns:

the activation (layer output) of the last specified input. Note that the returned array should be placed in the ArrayType.ACTIVATIONS workspace via the workspace manager

activate

INDArray activate(INDArray input,
                  boolean training,
                  LayerWorkspaceMgr mgr)

Perform forward pass and return the activations array with the specified input

Parameters:

input - the input to use

training - train or test mode

mgr - Workspace manager.

Returns:

Activations array. Note that the returned array should be placed in the ArrayType.ACTIVATIONS workspace via the workspace manager

getListeners

Collection<TrainingListener> getListeners()

Get the iteration listeners for this layer.

setListeners

void setListeners(TrainingListener... listeners)

Set the TrainingListeners for this model. If any listeners have previously been set, they will be replaced by this method

Specified by:

setListeners in interface Model

setListeners

void setListeners(Collection<TrainingListener> listeners)

Set the TrainingListeners for this model. If any listeners have previously been set, they will be replaced by this method

Specified by:

setListeners in interface Model

setIndex

void setIndex(int index)

Set the layer index.

getIndex

int getIndex()

Get the layer index.

getIterationCount

int getIterationCount()

Returns:

The current iteration count (number of parameter updates) for the layer/network

getEpochCount

int getEpochCount()

Returns:

The current epoch count (number of training epochs passed) for the layer/network

setIterationCount

void setIterationCount(int iterationCount)

Set the current iteration count (number of parameter updates) for the layer/network

setEpochCount

void setEpochCount(int epochCount)

Set the current epoch count (number of epochs passed ) for the layer/network

setInput

void setInput(INDArray input,
              LayerWorkspaceMgr workspaceMgr)

Set the layer input.

setInputMiniBatchSize

void setInputMiniBatchSize(int size)

Set current/last input mini-batch size.
Used for score and gradient calculations. Mini batch size may be different from getInput().size(0) due to reshaping operations - for example, when using RNNs with DenseLayer and OutputLayer. Called automatically during forward pass.

getInputMiniBatchSize

int getInputMiniBatchSize()

Get current/last input mini-batch size, as set by setInputMiniBatchSize(int)

See Also:

setInputMiniBatchSize(int)

setMaskArray

void setMaskArray(INDArray maskArray)

Set the mask array. Note: In general, feedForwardMaskArray(INDArray, MaskState, int) should be used in preference to this.

Parameters:

maskArray - Mask array to set

getMaskArray

INDArray getMaskArray()

isPretrainLayer

boolean isPretrainLayer()

Returns true if the layer can be trained in an unsupervised/pretrain manner (AE, VAE, etc)

Returns:

true if the layer can be pretrained (using fit(INDArray), false otherwise

clearNoiseWeightParams

void clearNoiseWeightParams()

allowInputModification

void allowInputModification(boolean allow)

A performance optimization: mark whether the layer is allowed to modify its input array in-place. In many cases, this is totally safe - in others, the input array will be shared by multiple layers, and hence it's not safe to modify the input array. This is usually used by ops such as dropout.

Parameters:

allow - If true: the input array is safe to modify. If false: the input array should be copied before it is modified (i.e., in-place modifications are un-safe)

feedForwardMaskArray

Pair<INDArray,MaskState> feedForwardMaskArray(INDArray maskArray,
                                              MaskState currentMaskState,
                                              int minibatchSize)

Feed forward the input mask array, setting in the layer as appropriate. This allows different layers to handle masks differently - for example, bidirectional RNNs and normal RNNs operate differently with masks (the former sets activations to 0 outside of the data present region (and keeps the mask active for future layers like dense layers), whereas normal RNNs don't zero out the activations/errors )instead relying on backpropagated error arrays to handle the variable length case.
This is also used for example for networks that contain global pooling layers, arbitrary preprocessors, etc.

Parameters:

maskArray - Mask array to set

currentMaskState - Current state of the mask - see MaskState

minibatchSize - Current minibatch size. Needs to be known as it cannot always be inferred from the activations array due to reshaping (such as a DenseLayer within a recurrent neural network)

Returns:

New mask array after this layer, along with the new mask state.

getHelper

LayerHelper getHelper()

Returns:

Get the layer helper, if any