BaseLayer (deeplearning4j-nn 1.0.0-beta API)

java.lang.Object
- org.deeplearning4j.nn.layers.AbstractLayer<LayerConfT>
- - org.deeplearning4j.nn.layers.BaseLayer<LayerConfT>

All Implemented Interfaces:

Serializable, Cloneable, Layer, Model

Direct Known Subclasses:

BaseOutputLayer, BasePretrainNetwork, BaseRecurrentLayer, BatchNormalization, CnnLossLayer, ConvolutionLayer, DenseLayer, DropoutLayer, ElementWiseMultiplicationLayer, EmbeddingLayer, EmbeddingSequenceLayer, LossLayer, RnnLossLayer
```
public abstract class BaseLayer<LayerConfT extends BaseLayer>
extends AbstractLayer<LayerConfT>
```
A layer with parameters

Author:

Adam Gibson

See Also:

Serialized Form

Nested Class Summary
- Nested classes/interfaces inherited from interface org.deeplearning4j.nn.api.Layer
  Layer.TrainingMode, Layer.Type

Field Summary

Fields
Modifier and Type	Field and Description
`protected Gradient`	`gradient`
`protected org.nd4j.linalg.api.ndarray.INDArray`	`gradientsFlattened`
`protected Map<String,org.nd4j.linalg.api.ndarray.INDArray>`	`gradientViews`
`protected ConvexOptimizer`	`optimizer`
`protected Map<String,org.nd4j.linalg.api.ndarray.INDArray>`	`params`
`protected org.nd4j.linalg.api.ndarray.INDArray`	`paramsFlattened`
`protected double`	`score`
`protected Solver`	`solver`
`protected Map<String,org.nd4j.linalg.api.ndarray.INDArray>`	`weightNoiseParams`

Fields inherited from class org.deeplearning4j.nn.layers.AbstractLayer
cacheMode, conf, dropoutApplied, dropoutMask, epochCount, index, input, iterationCount, maskArray, maskState, preOutput, trainingListeners

Constructor Summary

Constructors
Constructor and Description

BaseLayer(NeuralNetConfiguration conf)

BaseLayer(NeuralNetConfiguration conf, org.nd4j.linalg.api.ndarray.INDArray input)

Constructors
Constructor and Description
`BaseLayer(NeuralNetConfiguration conf)`
`BaseLayer(NeuralNetConfiguration conf, org.nd4j.linalg.api.ndarray.INDArray input)`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`accumulateScore(double accum)` Sets a rolling tally for the score.
`org.nd4j.linalg.api.ndarray.INDArray`	`activate(boolean training, LayerWorkspaceMgr workspaceMgr)` Perform forward pass and return the activations array with the last set input
`org.nd4j.linalg.primitives.Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray>`	`backpropGradient(org.nd4j.linalg.api.ndarray.INDArray epsilon, LayerWorkspaceMgr workspaceMgr)` Calculate the gradient relative to the error in the next layer
`double`	`calcL1(boolean backpropParamsOnly)` Calculate the l1 regularization term 0.0 if regularization is not used.
`double`	`calcL2(boolean backpropParamsOnly)` Calculate the l2 regularization term 0.0 if regularization is not used.
`void`	`clear()` Clear input
`void`	`clearNoiseWeightParams()`
`Layer`	`clone()` Clone the layer
`void`	`computeGradientAndScore(LayerWorkspaceMgr workspaceMgr)` Update the score
`void`	`fit()` All models have a fit method
`void`	`fit(org.nd4j.linalg.api.ndarray.INDArray input, LayerWorkspaceMgr workspaceMgr)` Fit the model to the given data
`org.nd4j.linalg.api.ndarray.INDArray`	`getGradientsViewArray()`
`ConvexOptimizer`	`getOptimizer()` Returns this models optimizer
`org.nd4j.linalg.api.ndarray.INDArray`	`getParam(String param)` Get the parameter
`protected org.nd4j.linalg.api.ndarray.INDArray`	`getParamWithNoise(String param, boolean training, LayerWorkspaceMgr workspaceMgr)` Get the parameter, after applying any weight noise (such as DropConnect) if necessary.
`Gradient`	`gradient()` Get the gradient.
`boolean`	`hasBias()` Does this layer have no bias term? Many layers (dense, convolutional, output, embedding) have biases by default, but no-bias versions are possible via configuration
`void`	`initParams()` Initialize the parameters
`LayerConfT`	`layerConf()`
`int`	`numParams()` The number of parameters for the model
`org.nd4j.linalg.api.ndarray.INDArray`	`params()` Returns the parameters of the neural network as a flattened row vector
`Map<String,org.nd4j.linalg.api.ndarray.INDArray>`	`paramTable()` The param table
`Map<String,org.nd4j.linalg.api.ndarray.INDArray>`	`paramTable(boolean backpropParamsOnly)` Table of parameters by key, for backprop For many models (dense layers, etc) - all parameters are backprop parameters
`protected org.nd4j.linalg.api.ndarray.INDArray`	`preOutput(boolean training, LayerWorkspaceMgr workspaceMgr)`
`double`	`score()` Objective function: the specified objective
`void`	`setBackpropGradientsViewArray(org.nd4j.linalg.api.ndarray.INDArray gradients)` Set the gradients array as a view of the full (backprop) network parameters NOTE: this is intended to be used internally in MultiLayerNetwork and ComputationGraph, not by users.
`void`	`setParam(String key, org.nd4j.linalg.api.ndarray.INDArray val)` Set the parameter with a new ndarray
`void`	`setParams(org.nd4j.linalg.api.ndarray.INDArray params)` Set the parameters for this model.
`protected void`	`setParams(org.nd4j.linalg.api.ndarray.INDArray params, char order)`
`void`	`setParamsViewArray(org.nd4j.linalg.api.ndarray.INDArray params)` Set the initial parameters array as a view of the full (backprop) network parameters NOTE: this is intended to be used internally in MultiLayerNetwork and ComputationGraph, not by users.
`void`	`setParamTable(Map<String,org.nd4j.linalg.api.ndarray.INDArray> paramTable)` Setter for the param table
`protected void`	`setScoreWithZ(org.nd4j.linalg.api.ndarray.INDArray z)`
`String`	`toString()`
`Layer`	`transpose()` Return a transposed copy of the weights/bias (this means reverse the number of inputs and outputs on the weights)
`void`	`update(Gradient gradient)` Update layer weights and biases with gradient change
`void`	`update(org.nd4j.linalg.api.ndarray.INDArray gradient, String paramType)` Perform one update applying the gradient

Methods inherited from class org.deeplearning4j.nn.layers.AbstractLayer
activate, addListeners, applyConstraints, applyDropOutIfNecessary, applyMask, assertInputSet, batchSize, conf, feedForwardMaskArray, getIndex, getInput, getInputMiniBatchSize, getListeners, getMaskArray, gradientAndScore, init, input, layerId, numParams, setCacheMode, setConf, setIndex, setInput, setInputMiniBatchSize, setListeners, setListeners, setMaskArray, type, validateInput

Methods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.deeplearning4j.nn.api.Layer
getEpochCount, getIterationCount, isPretrainLayer, setEpochCount, setIterationCount

- Field Detail
  - paramsFlattened
```
protected org.nd4j.linalg.api.ndarray.INDArray paramsFlattened
```
  - gradientsFlattened
```
protected org.nd4j.linalg.api.ndarray.INDArray gradientsFlattened
```
  - params
```
protected Map<String,org.nd4j.linalg.api.ndarray.INDArray> params
```
  - gradientViews
```
protected transient Map<String,org.nd4j.linalg.api.ndarray.INDArray> gradientViews
```
  - score
```
protected double score
```
  - optimizer
```
protected ConvexOptimizer optimizer
```
  - gradient
```
protected Gradient gradient
```
  - solver
```
protected Solver solver
```
  - weightNoiseParams
```
protected Map<String,org.nd4j.linalg.api.ndarray.INDArray> weightNoiseParams
```
- Constructor Detail
  - BaseLayer
```
public BaseLayer(NeuralNetConfiguration conf)
```
  - BaseLayer
```
public BaseLayer(NeuralNetConfiguration conf,
                 org.nd4j.linalg.api.ndarray.INDArray input)
```
- Method Detail
  - layerConf
```
public LayerConfT layerConf()
```
    Overrides:
    
    layerConf in class AbstractLayer<LayerConfT extends BaseLayer>
  - backpropGradient
```
public org.nd4j.linalg.primitives.Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray> backpropGradient(org.nd4j.linalg.api.ndarray.INDArray epsilon,
                                                                                                       LayerWorkspaceMgr workspaceMgr)
```
    Description copied from interface: Layer
    
    Calculate the gradient relative to the error in the next layer
    
    Parameters:
    
    epsilon - w^(L+1)*delta^(L+1). Or, equiv: dC/da, i.e., (dC/dz)*(dz/da) = dC/da, where C is cost function a=sigma(z) is activation.
    
    workspaceMgr - Workspace manager
    
    Returns:
    
    Pair where Gradient is gradient for this layer, INDArray is epsilon (activation gradient) needed by next layer, but before element-wise multiply by sigmaPrime(z). So for standard feed-forward layer, if this layer is L, then return.getSecond() == dL/dIn = (w^(L)*(delta^(L))^T)^T. Note that the returned array should be placed in the ArrayType.ACTIVATION_GRAD workspace via the workspace manager
  - fit
```
public void fit()
```
    Description copied from interface: Model
    
    All models have a fit method
    
    Specified by:
    
    fit in interface Model
    
    Overrides:
    
    fit in class AbstractLayer<LayerConfT extends BaseLayer>
  - computeGradientAndScore
```
public void computeGradientAndScore(LayerWorkspaceMgr workspaceMgr)
```
    Description copied from interface: Model
    
    Update the score
    
    Specified by:
    
    computeGradientAndScore in interface Model
    
    Overrides:
    
    computeGradientAndScore in class AbstractLayer<LayerConfT extends BaseLayer>
  - setScoreWithZ
```
protected void setScoreWithZ(org.nd4j.linalg.api.ndarray.INDArray z)
```
  - score
```
public double score()
```
    Objective function: the specified objective
    
    Specified by:
    
    score in interface Model
    
    Overrides:
    
    score in class AbstractLayer<LayerConfT extends BaseLayer>
    
    Returns:
    
    the score for the objective
  - gradient
```
public Gradient gradient()
```
    Description copied from interface: Model
    
    Get the gradient. Note that this method will not calculate the gradient, it will rather return the gradient that has been computed before. For calculating the gradient, see Model#computeGradientAndScore() .
    
    Specified by:
    
    gradient in interface Model
    
    Overrides:
    
    gradient in class AbstractLayer<LayerConfT extends BaseLayer>
    
    Returns:
    
    the gradient for this model, as calculated before
  - update
```
public void update(Gradient gradient)
```
    Description copied from interface: Model
    
    Update layer weights and biases with gradient change
    
    Specified by:
    
    update in interface Model
    
    Overrides:
    
    update in class AbstractLayer<LayerConfT extends BaseLayer>
  - update
```
public void update(org.nd4j.linalg.api.ndarray.INDArray gradient,
                   String paramType)
```
    Description copied from interface: Model
    
    Perform one update applying the gradient
    
    Specified by:
    
    update in interface Model
    
    Overrides:
    
    update in class AbstractLayer<LayerConfT extends BaseLayer>
    
    Parameters:
    
    gradient - the gradient to apply
  - getOptimizer
```
public ConvexOptimizer getOptimizer()
```
    Description copied from interface: Model
    
    Returns this models optimizer
    
    Specified by:
    
    getOptimizer in interface Model
    
    Overrides:
    
    getOptimizer in class AbstractLayer<LayerConfT extends BaseLayer>
    
    Returns:
    
    this models optimizer
  - params
```
public org.nd4j.linalg.api.ndarray.INDArray params()
```
    Returns the parameters of the neural network as a flattened row vector
    
    Specified by:
    
    params in interface Model
    
    Overrides:
    
    params in class AbstractLayer<LayerConfT extends BaseLayer>
    
    Returns:
    
    the parameters of the neural network
  - getParam
```
public org.nd4j.linalg.api.ndarray.INDArray getParam(String param)
```
    Description copied from interface: Model
    
    Get the parameter
    
    Specified by:
    
    getParam in interface Model
    
    Overrides:
    
    getParam in class AbstractLayer<LayerConfT extends BaseLayer>
    
    Parameters:
    
    param - the key of the parameter
    
    Returns:
    
    the parameter vector/matrix with that particular key
  - setParam
```
public void setParam(String key,
                     org.nd4j.linalg.api.ndarray.INDArray val)
```
    Description copied from interface: Model
    
    Set the parameter with a new ndarray
    
    Specified by:
    
    setParam in interface Model
    
    Overrides:
    
    setParam in class AbstractLayer<LayerConfT extends BaseLayer>
    
    Parameters:
    
    key - the key to se t
    
    val - the new ndarray
  - setParams
```
public void setParams(org.nd4j.linalg.api.ndarray.INDArray params)
```
    Description copied from interface: Model
    
    Set the parameters for this model. This expects a linear ndarray which then be unpacked internally relative to the expected ordering of the model
    
    Specified by:
    
    setParams in interface Model
    
    Overrides:
    
    setParams in class AbstractLayer<LayerConfT extends BaseLayer>
    
    Parameters:
    
    params - the parameters for the model
  - setParams
```
protected void setParams(org.nd4j.linalg.api.ndarray.INDArray params,
                         char order)
```
    Overrides:
    
    setParams in class AbstractLayer<LayerConfT extends BaseLayer>
  - setParamsViewArray
```
public void setParamsViewArray(org.nd4j.linalg.api.ndarray.INDArray params)
```
    Description copied from interface: Model
    
    Set the initial parameters array as a view of the full (backprop) network parameters NOTE: this is intended to be used internally in MultiLayerNetwork and ComputationGraph, not by users.
    
    Specified by:
    
    setParamsViewArray in interface Model
    
    Overrides:
    
    setParamsViewArray in class AbstractLayer<LayerConfT extends BaseLayer>
    
    Parameters:
    
    params - a 1 x nParams row vector that is a view of the larger (MLN/CG) parameters array
  - getGradientsViewArray
```
public org.nd4j.linalg.api.ndarray.INDArray getGradientsViewArray()
```
    Specified by:
    
    getGradientsViewArray in interface Model
    
    Overrides:
    
    getGradientsViewArray in class AbstractLayer<LayerConfT extends BaseLayer>
  - setBackpropGradientsViewArray
```
public void setBackpropGradientsViewArray(org.nd4j.linalg.api.ndarray.INDArray gradients)
```
    Description copied from interface: Model
    
    Set the gradients array as a view of the full (backprop) network parameters NOTE: this is intended to be used internally in MultiLayerNetwork and ComputationGraph, not by users.
    
    Specified by:
    
    setBackpropGradientsViewArray in interface Model
    
    Overrides:
    
    setBackpropGradientsViewArray in class AbstractLayer<LayerConfT extends BaseLayer>
    
    Parameters:
    
    gradients - a 1 x nParams row vector that is a view of the larger (MLN/CG) gradients array
  - setParamTable
```
public void setParamTable(Map<String,org.nd4j.linalg.api.ndarray.INDArray> paramTable)
```
    Description copied from interface: Model
    
    Setter for the param table
    
    Specified by:
    
    setParamTable in interface Model
    
    Overrides:
    
    setParamTable in class AbstractLayer<LayerConfT extends BaseLayer>
  - initParams
```
public void initParams()
```
    Description copied from interface: Model
    
    Initialize the parameters
    
    Specified by:
    
    initParams in interface Model
    
    Overrides:
    
    initParams in class AbstractLayer<LayerConfT extends BaseLayer>
  - paramTable
```
public Map<String,org.nd4j.linalg.api.ndarray.INDArray> paramTable()
```
    Description copied from interface: Model
    
    The param table
    
    Specified by:
    
    paramTable in interface Model
    
    Overrides:
    
    paramTable in class AbstractLayer<LayerConfT extends BaseLayer>
    
    Returns:
  - paramTable
```
public Map<String,org.nd4j.linalg.api.ndarray.INDArray> paramTable(boolean backpropParamsOnly)
```
    Description copied from interface: Model
    
    Table of parameters by key, for backprop For many models (dense layers, etc) - all parameters are backprop parameters
    
    Specified by:
    
    paramTable in interface Model
    
    Overrides:
    
    paramTable in class AbstractLayer<LayerConfT extends BaseLayer>
    
    Parameters:
    
    backpropParamsOnly - If true, return backprop params only. If false: return all params (equivalent to paramsTable())
  - getParamWithNoise
```
protected org.nd4j.linalg.api.ndarray.INDArray getParamWithNoise(String param,
                                                                 boolean training,
                                                                 LayerWorkspaceMgr workspaceMgr)
```
    Get the parameter, after applying any weight noise (such as DropConnect) if necessary. Note that during training, this will store the post-noise parameters, as these should be used for both forward pass and backprop, for a single iteration. Consequently, the parameters (post noise) should be cleared after each training iteration
    
    Parameters:
    
    param - Parameter key
    
    training - If true: during training
    
    Returns:
    
    The parameter, after applying any noise
  - preOutput
```
protected org.nd4j.linalg.api.ndarray.INDArray preOutput(boolean training,
                                                         LayerWorkspaceMgr workspaceMgr)
```
  - activate
```
public org.nd4j.linalg.api.ndarray.INDArray activate(boolean training,
                                                     LayerWorkspaceMgr workspaceMgr)
```
    Description copied from interface: Layer
    
    Perform forward pass and return the activations array with the last set input
    
    Parameters:
    
    training - training or test mode
    
    workspaceMgr - Workspace manager
    
    Returns:
    
    the activation (layer output) of the last specified input. Note that the returned array should be placed in the ArrayType.ACTIVATIONS workspace via the workspace manager
  - calcL2
```
public double calcL2(boolean backpropParamsOnly)
```
    Description copied from interface: Layer
    
    Calculate the l2 regularization term
    0.0 if regularization is not used. Or 0.5 * l2Coeff * l2Magnitude otherwise.
    Note that this does not divide by mini-batch size
    
    Specified by:
    
    calcL2 in interface Layer
    
    Overrides:
    
    calcL2 in class AbstractLayer<LayerConfT extends BaseLayer>
    
    Parameters:
    
    backpropParamsOnly - If true: calculate L2 based on backprop params only. If false: calculate based on all params (including pretrain params, if any)
    
    Returns:
    
    the l2 regularization term for this layer.
  - calcL1
```
public double calcL1(boolean backpropParamsOnly)
```
    Description copied from interface: Layer
    
    Calculate the l1 regularization term
    0.0 if regularization is not used. Or l1Coeff * l1Magnitude otherwise.
    Note that this does not divide by mini-batch size
    
    Specified by:
    
    calcL1 in interface Layer
    
    Overrides:
    
    calcL1 in class AbstractLayer<LayerConfT extends BaseLayer>
    
    Parameters:
    
    backpropParamsOnly - If true: calculate L1 based on backprop params only. If false: calculate based on all params (including pretrain params, if any)
    
    Returns:
    
    the l1 regularization term for this layer.
  - clone
```
public Layer clone()
```
    Description copied from interface: Layer
    
    Clone the layer
    
    Specified by:
    
    clone in interface Layer
    
    Specified by:
    
    clone in class AbstractLayer<LayerConfT extends BaseLayer>
    
    Returns:
  - numParams
```
public int numParams()
```
    The number of parameters for the model
    
    Specified by:
    
    numParams in interface Model
    
    Overrides:
    
    numParams in class AbstractLayer<LayerConfT extends BaseLayer>
    
    Returns:
    
    the number of parameters for the model
  - fit
```
public void fit(org.nd4j.linalg.api.ndarray.INDArray input,
                LayerWorkspaceMgr workspaceMgr)
```
    Description copied from interface: Model
    
    Fit the model to the given data
    
    Specified by:
    
    fit in interface Model
    
    Overrides:
    
    fit in class AbstractLayer<LayerConfT extends BaseLayer>
    
    Parameters:
    
    input - the data to fit the model to
  - toString
```
public String toString()
```
    Overrides:
    
    toString in class Object
  - transpose
```
public Layer transpose()
```
    Description copied from interface: Layer
    
    Return a transposed copy of the weights/bias (this means reverse the number of inputs and outputs on the weights)
    
    Specified by:
    
    transpose in interface Layer
    
    Overrides:
    
    transpose in class AbstractLayer<LayerConfT extends BaseLayer>
    
    Returns:
    
    the transposed layer
  - accumulateScore
```
public void accumulateScore(double accum)
```
    Description copied from interface: Model
    
    Sets a rolling tally for the score. This is useful for mini batch learning when you are accumulating error across a dataset.
    
    Specified by:
    
    accumulateScore in interface Model
    
    Overrides:
    
    accumulateScore in class AbstractLayer<LayerConfT extends BaseLayer>
    
    Parameters:
    
    accum - the amount to accum
  - clear
```
public void clear()
```
    Description copied from interface: Model
    
    Clear input
    
    Specified by:
    
    clear in interface Model
    
    Overrides:
    
    clear in class AbstractLayer<LayerConfT extends BaseLayer>
  - clearNoiseWeightParams
```
public void clearNoiseWeightParams()
```
  - hasBias
```
public boolean hasBias()
```
    Does this layer have no bias term? Many layers (dense, convolutional, output, embedding) have biases by default, but no-bias versions are possible via configuration
    
    Returns:
    
    True if a bias term is present, false otherwise

Class BaseLayer<LayerConfT extends BaseLayer>

Nested Class Summary

Nested classes/interfaces inherited from interface org.deeplearning4j.nn.api.Layer

Field Summary

Fields inherited from class org.deeplearning4j.nn.layers.AbstractLayer

Constructor Summary

Method Summary

Methods inherited from class org.deeplearning4j.nn.layers.AbstractLayer

Methods inherited from class java.lang.Object

Methods inherited from interface org.deeplearning4j.nn.api.Layer

Field Detail

paramsFlattened

gradientsFlattened

params

gradientViews

score

optimizer

gradient

solver

weightNoiseParams

Constructor Detail

BaseLayer

BaseLayer

Method Detail

layerConf

backpropGradient

fit

computeGradientAndScore

setScoreWithZ

score

gradient

update

update

getOptimizer

params

getParam

setParam

setParams

setParams

setParamsViewArray

getGradientsViewArray

setBackpropGradientsViewArray

setParamTable

initParams

paramTable

paramTable

getParamWithNoise

preOutput

activate

calcL2

calcL1

clone

numParams

fit

toString

transpose

accumulateScore

clear

clearNoiseWeightParams

hasBias