public class LocalResponseNormalization extends AbstractLayer<LocalResponseNormalization>
For a^i_{x,y} the activity of a neuron computed by applying kernel i at position (x,y) and applying ReLU nonlinearity, the response normalized activation b^i_{x,y} is given by:
x^2 = (a^j_{x,y})^2 unitScale = (k + alpha * sum_{j=max(0, i - n/2)}^{max(N-1, i + n/2)} (a^j_{x,y})^2 ) y = b^i_{x,y} = x * unitScale**-beta
gy = epsilon (aka deltas from previous layer) sumPart = sum(a^j_{x,y} * gb^j_{x,y}) gx = gy * unitScale**-beta - 2 * alpha * beta * sumPart/unitScale * a^i_{x,y}
Reference: http://www.cs.toronto.edu/~fritz/absps/imagenet.pdf https://github.com/vlfeat/matconvnet/issues/10 Chainer
Created by nyghtowl on 10/29/15.
Layer.TrainingMode, Layer.Type| Modifier and Type | Field and Description |
|---|---|
protected static org.slf4j.Logger |
log |
cacheMode, conf, dropoutApplied, dropoutMask, epochCount, index, input, iterationCount, maskArray, maskState, preOutput, trainingListeners| Constructor and Description |
|---|
LocalResponseNormalization(NeuralNetConfiguration conf) |
LocalResponseNormalization(NeuralNetConfiguration conf,
org.nd4j.linalg.api.ndarray.INDArray input) |
| Modifier and Type | Method and Description |
|---|---|
org.nd4j.linalg.api.ndarray.INDArray |
activate(boolean training,
LayerWorkspaceMgr workspaceMgr)
Perform forward pass and return the activations array with the last set input
|
org.nd4j.linalg.primitives.Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray> |
backpropGradient(org.nd4j.linalg.api.ndarray.INDArray epsilon,
LayerWorkspaceMgr workspaceMgr)
Calculate the gradient relative to the error in the next layer
|
double |
calcL1(boolean backpropParamsOnly)
Calculate the l1 regularization term
0.0 if regularization is not used. |
double |
calcL2(boolean backpropParamsOnly)
Calculate the l2 regularization term
0.0 if regularization is not used. |
void |
clearNoiseWeightParams() |
Layer |
clone()
Clone the layer
|
void |
fit(org.nd4j.linalg.api.ndarray.INDArray input,
LayerWorkspaceMgr workspaceMgr)
Fit the model to the given data
|
org.nd4j.linalg.api.ndarray.INDArray |
getParam(String param)
Get the parameter
|
boolean |
isPretrainLayer()
Returns true if the layer can be trained in an unsupervised/pretrain manner (AE, VAE, etc)
|
org.nd4j.linalg.api.ndarray.INDArray |
params()
Returns the parameters of the neural network as a flattened row vector
|
void |
setParams(org.nd4j.linalg.api.ndarray.INDArray params)
Set the parameters for this model.
|
Layer |
transpose()
Return a transposed copy of the weights/bias
(this means reverse the number of inputs and outputs on the weights)
|
Layer.Type |
type()
Returns the layer type
|
accumulateScore, activate, addListeners, applyConstraints, applyDropOutIfNecessary, applyMask, assertInputSet, batchSize, clear, computeGradientAndScore, conf, feedForwardMaskArray, fit, getGradientsViewArray, getIndex, getInput, getInputMiniBatchSize, getListeners, getMaskArray, getOptimizer, gradient, gradientAndScore, init, initParams, input, layerConf, layerId, numParams, numParams, paramTable, paramTable, score, setBackpropGradientsViewArray, setCacheMode, setConf, setIndex, setInput, setInputMiniBatchSize, setListeners, setListeners, setMaskArray, setParam, setParams, setParamsViewArray, setParamTable, update, update, validateInputequals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitgetEpochCount, getIterationCount, setEpochCount, setIterationCountpublic LocalResponseNormalization(NeuralNetConfiguration conf, org.nd4j.linalg.api.ndarray.INDArray input)
public LocalResponseNormalization(NeuralNetConfiguration conf)
public Layer clone()
Layerclone in interface Layerclone in class AbstractLayer<LocalResponseNormalization>public double calcL2(boolean backpropParamsOnly)
LayercalcL2 in interface LayercalcL2 in class AbstractLayer<LocalResponseNormalization>backpropParamsOnly - If true: calculate L2 based on backprop params only. If false: calculate
based on all params (including pretrain params, if any)public double calcL1(boolean backpropParamsOnly)
LayercalcL1 in interface LayercalcL1 in class AbstractLayer<LocalResponseNormalization>backpropParamsOnly - If true: calculate L1 based on backprop params only. If false: calculate
based on all params (including pretrain params, if any)public Layer.Type type()
Layertype in interface Layertype in class AbstractLayer<LocalResponseNormalization>public void fit(org.nd4j.linalg.api.ndarray.INDArray input,
LayerWorkspaceMgr workspaceMgr)
Modelfit in interface Modelfit in class AbstractLayer<LocalResponseNormalization>input - the data to fit the model topublic org.nd4j.linalg.primitives.Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray> backpropGradient(org.nd4j.linalg.api.ndarray.INDArray epsilon, LayerWorkspaceMgr workspaceMgr)
Layerepsilon - w^(L+1)*delta^(L+1). Or, equiv: dC/da, i.e., (dC/dz)*(dz/da) = dC/da, where C
is cost function a=sigma(z) is activation.workspaceMgr - Workspace managerArrayType.ACTIVATION_GRAD workspace via the workspace managerpublic org.nd4j.linalg.api.ndarray.INDArray activate(boolean training,
LayerWorkspaceMgr workspaceMgr)
Layertraining - training or test modeworkspaceMgr - Workspace managerArrayType.ACTIVATIONS workspace via the workspace managerpublic Layer transpose()
Layertranspose in interface Layertranspose in class AbstractLayer<LocalResponseNormalization>public boolean isPretrainLayer()
Layerpublic void clearNoiseWeightParams()
public org.nd4j.linalg.api.ndarray.INDArray params()
AbstractLayerparams in interface Modelparams in class AbstractLayer<LocalResponseNormalization>public org.nd4j.linalg.api.ndarray.INDArray getParam(String param)
ModelgetParam in interface ModelgetParam in class AbstractLayer<LocalResponseNormalization>param - the key of the parameterpublic void setParams(org.nd4j.linalg.api.ndarray.INDArray params)
ModelsetParams in interface ModelsetParams in class AbstractLayer<LocalResponseNormalization>params - the parameters for the modelCopyright © 2018. All rights reserved.