org.deeplearning4j.nn.updater

Class UpdaterBlock

java.lang.Object

org.deeplearning4j.nn.updater.UpdaterBlock

public class UpdaterBlock
extends Object

UpdaterBlock: used in BaseMultiLayerUpdater, this class implements updating (i.e., Adam, RMSProp, Momentum, etc) across multiple contiguous layers/parameters, as described in the BaseMultiLayerUpdater javadoc.

Author:

Alex Black

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	UpdaterBlock.ParamState

Constructor Summary

Constructors

Constructor and Description
UpdaterBlock(int paramOffsetStart, int paramOffsetEnd, int updaterViewOffsetStart, int updaterViewOffsetEnd, List<UpdaterBlock.ParamState> layersAndVariablesInBlock)

Method Summary

Modifier and Type	Method and Description
protected void	applyRegularization(Regularization.ApplyStep step, Trainable layer, String paramName, INDArray gradientView, INDArray paramsView, int iter, int epoch, double lr) Apply L1 and L2 regularization, if necessary.
protected void	applyRegularizationAllVariables(Regularization.ApplyStep applyStep, int iteration, int epoch, boolean externalGradient, INDArray fullNetworkGradientView, INDArray fullNetworkParamsArray)
GradientUpdater	getGradientUpdater()
void	init()
boolean	isPretrainUpdaterBlock()
boolean	skipDueToPretrainConfig(boolean isLayerUpdater)
void	update(int iteration, int epoch) Update the gradient for this block
void	updateExternalGradient(int iteration, int epoch, INDArray fullNetworkGradientView, INDArray fullNetworkParamsArray)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

UpdaterBlock

public UpdaterBlock(int paramOffsetStart,
                    int paramOffsetEnd,
                    int updaterViewOffsetStart,
                    int updaterViewOffsetEnd,
                    List<UpdaterBlock.ParamState> layersAndVariablesInBlock)

Parameters:

paramOffsetStart - Start offset of the parameters in this block (relative to overall net params view array)

paramOffsetEnd - End offset of the parameters in this block (relative to overall net params view array)

updaterViewOffsetStart - Start offset of the updater state array in this block (relative to overall net updater state view array)

updaterViewOffsetEnd - End offset of the updater state array in this block (relative to overall net updater state view array)

layersAndVariablesInBlock - List of layers and variables in this updater block. By definition, all layers and variables in this list must have an identical updater configuration.

Method Detail

init

public void init()

isPretrainUpdaterBlock

public boolean isPretrainUpdaterBlock()

skipDueToPretrainConfig

public boolean skipDueToPretrainConfig(boolean isLayerUpdater)

getGradientUpdater

public GradientUpdater getGradientUpdater()

update

public void update(int iteration,
                   int epoch)

Update the gradient for this block

Parameters:

iteration - The current iteration (i.e., total number of parameter updates so far)

updateExternalGradient

public void updateExternalGradient(int iteration,
                                   int epoch,
                                   INDArray fullNetworkGradientView,
                                   INDArray fullNetworkParamsArray)

applyRegularizationAllVariables

protected void applyRegularizationAllVariables(Regularization.ApplyStep applyStep,
                                               int iteration,
                                               int epoch,
                                               boolean externalGradient,
                                               INDArray fullNetworkGradientView,
                                               INDArray fullNetworkParamsArray)

applyRegularization

protected void applyRegularization(Regularization.ApplyStep step,
                                   Trainable layer,
                                   String paramName,
                                   INDArray gradientView,
                                   INDArray paramsView,
                                   int iter,
                                   int epoch,
                                   double lr)

Apply L1 and L2 regularization, if necessary. Note that L1/L2 may differ for different layers in the same block

Parameters:

layer - The layer to apply L1/L2 to

paramName - Parameter name in the given layer

gradientView - Gradient view array for the layer + param

paramsView - Parameter view array for the layer + param