org.deeplearning4j.nn.updater

Class LayerUpdater

java.lang.Object

org.deeplearning4j.nn.updater.LayerUpdater

All Implemented Interfaces:

Serializable, Cloneable, Updater

public class LayerUpdater
extends Object
implements Updater

Author:

Adam Gibson

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected Map<String,org.nd4j.linalg.learning.GradientUpdater>	updaterForVariable
protected org.nd4j.linalg.api.ndarray.INDArray	viewArray

Constructor Summary

Constructors

Constructor and Description
LayerUpdater()

Method Summary

Modifier and Type	Method and Description
void	applyLrDecayPolicy(LearningRatePolicy decay, Layer layer, int iteration, String variable) Update learning rate based on policy
void	applyMomentumDecayPolicy(Layer layer, int iteration, String variable) Update momentum if schedule exist
Updater	clone()
boolean	equals(Object other)
org.nd4j.linalg.api.ndarray.INDArray	getStateViewArray()
Map<String,org.nd4j.linalg.learning.GradientUpdater>	getUpdaterForVariable()
int	hashCode()
void	init()
org.nd4j.linalg.learning.GradientUpdater	init(String variable, Layer layer)
void	postApply(Layer layer, org.nd4j.linalg.api.ndarray.INDArray gradient, String param, int miniBatchSize) Apply the regularization
void	preApply(Layer layer, Gradient gradient, int iteration) Apply gradient normalization: scale based on L2, clipping etc.
void	setStateViewArray(Layer layer, org.nd4j.linalg.api.ndarray.INDArray viewArray, boolean initialize) Set the internal (historical) state view array for this updater
int	stateSizeForLayer(Layer layer) Calculate and return the state size for this updater (for the given layer).
void	update(Layer layer, Gradient gradient, int iteration, int miniBatchSize) Updater: updates the model

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Field Detail

updaterForVariable

protected Map<String,org.nd4j.linalg.learning.GradientUpdater> updaterForVariable

viewArray

protected org.nd4j.linalg.api.ndarray.INDArray viewArray

Constructor Detail

LayerUpdater

public LayerUpdater()

Method Detail

setStateViewArray

public void setStateViewArray(Layer layer,
                              org.nd4j.linalg.api.ndarray.INDArray viewArray,
                              boolean initialize)

Description copied from interface: Updater

Set the internal (historical) state view array for this updater

Specified by:

setStateViewArray in interface Updater

Parameters:

layer - Layer that this updater belongs to

viewArray - View array

initialize - Whether to initialize the array or not

getUpdaterForVariable

public Map<String,org.nd4j.linalg.learning.GradientUpdater> getUpdaterForVariable()

getStateViewArray

public org.nd4j.linalg.api.ndarray.INDArray getStateViewArray()

Specified by:

getStateViewArray in interface Updater

Returns:

the view array for this updater

stateSizeForLayer

public int stateSizeForLayer(Layer layer)

Description copied from interface: Updater

Calculate and return the state size for this updater (for the given layer). How many parameters/values does this updater have?

Specified by:

stateSizeForLayer in interface Updater

Parameters:

layer - Layer that this updater belongs to

Returns:

number of parameters/values in the updater state

update

public void update(Layer layer,
                   Gradient gradient,
                   int iteration,
                   int miniBatchSize)

Description copied from interface: Updater

Updater: updates the model

Specified by:

update in interface Updater

postApply

public void postApply(Layer layer,
                      org.nd4j.linalg.api.ndarray.INDArray gradient,
                      String param,
                      int miniBatchSize)

Apply the regularization

Parameters:

layer -

gradient -

param -

applyMomentumDecayPolicy

public void applyMomentumDecayPolicy(Layer layer,
                                     int iteration,
                                     String variable)

Update momentum if schedule exist

applyLrDecayPolicy

public void applyLrDecayPolicy(LearningRatePolicy decay,
                               Layer layer,
                               int iteration,
                               String variable)

Update learning rate based on policy

preApply

public void preApply(Layer layer,
                     Gradient gradient,
                     int iteration)

Apply gradient normalization: scale based on L2, clipping etc. RenormalizeL2PerLayer: divide all layer gradients by L2 to rescale RenormalizeL2PerParamType: divide each parameter type gradient in a layer by L2 to rescale ClipElementWiseAbsoluteValue: clip gradients per-element ClipL2PerLayer: same as RenormalizeL2PerLayer but limited by gradient L2 norm for the layer meeting a threshold ClipL2PerParamType: same as RenormalizeL2PerParamType but limited by gradient L2 norm for each parameter type in a layer meeting a threshold

init

public void init()

init

public org.nd4j.linalg.learning.GradientUpdater init(String variable,
                                                     Layer layer)

equals

public boolean equals(Object other)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

clone

public Updater clone()

Specified by:

clone in interface Updater

Overrides:

clone in class Object