org.deeplearning4j.nn.updater

Class BaseUpdater

java.lang.Object

org.deeplearning4j.nn.updater.BaseUpdater

All Implemented Interfaces:

Serializable, Cloneable, Updater

Direct Known Subclasses:

AdaDeltaUpdater, AdaGradUpdater, AdamUpdater, NesterovsUpdater, NoOpUpdater, RmsPropUpdater, SgdUpdater

public abstract class BaseUpdater
extends Object
implements Updater

Author:

Adam Gibson

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	BaseUpdater.UpdaterAggregatorImpl

Field Summary

Fields

Modifier and Type	Field and Description
protected Map<String,org.nd4j.linalg.learning.GradientUpdater>	updaterForVariable

Constructor Summary

Constructors

Constructor and Description
BaseUpdater()

Method Summary

Modifier and Type	Method and Description
void	applyLrDecayPolicy(LearningRatePolicy decay, Layer layer, int iteration, String variable) Update learning rate based on policy
void	applyMomentumDecayPolicy(Layer layer, int iteration, String variable) Update momentum if schedule exist
Updater	clone()
boolean	equals(Object other)
abstract void	init()
abstract org.nd4j.linalg.learning.GradientUpdater	init(String variable, org.nd4j.linalg.api.ndarray.INDArray gradient, Layer layer)
void	postApply(Layer layer, org.nd4j.linalg.api.ndarray.INDArray gradient, String param, int miniBatchSize) Apply the regularization
void	preApply(Layer layer, Gradient gradient, int iteration) Apply gradient normalization: scale based on L2, clipping etc.
void	update(Layer layer, Gradient gradient, int iteration, int miniBatchSize) Updater: updates the model

Methods inherited from class java.lang.Object

finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.deeplearning4j.nn.api.Updater

getAggregator

Field Detail

updaterForVariable

protected Map<String,org.nd4j.linalg.learning.GradientUpdater> updaterForVariable

Constructor Detail

BaseUpdater

public BaseUpdater()

Method Detail

update

public void update(Layer layer,
                   Gradient gradient,
                   int iteration,
                   int miniBatchSize)

Description copied from interface: Updater

Updater: updates the model

Specified by:

update in interface Updater

postApply

public void postApply(Layer layer,
                      org.nd4j.linalg.api.ndarray.INDArray gradient,
                      String param,
                      int miniBatchSize)

Apply the regularization

Parameters:

layer -

gradient -

param -

applyMomentumDecayPolicy

public void applyMomentumDecayPolicy(Layer layer,
                                     int iteration,
                                     String variable)

Update momentum if schedule exist

applyLrDecayPolicy

public void applyLrDecayPolicy(LearningRatePolicy decay,
                               Layer layer,
                               int iteration,
                               String variable)

Update learning rate based on policy

preApply

public void preApply(Layer layer,
                     Gradient gradient,
                     int iteration)

Apply gradient normalization: scale based on L2, clipping etc. RenormalizeL2PerLayer: divide all layer gradients by L2 to rescale RenormalizeL2PerParamType: divide each parameter type gradient in a layer by L2 to rescale ClipElementWiseAbsoluteValue: clip gradients per-element ClipL2PerLayer: same as RenormalizeL2PerLayer but limited by gradient L2 norm for the layer meeting a threshold ClipL2PerParamType: same as RenormalizeL2PerParamType but limited by gradient L2 norm for each parameter type in a layer meeting a threshold

init

public abstract void init()

init

public abstract org.nd4j.linalg.learning.GradientUpdater init(String variable,
                                                              org.nd4j.linalg.api.ndarray.INDArray gradient,
                                                              Layer layer)

equals

public boolean equals(Object other)

Overrides:

equals in class Object

clone

public Updater clone()

Specified by:

clone in interface Updater

Overrides:

clone in class Object