org.deeplearning4j.nn.updater

Class BaseUpdater

java.lang.Object

org.deeplearning4j.nn.updater.BaseUpdater

All Implemented Interfaces:

Serializable, Updater

Direct Known Subclasses:

AdaDeltaUpdater, AdaGradUpdater, AdamUpdater, NesterovsUpdater, NoOpUpdater, RmsPropUpdater, SgdUpdater

public abstract class BaseUpdater
extends Object
implements Updater

Author:

Adam Gibson

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	BaseUpdater.UpdaterAggregatorImpl

Field Summary

Fields

Modifier and Type	Field and Description
protected Map<String,org.nd4j.linalg.learning.GradientUpdater>	updaterForVariable

Constructor Summary

Constructors

Constructor and Description
BaseUpdater()

Method Summary

Modifier and Type	Method and Description
void	checkSchedules(Layer layer, int iteration, String param) Update learningRate and/or momentum if schedules exist
boolean	equals(Object other)
abstract void	init()
abstract org.nd4j.linalg.learning.GradientUpdater	init(String variable, org.nd4j.linalg.api.ndarray.INDArray gradient, Layer layer)
void	postApply(Layer layer, org.nd4j.linalg.api.ndarray.INDArray gradient, String param, int miniBatchSize) Apply the regularization
void	preApply(Layer layer, Gradient gradient, int iteration) Apply gradient normalization: scale based on L2, clipping etc.
void	update(Layer layer, Gradient gradient, int iteration, int miniBatchSize) Updater: updates the model

Methods inherited from class java.lang.Object

clone, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.deeplearning4j.nn.api.Updater

getAggregator

Field Detail

updaterForVariable

protected Map<String,org.nd4j.linalg.learning.GradientUpdater> updaterForVariable

Constructor Detail

BaseUpdater

public BaseUpdater()

Method Detail

update

public void update(Layer layer,
                   Gradient gradient,
                   int iteration,
                   int miniBatchSize)

Description copied from interface: Updater

Updater: updates the model

Specified by:

update in interface Updater

postApply

public void postApply(Layer layer,
                      org.nd4j.linalg.api.ndarray.INDArray gradient,
                      String param,
                      int miniBatchSize)

Apply the regularization

Parameters:

layer -

gradient -

param -

checkSchedules

public void checkSchedules(Layer layer,
                           int iteration,
                           String param)

Update learningRate and/or momentum if schedules exist

preApply

public void preApply(Layer layer,
                     Gradient gradient,
                     int iteration)

Apply gradient normalization: scale based on L2, clipping etc. RenormalizeL2PerLayer: divide all layer gradients by L2 to rescale RenormalizeL2PerParamType: divide each parameter type gradient in a layer by L2 to rescale ClipElementWiseAbsoluteValue: clip gradients per-element ClipL2PerLayer: same as RenormalizeL2PerLayer but limited by gradient L2 norm for the layer meeting a threshold ClipL2PerParamType: same as RenormalizeL2PerParamType but limited by gradient L2 norm for each parameter type in a layer meeting a threshold

init

public abstract void init()

init

public abstract org.nd4j.linalg.learning.GradientUpdater init(String variable,
                                                              org.nd4j.linalg.api.ndarray.INDArray gradient,
                                                              Layer layer)

equals

public boolean equals(Object other)

Overrides:

equals in class Object