Package org.nd4j.linalg.learning.regularization

Class WeightDecay

java.lang.Object

org.nd4j.linalg.learning.regularization.WeightDecay

All Implemented Interfaces:

Serializable, Regularization

public class WeightDecay
extends Object
implements Regularization

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.nd4j.linalg.learning.regularization.Regularization

Regularization.ApplyStep

Field Summary

Fields

Modifier and Type	Field	Description
protected boolean	applyLR
protected ISchedule	coeff

Constructor Summary

Constructors

Constructor	Description
WeightDecay(double coeff, boolean applyLR)
WeightDecay(@NonNull ISchedule coeff, boolean applyLR)

Method Summary

Modifier and Type	Method	Description
void	apply(INDArray param, INDArray gradView, double lr, int iteration, int epoch)	Apply the regularization by modifying the gradient array in-place
Regularization.ApplyStep	applyStep()
Regularization	clone()
double	score(INDArray param, int iteration, int epoch)	Calculate the loss function score component for the regularization. For example, in L2 regularization, this would return L = 0.5 * sum_i param[i]^2 For regularization types that don't have a score component, this method can return 0.

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

coeff

protected final ISchedule coeff

applyLR

protected final boolean applyLR

Constructor Detail

WeightDecay

public WeightDecay(double coeff,
                   boolean applyLR)

Parameters:

coeff - Weight decay regularization coefficient

applyLR - If true, multiply the regularization coefficient by the current learning rate. If false, do not multiply by LR.

WeightDecay

public WeightDecay(@NonNull
                   @NonNull ISchedule coeff,
                   boolean applyLR)

Parameters:

coeff - Weight decay regularization coefficient (schedule)

applyLR - If true, multiply the regularization coefficient by the current learning rate. If false, do not multiply by LR.

Method Detail

applyStep

public Regularization.ApplyStep applyStep()

Specified by:

applyStep in interface Regularization

Returns:

The step that the regularization should be applied, as defined by Regularization.ApplyStep

apply

public void apply(INDArray param,
                  INDArray gradView,
                  double lr,
                  int iteration,
                  int epoch)

Description copied from interface: Regularization

Apply the regularization by modifying the gradient array in-place

Specified by:

apply in interface Regularization

Parameters:

param - Input array (usually parameters)

gradView - Gradient view array (should be modified/updated). Same shape and type as the input array.

lr - Current learning rate

iteration - Current network training iteration

epoch - Current network training epoch

score

public double score(INDArray param,
                    int iteration,
                    int epoch)

Description copied from interface: Regularization

Calculate the loss function score component for the regularization.
For example, in L2 regularization, this would return L = 0.5 * sum_i param[i]^2
For regularization types that don't have a score component, this method can return 0. However, note that this may make the regularization type not gradient checkable.

Specified by:

score in interface Regularization

Parameters:

param - Input array (usually parameters)

iteration - Current network training iteration

epoch - Current network training epoch

Returns:

Loss function score component based on the input/parameters array

clone

public Regularization clone()

Specified by:

clone in interface Regularization

Overrides:

clone in class Object

Returns:

An independent copy of the regularization instance