org.nd4j.linalg.learning

Class AdaGrad

java.lang.Object

org.nd4j.linalg.learning.AdaGrad

All Implemented Interfaces:

Serializable, GradientUpdater

public class AdaGrad
extends Object
implements Serializable, GradientUpdater

Vectorized Learning Rate used per Connection Weight

Adapted from: http://xcorr.net/2014/01/23/adagrad-eliminating-learning-rates-in-stochastic-gradient-descent/

Author:

Adam Gibson

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
INDArray	historicalGradient
protected double	learningRate
protected int	numIterations
int[]	shape

Constructor Summary

Constructors

Constructor and Description
AdaGrad(double learningRate)
AdaGrad(int[] shape, double learningRate)
AdaGrad(int rows, int cols)
AdaGrad(int rows, int cols, double learningRate)

Method Summary

Modifier and Type	Method and Description
AdaGrad	createSubset(int index)
double	getGradient(double gradient, int column, int[] shape)
INDArray	getGradient(INDArray gradient, int iteration) Gets feature specific learning rates Adagrad keeps a history of gradients being passed in.
INDArray	getGradient(INDArray gradient, int slice, int[] shape)
void	update(Object... args) update(learningRate,momentum)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

historicalGradient

public INDArray historicalGradient

shape

public int[] shape

learningRate

protected double learningRate

numIterations

protected int numIterations

Constructor Detail

AdaGrad

public AdaGrad(int rows,
               int cols,
               double learningRate)

Parameters:

rows -

cols -

learningRate -

AdaGrad

public AdaGrad(int rows,
               int cols)

AdaGrad

public AdaGrad(int[] shape,
               double learningRate)

AdaGrad

public AdaGrad(double learningRate)

Method Detail

update

public void update(Object... args)

Description copied from interface: GradientUpdater

update(learningRate,momentum)

Specified by:

update in interface GradientUpdater

getGradient

public INDArray getGradient(INDArray gradient,
                            int iteration)

Gets feature specific learning rates Adagrad keeps a history of gradients being passed in. Note that each gradient passed in becomes adapted over time, hence the name adagrad

Specified by:

getGradient in interface GradientUpdater

Parameters:

gradient - the gradient to get learning rates for

iteration -

Returns:

the feature specific learning rates

getGradient

public double getGradient(double gradient,
                          int column,
                          int[] shape)

getGradient

public INDArray getGradient(INDArray gradient,
                            int slice,
                            int[] shape)

createSubset

public AdaGrad createSubset(int index)