org.deeplearning4j.optimize.solvers

Class BaseOptimizer

java.lang.Object

org.deeplearning4j.optimize.solvers.BaseOptimizer

All Implemented Interfaces:

Serializable, ConvexOptimizer

Direct Known Subclasses:

ConjugateGradient, GradientAscent, IterationGradientDescent, LBFGS, StochasticHessianFree

public abstract class BaseOptimizer
extends Object
implements ConvexOptimizer

Base optimizer

Author:

Adam Gibson

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected org.nd4j.linalg.learning.AdaGrad	adaGrad
protected Map<String,org.nd4j.linalg.learning.AdaGrad>	adaGradForVariable
protected NeuralNetConfiguration	conf
static String	GRADIENT_KEY
protected int	iteration
protected Collection<IterationListener>	iterationListeners
protected BackTrackLineSearch	lineMaximizer
protected static org.slf4j.Logger	log
protected Model	model
protected double	oldScore
static String	PARAMS_KEY
protected double	score
static String	SCORE_KEY
protected Map<String,Object>	searchState
protected double	step
protected StepFunction	stepFunction
protected double	stpMax
protected Collection<TerminationCondition>	terminationConditions

Constructor Summary

Constructors

Constructor and Description
BaseOptimizer(NeuralNetConfiguration conf, StepFunction stepFunction, Collection<IterationListener> iterationListeners, Collection<TerminationCondition> terminationConditions, Model model)
BaseOptimizer(NeuralNetConfiguration conf, StepFunction stepFunction, Collection<IterationListener> iterationListeners, Model model)

Method Summary

Modifier and Type	Method and Description
Map<String,org.nd4j.linalg.learning.AdaGrad>	adaGradForVariables() Return the ada grad look up table
int	batchSize() The batch size for the optimizer
org.nd4j.linalg.learning.AdaGrad	getAdaGrad() The adagrad in this model
org.nd4j.linalg.learning.AdaGrad	getAdaGradForVariable(String variable) Get adagrad for a variable
Pair<Gradient,Double>	gradientAndScore() The gradient and score for this optimizer
boolean	optimize() Optimize call.
protected void	postFirstStep(org.nd4j.linalg.api.ndarray.INDArray gradient)
void	postStep() Post step (conjugate gradient among other methods needs this)
protected boolean	preFirstStepProcess(org.nd4j.linalg.api.ndarray.INDArray gradient)
void	preProcessLine(org.nd4j.linalg.api.ndarray.INDArray line) Pre process the line (scaling and the like)
double	score() The score for the optimizer so far
void	setBatchSize(int batchSize) Set the batch size for the optimizer
void	setupSearchState(Pair<Gradient,Double> pair) Setup the initial search state
void	updateGradientAccordingToParams(Gradient gradient, Model params, int batchSize) Update the gradient according to the configuration such as adagrad, momentum, and sparsity
void	updateGradientAccordingToParams(org.nd4j.linalg.api.ndarray.INDArray gradient, org.nd4j.linalg.api.ndarray.INDArray params, int batchSize) Update the gradient according to the configuration such as adagrad, momentum, and sparsity

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

conf

protected NeuralNetConfiguration conf

adaGrad

protected org.nd4j.linalg.learning.AdaGrad adaGrad

iteration

protected int iteration

log

protected static final org.slf4j.Logger log

stepFunction

protected StepFunction stepFunction

iterationListeners

protected Collection<IterationListener> iterationListeners

terminationConditions

protected Collection<TerminationCondition> terminationConditions

model

protected Model model

lineMaximizer

protected BackTrackLineSearch lineMaximizer

step

protected double step

score

protected double score

oldScore

protected double oldScore

stpMax

protected double stpMax

GRADIENT_KEY

public static final String GRADIENT_KEY

See Also:

Constant Field Values

SCORE_KEY

public static final String SCORE_KEY

See Also:

Constant Field Values

PARAMS_KEY

public static final String PARAMS_KEY

See Also:

Constant Field Values

adaGradForVariable

protected Map<String,org.nd4j.linalg.learning.AdaGrad> adaGradForVariable

searchState

protected Map<String,Object> searchState

Constructor Detail

BaseOptimizer

public BaseOptimizer(NeuralNetConfiguration conf,
                     StepFunction stepFunction,
                     Collection<IterationListener> iterationListeners,
                     Model model)

Parameters:

conf -

stepFunction -

iterationListeners -

model -

BaseOptimizer

public BaseOptimizer(NeuralNetConfiguration conf,
                     StepFunction stepFunction,
                     Collection<IterationListener> iterationListeners,
                     Collection<TerminationCondition> terminationConditions,
                     Model model)

Parameters:

conf -

stepFunction -

iterationListeners -

terminationConditions -

model -

Method Detail

updateGradientAccordingToParams

public void updateGradientAccordingToParams(org.nd4j.linalg.api.ndarray.INDArray gradient,
                                            org.nd4j.linalg.api.ndarray.INDArray params,
                                            int batchSize)

Update the gradient according to the configuration such as adagrad, momentum, and sparsity

Specified by:

updateGradientAccordingToParams in interface ConvexOptimizer

Parameters:

gradient - the gradient to modify

score

public double score()

Description copied from interface: ConvexOptimizer

The score for the optimizer so far

Specified by:

score in interface ConvexOptimizer

Returns:

the score for this optimizer so far

gradientAndScore

public Pair<Gradient,Double> gradientAndScore()

Description copied from interface: ConvexOptimizer

The gradient and score for this optimizer

Specified by:

gradientAndScore in interface ConvexOptimizer

Returns:

the gradient and score for this optimizer

optimize

public boolean optimize()

Optimize call. This runs the optimizer.

Specified by:

optimize in interface ConvexOptimizer

Returns:

whether it converged or not

postFirstStep

protected void postFirstStep(org.nd4j.linalg.api.ndarray.INDArray gradient)

preFirstStepProcess

protected boolean preFirstStepProcess(org.nd4j.linalg.api.ndarray.INDArray gradient)

batchSize

public int batchSize()

Description copied from interface: ConvexOptimizer

The batch size for the optimizer

Specified by:

batchSize in interface ConvexOptimizer

Returns:

setBatchSize

public void setBatchSize(int batchSize)

Description copied from interface: ConvexOptimizer

Set the batch size for the optimizer

Specified by:

setBatchSize in interface ConvexOptimizer

preProcessLine

public void preProcessLine(org.nd4j.linalg.api.ndarray.INDArray line)

Pre process the line (scaling and the like)

Specified by:

preProcessLine in interface ConvexOptimizer

Parameters:

line - the line to pre process

postStep

public void postStep()

Post step (conjugate gradient among other methods needs this)

Specified by:

postStep in interface ConvexOptimizer

getAdaGrad

public org.nd4j.linalg.learning.AdaGrad getAdaGrad()

Description copied from interface: ConvexOptimizer

The adagrad in this model

Specified by:

getAdaGrad in interface ConvexOptimizer

Returns:

the adagrad in this model

adaGradForVariables

public Map<String,org.nd4j.linalg.learning.AdaGrad> adaGradForVariables()

Description copied from interface: ConvexOptimizer

Return the ada grad look up table

Specified by:

adaGradForVariables in interface ConvexOptimizer

Returns:

the ada grad for variables

getAdaGradForVariable

public org.nd4j.linalg.learning.AdaGrad getAdaGradForVariable(String variable)

Description copied from interface: ConvexOptimizer

Get adagrad for a variable

Specified by:

getAdaGradForVariable in interface ConvexOptimizer

Returns:

updateGradientAccordingToParams

public void updateGradientAccordingToParams(Gradient gradient,
                                            Model params,
                                            int batchSize)

Description copied from interface: ConvexOptimizer

Update the gradient according to the configuration such as adagrad, momentum, and sparsity

Specified by:

updateGradientAccordingToParams in interface ConvexOptimizer

Parameters:

gradient - the gradient to modify

params - the parameters to update

setupSearchState

public void setupSearchState(Pair<Gradient,Double> pair)

Setup the initial search state

Specified by:

setupSearchState in interface ConvexOptimizer

Parameters:

pair -