org.deeplearning4j.optimize.solvers

Class StochasticHessianFree

java.lang.Object

org.deeplearning4j.optimize.solvers.StochasticHessianFree

All Implemented Interfaces:

OptimizerMatrix

public class StochasticHessianFree
extends Object
implements OptimizerMatrix

Hessian Free Optimization by Ryan Kiros http://www.cs.toronto.edu/~rkiros/papers/shf13.pdf

Author:

Adam Gibson

Constructor Summary

Constructors

Constructor and Description
StochasticHessianFree(OptimizableByGradientValueMatrix function, BaseMultiLayerNetwork network)
StochasticHessianFree(OptimizableByGradientValueMatrix function, double initialStepSize, BaseMultiLayerNetwork network)
StochasticHessianFree(OptimizableByGradientValueMatrix function, double initialStepSize, IterationListener listener, BaseMultiLayerNetwork network)
StochasticHessianFree(OptimizableByGradientValueMatrix function, IterationListener listener, BaseMultiLayerNetwork network)

Method Summary

Modifier and Type	Method and Description
Pair<org.nd4j.linalg.api.ndarray.INDArray,Double>	cgBackTrack(List<org.nd4j.linalg.api.ndarray.INDArray> chs, org.nd4j.linalg.api.ndarray.INDArray p) Iterate through the current applyTransformToDestination of gradients and backtrack upon an optimal step that improves the current score
Pair<List<Integer>,List<org.nd4j.linalg.api.ndarray.INDArray>>	conjGradient(org.nd4j.linalg.api.ndarray.INDArray b, org.nd4j.linalg.api.ndarray.INDArray x0, org.nd4j.linalg.api.ndarray.INDArray preCon, int numIterations)
int	getMaxIterations()
boolean	isConverged() Whether the algorithm is converged
double	lineSearch(double newScore, org.nd4j.linalg.api.ndarray.INDArray params, org.nd4j.linalg.api.ndarray.INDArray p) Search with the proposed objective
boolean	optimize() Run optimize
boolean	optimize(int numIterations) Run optimize up to the specified number of epochs
void	reset()
void	setMaxIterations(int maxIterations) The default max number of iterations to run
void	setTolerance(double tolerance) The tolerance for change when running
void	setTrainingEvaluator(TrainingEvaluator eval) Sets the training evaluator

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StochasticHessianFree

public StochasticHessianFree(OptimizableByGradientValueMatrix function,
                             double initialStepSize,
                             BaseMultiLayerNetwork network)

StochasticHessianFree

public StochasticHessianFree(OptimizableByGradientValueMatrix function,
                             IterationListener listener,
                             BaseMultiLayerNetwork network)

StochasticHessianFree

public StochasticHessianFree(OptimizableByGradientValueMatrix function,
                             double initialStepSize,
                             IterationListener listener,
                             BaseMultiLayerNetwork network)

StochasticHessianFree

public StochasticHessianFree(OptimizableByGradientValueMatrix function,
                             BaseMultiLayerNetwork network)

Method Detail

isConverged

public boolean isConverged()

Description copied from interface: OptimizerMatrix

Whether the algorithm is converged

Specified by:

isConverged in interface OptimizerMatrix

Returns:

true if the algorithm converged, false otherwise

optimize

public boolean optimize()

Description copied from interface: OptimizerMatrix

Run optimize

Specified by:

optimize in interface OptimizerMatrix

Returns:

whether the algorithm converged properly

conjGradient

public Pair<List<Integer>,List<org.nd4j.linalg.api.ndarray.INDArray>> conjGradient(org.nd4j.linalg.api.ndarray.INDArray b,
                                                                                   org.nd4j.linalg.api.ndarray.INDArray x0,
                                                                                   org.nd4j.linalg.api.ndarray.INDArray preCon,
                                                                                   int numIterations)

lineSearch

public double lineSearch(double newScore,
                         org.nd4j.linalg.api.ndarray.INDArray params,
                         org.nd4j.linalg.api.ndarray.INDArray p)

Search with the proposed objective

Parameters:

newScore - the new score to start with

params - the params of the proposed step

Returns:

the rate to step by

cgBackTrack

public Pair<org.nd4j.linalg.api.ndarray.INDArray,Double> cgBackTrack(List<org.nd4j.linalg.api.ndarray.INDArray> chs,
                                                                     org.nd4j.linalg.api.ndarray.INDArray p)

Iterate through the current applyTransformToDestination of gradients and backtrack upon an optimal step that improves the current score

Parameters:

chs - the proposed changes

Returns:

the new changed path and score for that path

optimize

public boolean optimize(int numIterations)

Description copied from interface: OptimizerMatrix

Run optimize up to the specified number of epochs

Specified by:

optimize in interface OptimizerMatrix

Parameters:

numIterations - the max number of epochs to run

Returns:

whether the algorihtm converged properly

setTrainingEvaluator

public void setTrainingEvaluator(TrainingEvaluator eval)

Sets the training evaluator

Specified by:

setTrainingEvaluator in interface OptimizerMatrix

Parameters:

eval - the evaluator to use

reset

public void reset()

getMaxIterations

public int getMaxIterations()

setMaxIterations

public void setMaxIterations(int maxIterations)

Description copied from interface: OptimizerMatrix

The default max number of iterations to run

Specified by:

setMaxIterations in interface OptimizerMatrix

setTolerance

public void setTolerance(double tolerance)

The tolerance for change when running

Specified by:

setTolerance in interface OptimizerMatrix

Parameters:

tolerance -