org.deeplearning4j.optimize.solvers

Class StochasticHessianFree

java.lang.Object

org.deeplearning4j.optimize.solvers.BaseOptimizer

org.deeplearning4j.optimize.solvers.StochasticHessianFree

All Implemented Interfaces:

Serializable, ConvexOptimizer

public class StochasticHessianFree
extends BaseOptimizer

Hessian Free Optimization by Ryan Kiros http://www.cs.toronto.edu/~rkiros/papers/shf13.pdf

Author:

Adam Gibson

See Also:

Serialized Form

Field Summary

Fields inherited from class org.deeplearning4j.optimize.solvers.BaseOptimizer

conf, GRADIENT_KEY, iteration, iterationListeners, lineMaximizer, log, model, oldScore, PARAMS_KEY, SCORE_KEY, SEARCH_DIR, searchState, step, stepFunction, stepMax, terminationConditions, updater

Constructor Summary

Constructors

Constructor and Description
StochasticHessianFree(NeuralNetConfiguration conf, StepFunction stepFunction, Collection<IterationListener> iterationListeners, Collection<TerminationCondition> terminationConditions, Model model)
StochasticHessianFree(NeuralNetConfiguration conf, StepFunction stepFunction, Collection<IterationListener> iterationListeners, Model model)

Method Summary

Modifier and Type	Method and Description
Pair<org.nd4j.linalg.api.ndarray.INDArray,Double>	cgBackTrack(List<org.nd4j.linalg.api.ndarray.INDArray> chs, org.nd4j.linalg.api.ndarray.INDArray p) Iterate through the current list of gradients and backtrack upon an optimal step that improves the current score
Pair<List<Integer>,List<org.nd4j.linalg.api.ndarray.INDArray>>	conjGradient(org.nd4j.linalg.api.ndarray.INDArray b, org.nd4j.linalg.api.ndarray.INDArray x0, org.nd4j.linalg.api.ndarray.INDArray preCon, int numIterations)
boolean	isConverged()
double	lineSearch(double newScore, org.nd4j.linalg.api.ndarray.INDArray params, org.nd4j.linalg.api.ndarray.INDArray p) Search with the proposed objective
boolean	optimize() Optimize call.

Methods inherited from class org.deeplearning4j.optimize.solvers.BaseOptimizer

batchSize, checkTerminalConditions, getConf, getDefaultStepFunctionForOptimizer, getUpdater, gradientAndScore, postFirstStep, postStep, preProcessLine, score, setBatchSize, setupSearchState, updateGradientAccordingToParams

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StochasticHessianFree

public StochasticHessianFree(NeuralNetConfiguration conf,
                             StepFunction stepFunction,
                             Collection<IterationListener> iterationListeners,
                             Model model)

StochasticHessianFree

public StochasticHessianFree(NeuralNetConfiguration conf,
                             StepFunction stepFunction,
                             Collection<IterationListener> iterationListeners,
                             Collection<TerminationCondition> terminationConditions,
                             Model model)

Method Detail

isConverged

public boolean isConverged()

conjGradient

public Pair<List<Integer>,List<org.nd4j.linalg.api.ndarray.INDArray>> conjGradient(org.nd4j.linalg.api.ndarray.INDArray b,
                                                                                   org.nd4j.linalg.api.ndarray.INDArray x0,
                                                                                   org.nd4j.linalg.api.ndarray.INDArray preCon,
                                                                                   int numIterations)

lineSearch

public double lineSearch(double newScore,
                         org.nd4j.linalg.api.ndarray.INDArray params,
                         org.nd4j.linalg.api.ndarray.INDArray p)

Search with the proposed objective

Parameters:

newScore - the new score to start with

params - the params of the proposed step

Returns:

the rate to step by

cgBackTrack

public Pair<org.nd4j.linalg.api.ndarray.INDArray,Double> cgBackTrack(List<org.nd4j.linalg.api.ndarray.INDArray> chs,
                                                                     org.nd4j.linalg.api.ndarray.INDArray p)

Iterate through the current list of gradients and backtrack upon an optimal step that improves the current score

Parameters:

chs - the proposed changes

Returns:

the new changed path and score for that path

optimize

public boolean optimize()

Description copied from class: BaseOptimizer

Optimize call. This runs the optimizer.

Specified by:

optimize in interface ConvexOptimizer

Overrides:

optimize in class BaseOptimizer

Returns:

whether it converged or not