Package smile.deep.optimizer

Class Adam

java.lang.Object
smile.deep.optimizer.Adam
All Implemented Interfaces:
Serializable, Optimizer
public class Adam extends Object implements Optimizer
Adaptive Moment optimizer. Adam computes adaptive learning rates for each parameter. In addition to storing an exponentially decaying average of past squared gradients RMSProp, Adam also keeps an exponentially decaying average of past gradients, similar to momentum. Whereas momentum can be seen as a ball running down a slope, Adam behaves like a heavy ball with friction, which thus prefers flat minima in the error surface.
See Also:

  • Constructor Summary

    Constructors
    Constructor
    Description
    Adam()
    Constructor.
    Adam(smile.math.TimeFunction learningRate)
    Constructor.
    Adam(smile.math.TimeFunction learningRate, double beta1, double beta2)
    Constructor.
    Adam(smile.math.TimeFunction learningRate, double beta1, double beta2, double epsilon)
    Constructor.

  • Method Summary

    Modifier and Type
    Method
    Description
    String
    toString()
     
    void
    update(Layer layer, int m, int t)
    Updates a layer.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Constructor Details

    • Adam

      public Adam()
      Constructor.

    • Adam

      public Adam(smile.math.TimeFunction learningRate)
      Constructor.
      Parameters:
      learningRate - the learning rate.

    • Adam

      public Adam(smile.math.TimeFunction learningRate, double beta1, double beta2)
      Constructor.
      Parameters:
      learningRate - the learning rate.
      beta1 - the exponential decay rate for the 1st moment estimates.
      beta2 - the exponential decay rate for the 2nd moment estimates.

    • Adam

      public Adam(smile.math.TimeFunction learningRate, double beta1, double beta2, double epsilon)
      Constructor.
      Parameters:
      learningRate - the learning rate.
      beta1 - the exponential decay rate for the 1st moment estimates.
      beta2 - the exponential decay rate for the 2nd moment estimates.
      epsilon - a small constant for numerical stability.

  • Method Details

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • update

      public void update(Layer layer, int m, int t)
      Description copied from interface: Optimizer
      Updates a layer.
      Specified by:
      update in interface Optimizer
      Parameters:
      layer - a neural network layer.
      m - the size of mini-batch.
      t - the time step, i.e. the number of training iterations so far.