BaseMultiLayerUpdater (deeplearning4j-nn 1.0.0-beta4 API)

java.lang.Object
- org.deeplearning4j.nn.updater.BaseMultiLayerUpdater<T>

All Implemented Interfaces:

Serializable, Updater

Direct Known Subclasses:

ComputationGraphUpdater, LayerUpdater, MultiLayerUpdater
```
public abstract class BaseMultiLayerUpdater<T extends Model>
extends Object
implements Updater
```
BaseMultiLayerUpdater - core functionality for applying updaters to MultiLayerNetwork and ComputationGraph.
This implements updater combining: that is, for any layers (and variables) that:
(a) have contiguous parameters/gradients in the view arrays, and
(b) have identical updater configuration (including updater, LR, LR/momentum schedules, etc - different L1/L2 are OK, however)
are combined into a single GradientUpdater operation, instead of having a set of smaller operations. A smaller number of larger operations improves performance, especially for GPUs.

Author:

Alex Black

See Also:

Serialized Form

Field Summary

Fields
Modifier and Type	Field and Description
`protected List<INDArray>`	`gradientsForMinibatchDivision`
`protected boolean`	`initializedMinibatchDivision`
`protected Map<String,Trainable>`	`layersByName`
`protected T`	`network`
`protected List<UpdaterBlock>`	`updaterBlocks`
`protected INDArray`	`updaterStateViewArray`

Constructor Summary

Constructors
Constructor and Description

BaseMultiLayerUpdater(T network)

BaseMultiLayerUpdater(T network, INDArray updaterState)

Constructors
Constructor and Description
`BaseMultiLayerUpdater(T network)`
`BaseMultiLayerUpdater(T network, INDArray updaterState)`

Method Summary

All Methods Instance Methods Abstract Methods Concrete Methods
Modifier and Type	Method and Description
`protected void`	`divideByMinibatch(boolean isExternal, Gradient gradient, int batchSize)`
`boolean`	`equals(Object o)`
`protected abstract INDArray`	`getFlattenedGradientsView()`
`protected List<INDArray>`	`getMinibatchDivisionSubsets(INDArray from)`
`protected abstract Trainable[]`	`getOrderedLayers()`
`protected abstract INDArray`	`getParams()`
`INDArray`	`getStateViewArray()`
`INDArray`	`getStateViewArrayCopy()` A synchronized version of `getStateViewArray()` that duplicates the view array internally.
`int`	`hashCode()`
`protected abstract boolean`	`isMiniBatch()`
`protected boolean`	`isSingleLayerUpdater()`
`void`	`preApply(Trainable layer, Gradient gradient, int iteration)` Pre-apply: Apply gradient normalization/clipping
`void`	`setStateViewArray(INDArray viewArray)` Set the view array.
`void`	`setStateViewArray(Trainable layer, INDArray viewArray, boolean initialize)` Set the internal (historical) state view array for this updater
`void`	`update(Gradient gradient, int iteration, int epoch, int batchSize, LayerWorkspaceMgr workspaceMgr)` Update the gradient for the model.
`void`	`update(Trainable layer, Gradient gradient, int iteration, int epoch, int batchSize, LayerWorkspaceMgr workspaceMgr)` Updater: updates the model

Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - network
```
protected final T extends Model network
```
  - layersByName
```
protected Map<String,Trainable> layersByName
```
  - updaterBlocks
```
protected final List<UpdaterBlock> updaterBlocks
```
  - updaterStateViewArray
```
protected INDArray updaterStateViewArray
```
  - initializedMinibatchDivision
```
protected boolean initializedMinibatchDivision
```
  - gradientsForMinibatchDivision
```
protected List<INDArray> gradientsForMinibatchDivision
```
- Constructor Detail
  - BaseMultiLayerUpdater
```
public BaseMultiLayerUpdater(T network)
```
  - BaseMultiLayerUpdater
```
public BaseMultiLayerUpdater(T network,
                             INDArray updaterState)
```
    Parameters:
    
    network - Network to create the updater for
    
    updaterState - The updater state to use. Note: This array is used *directly* and isn't copied/cloned
- Method Detail
  - getOrderedLayers
```
protected abstract Trainable[] getOrderedLayers()
```
    Returns:
    
    Array of layers, in the correct order (i.e., same order as the parameter/gradient/updater flattening order - input to output for MultiLayerNetwork, or topological order for ComputationGraph)
  - getFlattenedGradientsView
```
protected abstract INDArray getFlattenedGradientsView()
```
    Returns:
    
    The flattened gradient view array for the model
  - getParams
```
protected abstract INDArray getParams()
```
    Returns:
    
    The flattened parameter array for the model
  - isMiniBatch
```
protected abstract boolean isMiniBatch()
```
    Returns:
    
    True if the configuration for the model is set to minibatch (divide by minibatch size), false otherwise
  - setStateViewArray
```
public void setStateViewArray(INDArray viewArray)
```
    Set the view array. Note that this does an assign operation - the provided array is not stored internally.
    
    Parameters:
    
    viewArray - The new updater state
  - setStateViewArray
```
public void setStateViewArray(Trainable layer,
                              INDArray viewArray,
                              boolean initialize)
```
    Description copied from interface: Updater
    
    Set the internal (historical) state view array for this updater
    
    Specified by:
    
    setStateViewArray in interface Updater
    
    Parameters:
    
    layer - Layer that this updater belongs to
    
    viewArray - View array
    
    initialize - Whether to initialize the array or not
  - getStateViewArray
```
public INDArray getStateViewArray()
```
    Specified by:
    
    getStateViewArray in interface Updater
    
    Returns:
    
    the view array for this updater
  - getStateViewArrayCopy
```
public INDArray getStateViewArrayCopy()
```
    A synchronized version of getStateViewArray() that duplicates the view array internally. This should be used in preference to getStateViewArray() when the updater state is accessed in one thread while another thread is using the updater for training.
    
    Returns:
    
    A copy (duplicate) of the updater state
  - update
```
public void update(Trainable layer,
                   Gradient gradient,
                   int iteration,
                   int epoch,
                   int batchSize,
                   LayerWorkspaceMgr workspaceMgr)
```
    Description copied from interface: Updater
    
    Updater: updates the model
    
    Specified by:
    
    update in interface Updater
  - update
```
public void update(Gradient gradient,
                   int iteration,
                   int epoch,
                   int batchSize,
                   LayerWorkspaceMgr workspaceMgr)
```
    Update the gradient for the model. This operates in 3 steps: 1. Pre-apply: gradient clipping, etc on a per-layer basis 2. Execute the updater (Adam, Nesterov momentum, etc) - in blocks of layers at a time 3. Divide by minibatch size
    
    Parameters:
    
    gradient - Gradient to updater
    
    iteration - The current iteration (i.e., number of parameter updates so far)
    
    batchSize - The current minibatch size (number of examples)
  - divideByMinibatch
```
protected void divideByMinibatch(boolean isExternal,
                                 Gradient gradient,
                                 int batchSize)
```
  - getMinibatchDivisionSubsets
```
protected List<INDArray> getMinibatchDivisionSubsets(INDArray from)
```
  - isSingleLayerUpdater
```
protected boolean isSingleLayerUpdater()
```
  - preApply
```
public void preApply(Trainable layer,
                     Gradient gradient,
                     int iteration)
```
    Pre-apply: Apply gradient normalization/clipping
    
    Parameters:
    
    layer - Layer to apply gradient normalization/clipping for
    
    gradient - Gradient to update
    
    iteration - The current iteration (i.e., number of parameter updates so far)
  - equals
```
public boolean equals(Object o)
```
    Overrides:
    
    equals in class Object
  - hashCode
```
public int hashCode()
```
    Overrides:
    
    hashCode in class Object

Class BaseMultiLayerUpdater<T extends Model>

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

network

layersByName

updaterBlocks

updaterStateViewArray

initializedMinibatchDivision

gradientsForMinibatchDivision

Constructor Detail

BaseMultiLayerUpdater

BaseMultiLayerUpdater

Method Detail

getOrderedLayers

getFlattenedGradientsView

getParams

isMiniBatch

setStateViewArray

setStateViewArray

getStateViewArray

getStateViewArrayCopy

update

update

divideByMinibatch

getMinibatchDivisionSubsets

isSingleLayerUpdater

preApply

equals

hashCode