org.deeplearning4j.optimize.solvers.accumulation

Class BasicGradientsAccumulator

java.lang.Object

org.deeplearning4j.optimize.solvers.accumulation.BasicGradientsAccumulator

All Implemented Interfaces:

Serializable, GradientsAccumulator

public class BasicGradientsAccumulator
extends Object
implements GradientsAccumulator

This class provides accumulation for gradients for both input (i.e. updates coming from network) and output (comint from one ore more models training at the same time)

Author:

[email protected]

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected CyclicBarrier	barrier
protected List<org.nd4j.linalg.api.ndarray.INDArray>	candidates
protected AtomicLong	extCounter
protected AtomicLong	firstOne
protected Queue<org.nd4j.linalg.api.ndarray.INDArray>	gradients
protected MessageHandler	handler
protected AtomicBoolean	hasSomething
protected char	ordering
protected AtomicLong	ownCounter
protected int	parties
protected int[]	shape
protected org.nd4j.linalg.api.ndarray.INDArray	storage
protected org.nd4j.linalg.api.ndarray.INDArray	updates
protected ReentrantReadWriteLock	updatesLock

Constructor Summary

Constructors

Constructor and Description
BasicGradientsAccumulator(int parties) Creates new GradientsAccumulator with starting threshold of 1e-3
BasicGradientsAccumulator(int parties, MessageHandler handler) Creates new GradientsAccumulator with custom starting threshold

Method Summary

Modifier and Type	Method and Description
void	applyUpdate(StepFunction function, org.nd4j.linalg.api.ndarray.INDArray params, org.nd4j.linalg.api.ndarray.INDArray grad) This method applies accumulated updates via given StepFunction
void	applyUpdate(StepFunction function, org.nd4j.linalg.api.ndarray.INDArray params, org.nd4j.linalg.api.ndarray.INDArray grad, double alpha) This method applies accumulated updates via given StepFunction
void	receiveUpdate(org.nd4j.linalg.api.ndarray.INDArray array) This method accepts updates suitable for StepFunction and puts them to the queue, which is used in backpropagation loop PLEASE NOTE: array is expected to be ready for use and match params dimensionality
void	reset() This method resets all accumulated updates (if any)
void	setExternalSource(Queue<org.nd4j.linalg.api.ndarray.INDArray> source) This method allows to pass external updates to accumulator, they will be populated across all workers using this GradientsAccumulator instance
void	storeUpdate(org.nd4j.linalg.api.ndarray.INDArray array) This method accepts updates suitable for StepFunction, and accumulates/propagates it across all workers
void	touch() This method does initialization of given worker wrt Thread-Device Affinity

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

handler

protected MessageHandler handler

gradients

protected transient Queue<org.nd4j.linalg.api.ndarray.INDArray> gradients

storage

protected transient org.nd4j.linalg.api.ndarray.INDArray storage

updates

protected transient org.nd4j.linalg.api.ndarray.INDArray updates

ownCounter

protected transient AtomicLong ownCounter

extCounter

protected transient AtomicLong extCounter

shape

protected int[] shape

ordering

protected char ordering

parties

protected int parties

barrier

protected CyclicBarrier barrier

firstOne

protected AtomicLong firstOne

candidates

protected List<org.nd4j.linalg.api.ndarray.INDArray> candidates

updatesLock

protected ReentrantReadWriteLock updatesLock

hasSomething

protected AtomicBoolean hasSomething

Constructor Detail

BasicGradientsAccumulator

public BasicGradientsAccumulator(int parties)

Creates new GradientsAccumulator with starting threshold of 1e-3

BasicGradientsAccumulator

public BasicGradientsAccumulator(int parties,
                                 @NonNull
                                 MessageHandler handler)

Creates new GradientsAccumulator with custom starting threshold

Parameters:

handler - MessageHandler instance that'll be used for communication purposes

Method Detail

applyUpdate

public void applyUpdate(StepFunction function,
                        org.nd4j.linalg.api.ndarray.INDArray params,
                        org.nd4j.linalg.api.ndarray.INDArray grad)

This method applies accumulated updates via given StepFunction

Specified by:

applyUpdate in interface GradientsAccumulator

Parameters:

function -

params -

applyUpdate

public void applyUpdate(StepFunction function,
                        org.nd4j.linalg.api.ndarray.INDArray params,
                        org.nd4j.linalg.api.ndarray.INDArray grad,
                        double alpha)

This method applies accumulated updates via given StepFunction

Specified by:

applyUpdate in interface GradientsAccumulator

Parameters:

function -

params -

storeUpdate

public void storeUpdate(org.nd4j.linalg.api.ndarray.INDArray array)

This method accepts updates suitable for StepFunction, and accumulates/propagates it across all workers

Specified by:

storeUpdate in interface GradientsAccumulator

Parameters:

array -

receiveUpdate

public void receiveUpdate(org.nd4j.linalg.api.ndarray.INDArray array)

This method accepts updates suitable for StepFunction and puts them to the queue, which is used in backpropagation loop PLEASE NOTE: array is expected to be ready for use and match params dimensionality

Specified by:

receiveUpdate in interface GradientsAccumulator

Parameters:

array -

reset

public void reset()

This method resets all accumulated updates (if any)

Specified by:

reset in interface GradientsAccumulator

touch

public void touch()

This method does initialization of given worker wrt Thread-Device Affinity

Specified by:

touch in interface GradientsAccumulator

setExternalSource

public void setExternalSource(Queue<org.nd4j.linalg.api.ndarray.INDArray> source)

Description copied from interface: GradientsAccumulator

This method allows to pass external updates to accumulator, they will be populated across all workers using this GradientsAccumulator instance

Specified by:

setExternalSource in interface GradientsAccumulator