org.deeplearning4j.optimize.solvers.accumulation

Class SmartFancyBlockingQueue

java.lang.Object

org.deeplearning4j.optimize.solvers.accumulation.FancyBlockingQueue<INDArray>

org.deeplearning4j.optimize.solvers.accumulation.SmartFancyBlockingQueue

All Implemented Interfaces:

Iterable<INDArray>, Collection<INDArray>, BlockingQueue<INDArray>, Queue<INDArray>, Registerable

public class SmartFancyBlockingQueue
extends FancyBlockingQueue<INDArray>

This class provides additional functionality to FancyBlockingQueue: it tracks memory use of stored compressed INDArrays, and if their size becomes too big, it: a) decompresses them into single INDArray b) removes original updates messages c) keeps updating single INDArray until it gets consumed d) once that happened - it automatically switches back to original behavior

Author:

[email protected]

Field Summary

Fields

Modifier and Type	Field and Description
protected AtomicBoolean	collapsedMode
protected int	decompressionThreshold
protected char	paramsOrder
protected long[]	paramsShape
protected EDU.oswego.cs.dl.util.concurrent.ReaderPreferenceReadWriteLock	smartLock

Fields inherited from class org.deeplearning4j.optimize.solvers.accumulation.FancyBlockingQueue

backingQueue, barrier, bypassMode, consumers, currentConsumers, currentStep, isDebug, isDone, isFirst, lock, numElementsDrained, numElementsReady, secondary, state, step

Constructor Summary

Constructors

Constructor and Description
SmartFancyBlockingQueue(int decompressionThreshold, BlockingQueue<INDArray> queue, INDArray paramsMatrix)
SmartFancyBlockingQueue(int decompressionThreshold, INDArray paramsMatrix)

Method Summary

Modifier and Type	Method and Description
boolean	isEmpty()
INDArray	poll()
void	put(INDArray array)
protected INDArray	smartDecompress(INDArray encoded, INDArray target)

Methods inherited from class org.deeplearning4j.optimize.solvers.accumulation.FancyBlockingQueue

add, addAll, clear, contains, containsAll, drainTo, drainTo, element, fallbackToSingleConsumerMode, iterator, offer, offer, peek, poll, registerConsumers, remainingCapacity, remove, remove, removeAll, retainAll, size, synchronize, take, toArray, toArray

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Collection

equals, hashCode, parallelStream, removeIf, spliterator, stream

Methods inherited from interface java.lang.Iterable

forEach

Field Detail

smartLock

protected final EDU.oswego.cs.dl.util.concurrent.ReaderPreferenceReadWriteLock smartLock

decompressionThreshold

protected int decompressionThreshold

collapsedMode

protected AtomicBoolean collapsedMode

paramsShape

protected final long[] paramsShape

paramsOrder

protected final char paramsOrder

Constructor Detail

SmartFancyBlockingQueue

public SmartFancyBlockingQueue(int decompressionThreshold,
                               @NonNull
                               INDArray paramsMatrix)

SmartFancyBlockingQueue

public SmartFancyBlockingQueue(int decompressionThreshold,
                               BlockingQueue<INDArray> queue,
                               @NonNull
                               INDArray paramsMatrix)

Method Detail

smartDecompress

protected INDArray smartDecompress(INDArray encoded,
                                   INDArray target)

isEmpty

public boolean isEmpty()

Specified by:

isEmpty in interface Collection<INDArray>

Overrides:

isEmpty in class FancyBlockingQueue<INDArray>

put

public void put(INDArray array)
         throws InterruptedException

Specified by:

put in interface BlockingQueue<INDArray>

Overrides:

put in class FancyBlockingQueue<INDArray>

Throws:

InterruptedException

poll

public INDArray poll()

Specified by:

poll in interface Queue<INDArray>

Overrides:

poll in class FancyBlockingQueue<INDArray>