org.deeplearning4j.datasets.iterator.parallel

Class FileSplitParallelDataSetIterator

java.lang.Object

org.deeplearning4j.datasets.iterator.parallel.BaseParallelDataSetIterator

org.deeplearning4j.datasets.iterator.parallel.FileSplitParallelDataSetIterator

All Implemented Interfaces:

Serializable, Iterator<org.nd4j.linalg.dataset.DataSet>, org.nd4j.linalg.dataset.api.iterator.DataSetIterator, org.nd4j.linalg.dataset.api.iterator.ParallelDataSetIterator

public class FileSplitParallelDataSetIterator
extends BaseParallelDataSetIterator

Author:

[email protected]

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected List<org.nd4j.linalg.dataset.api.iterator.DataSetIterator>	asyncIterators
static String	DEFAULT_PATTERN

Fields inherited from class org.deeplearning4j.datasets.iterator.parallel.BaseParallelDataSetIterator

allDepleted, counter, inequalityHandling, numProducers, producerAffinity, resetTracker, states

Constructor Summary

Constructors

Constructor and Description
FileSplitParallelDataSetIterator(File rootFolder, String pattern, FileCallback callback)
FileSplitParallelDataSetIterator(File rootFolder, String pattern, FileCallback callback, int numThreads)
FileSplitParallelDataSetIterator(File rootFolder, String pattern, FileCallback callback, int numThreads, org.nd4j.linalg.dataset.api.iterator.enums.InequalityHandling inequalityHandling)
FileSplitParallelDataSetIterator(File rootFolder, String pattern, FileCallback callback, int numThreads, int bufferPerThread, org.nd4j.linalg.dataset.api.iterator.enums.InequalityHandling inequalityHandling)

Method Summary

Modifier and Type	Method and Description
boolean	hasNextFor(int consumer)
org.nd4j.linalg.dataset.DataSet	nextFor(int consumer)
protected void	reset(int consumer)

Methods inherited from class org.deeplearning4j.datasets.iterator.parallel.BaseParallelDataSetIterator

asyncSupported, attachThread, batch, cursor, getCurrentProducerIndex, getLabels, getPreProcessor, hasNext, hasNextFor, inputColumns, next, next, nextFor, numExamples, remove, reset, resetSupported, setPreProcessor, stepForward, totalExamples, totalOutcomes

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Iterator

forEachRemaining

Field Detail

DEFAULT_PATTERN

public static final String DEFAULT_PATTERN

See Also:

Constant Field Values

asyncIterators

protected List<org.nd4j.linalg.dataset.api.iterator.DataSetIterator> asyncIterators

Constructor Detail

FileSplitParallelDataSetIterator

public FileSplitParallelDataSetIterator(@NonNull
                                        File rootFolder,
                                        @NonNull
                                        String pattern,
                                        @NonNull
                                        FileCallback callback)

FileSplitParallelDataSetIterator

public FileSplitParallelDataSetIterator(@NonNull
                                        File rootFolder,
                                        @NonNull
                                        String pattern,
                                        @NonNull
                                        FileCallback callback,
                                        int numThreads)

FileSplitParallelDataSetIterator

public FileSplitParallelDataSetIterator(@NonNull
                                        File rootFolder,
                                        @NonNull
                                        String pattern,
                                        @NonNull
                                        FileCallback callback,
                                        int numThreads,
                                        @NonNull
                                        org.nd4j.linalg.dataset.api.iterator.enums.InequalityHandling inequalityHandling)

FileSplitParallelDataSetIterator

public FileSplitParallelDataSetIterator(@NonNull
                                        File rootFolder,
                                        @NonNull
                                        String pattern,
                                        @NonNull
                                        FileCallback callback,
                                        int numThreads,
                                        int bufferPerThread,
                                        @NonNull
                                        org.nd4j.linalg.dataset.api.iterator.enums.InequalityHandling inequalityHandling)

Method Detail

hasNextFor

public boolean hasNextFor(int consumer)

Specified by:

hasNextFor in interface org.nd4j.linalg.dataset.api.iterator.ParallelDataSetIterator

Specified by:

hasNextFor in class BaseParallelDataSetIterator

nextFor

public org.nd4j.linalg.dataset.DataSet nextFor(int consumer)

Specified by:

nextFor in interface org.nd4j.linalg.dataset.api.iterator.ParallelDataSetIterator

Specified by:

nextFor in class BaseParallelDataSetIterator

reset

protected void reset(int consumer)

Specified by:

reset in class BaseParallelDataSetIterator