Class BaseDataFetcher
- java.lang.Object
-
- org.nd4j.linalg.dataset.api.iterator.fetcher.BaseDataFetcher
-
- All Implemented Interfaces:
Serializable
,DataSetFetcher
public abstract class BaseDataFetcher extends Object implements DataSetFetcher
- See Also:
- Serialized Form
-
-
Field Summary
Fields Modifier and Type Field Description protected DataSet
curr
protected int
cursor
protected int
inputColumns
protected static org.slf4j.Logger
log
protected int
numOutcomes
protected File
topLevelDir
protected int
totalExamples
-
Constructor Summary
Constructors Constructor Description BaseDataFetcher()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description protected INDArray
createInputMatrix(int numRows)
Creates a feature vectorprotected INDArray
createOutputMatrix(int numRows)
protected INDArray
createOutputVector(int outcomeLabel)
Creates an output label matrixint
cursor()
Direct access to a number represenative of iterating through a datasetboolean
hasMore()
Whether the dataset has more to loadprotected void
initializeCurrFromList(List<DataSet> examples)
Initializes this data transform fetcher from the passed in datasetsint
inputColumns()
The length of a feature vector for an individual exampleDataSet
next()
Returns the next data applyTransformToDestinationvoid
reset()
Returns the fetcher back to the beginning of the datasetvoid
setTopLevelDir(File topLevelDir)
For file download datasets this sets a top level directory outside the defaultFile
topLevelDir()
The top level directory to download datasets from This will usually be the default dl4j resources directory at user home/.deeplearning4jint
totalExamples()
The total number of examplesint
totalOutcomes()
The number of labels for a dataset-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.nd4j.linalg.dataset.api.iterator.fetcher.DataSetFetcher
fetch
-
-
-
-
Method Detail
-
topLevelDir
public File topLevelDir()
Description copied from interface:DataSetFetcher
The top level directory to download datasets from This will usually be the default dl4j resources directory at user home/.deeplearning4j- Specified by:
topLevelDir
in interfaceDataSetFetcher
- Returns:
-
setTopLevelDir
public void setTopLevelDir(File topLevelDir)
Description copied from interface:DataSetFetcher
For file download datasets this sets a top level directory outside the default- Specified by:
setTopLevelDir
in interfaceDataSetFetcher
- Parameters:
topLevelDir
- the top level directory to use
-
createInputMatrix
protected INDArray createInputMatrix(int numRows)
Creates a feature vector- Parameters:
numRows
- the number of examples- Returns:
- a feature vector
-
createOutputVector
protected INDArray createOutputVector(int outcomeLabel)
Creates an output label matrix- Parameters:
outcomeLabel
- the outcome label to use- Returns:
- a binary vector where 1 is transform to the index specified by outcomeLabel
-
createOutputMatrix
protected INDArray createOutputMatrix(int numRows)
-
initializeCurrFromList
protected void initializeCurrFromList(List<DataSet> examples)
Initializes this data transform fetcher from the passed in datasets- Parameters:
examples
- the examples to use
-
hasMore
public boolean hasMore()
Description copied from interface:DataSetFetcher
Whether the dataset has more to load- Specified by:
hasMore
in interfaceDataSetFetcher
- Returns:
- whether the data applyTransformToDestination has more to load
-
next
public DataSet next()
Description copied from interface:DataSetFetcher
Returns the next data applyTransformToDestination- Specified by:
next
in interfaceDataSetFetcher
- Returns:
- the next dataset
-
totalOutcomes
public int totalOutcomes()
Description copied from interface:DataSetFetcher
The number of labels for a dataset- Specified by:
totalOutcomes
in interfaceDataSetFetcher
- Returns:
- the number of labels for a dataset
-
inputColumns
public int inputColumns()
Description copied from interface:DataSetFetcher
The length of a feature vector for an individual example- Specified by:
inputColumns
in interfaceDataSetFetcher
- Returns:
- the length of a feature vector for an individual example
-
totalExamples
public int totalExamples()
Description copied from interface:DataSetFetcher
The total number of examples- Specified by:
totalExamples
in interfaceDataSetFetcher
- Returns:
- the total number of examples
-
reset
public void reset()
Description copied from interface:DataSetFetcher
Returns the fetcher back to the beginning of the dataset- Specified by:
reset
in interfaceDataSetFetcher
-
cursor
public int cursor()
Description copied from interface:DataSetFetcher
Direct access to a number represenative of iterating through a dataset- Specified by:
cursor
in interfaceDataSetFetcher
- Returns:
- a cursor similar to an index
-
-