Class KFoldIterator
- java.lang.Object
-
- org.nd4j.linalg.dataset.api.iterator.KFoldIterator
-
- All Implemented Interfaces:
Serializable
,Iterator<DataSet>
,DataSetIterator
public class KFoldIterator extends Object implements DataSetIterator
- See Also:
- Serialized Form
-
-
Field Summary
Fields Modifier and Type Field Description protected DataSet
allData
protected int[]
intervalBoundaries
protected int
k
protected int
kCursor
protected int
N
protected DataSetPreProcessor
preProcessor
protected DataSet
test
protected DataSet
train
-
Constructor Summary
Constructors Constructor Description KFoldIterator(int k, DataSet allData)
Create an iterator given the dataset with given k train-test splits N number of samples are split into k batches.KFoldIterator(DataSet allData)
Create a k-fold cross-validation iterator given the dataset and k=10 train-test splits.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description boolean
asyncSupported()
Does this DataSetIterator support asynchronous prefetching of multiple DataSet objects? Most DataSetIterators do, but in some cases it may not make sense to wrap this iterator in an iterator that does asynchronous prefetching.int
batch()
The number of examples in every fold is (N / k), except when (N % k) > 0, when the first (N % k) folds contain (N / k) + 1 examplesList<String>
getLabels()
Get dataset iterator class labels, if any.DataSetPreProcessor
getPreProcessor()
Returns preprocessors, if definedboolean
hasNext()
int
inputColumns()
Input columns for the datasetDataSet
next()
DataSet
next(int num)
Like the standard next method but allows a customizable number of examples returnedprotected void
nextFold()
void
remove()
void
reset()
Shuffles the dataset and resets to the first foldboolean
resetSupported()
Is resetting supported by this DataSetIterator? Many DataSetIterators do support resetting, but some don'tvoid
setPreProcessor(DataSetPreProcessor preProcessor)
Set a pre processorDataSet
testFold()
int
totalExamples()
Returns total number of examples in the dataset (all k folds)int
totalOutcomes()
The number of labels for the dataset-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface java.util.Iterator
forEachRemaining
-
-
-
-
Field Detail
-
allData
protected DataSet allData
-
k
protected int k
-
N
protected int N
-
intervalBoundaries
protected int[] intervalBoundaries
-
kCursor
protected int kCursor
-
test
protected DataSet test
-
train
protected DataSet train
-
preProcessor
protected DataSetPreProcessor preProcessor
-
-
Constructor Detail
-
KFoldIterator
public KFoldIterator(DataSet allData)
Create a k-fold cross-validation iterator given the dataset and k=10 train-test splits. N number of samples are split into k batches. The first (N%k) batches contain (N/k)+1 samples, while the remaining batches contain (N/k) samples. In case the number of samples (N) in the dataset is a multiple of k, all batches will contain (N/k) samples.- Parameters:
allData
- DataSet to split into k folds
-
KFoldIterator
public KFoldIterator(int k, DataSet allData)
Create an iterator given the dataset with given k train-test splits N number of samples are split into k batches. The first (N%k) batches contain (N/k)+1 samples, while the remaining batches contain (N/k) samples. In case the number of samples (N) in the dataset is a multiple of k, all batches will contain (N/k) samples.- Parameters:
k
- number of folds (optional, defaults to 10)allData
- DataSet to split into k folds
-
-
Method Detail
-
next
public DataSet next(int num) throws UnsupportedOperationException
Description copied from interface:DataSetIterator
Like the standard next method but allows a customizable number of examples returned- Specified by:
next
in interfaceDataSetIterator
- Parameters:
num
- the number of examples- Returns:
- the next data applyTransformToDestination
- Throws:
UnsupportedOperationException
-
totalExamples
public int totalExamples()
Returns total number of examples in the dataset (all k folds)- Returns:
- total number of examples in the dataset including all k folds
-
inputColumns
public int inputColumns()
Description copied from interface:DataSetIterator
Input columns for the dataset- Specified by:
inputColumns
in interfaceDataSetIterator
- Returns:
-
totalOutcomes
public int totalOutcomes()
Description copied from interface:DataSetIterator
The number of labels for the dataset- Specified by:
totalOutcomes
in interfaceDataSetIterator
- Returns:
-
resetSupported
public boolean resetSupported()
Description copied from interface:DataSetIterator
Is resetting supported by this DataSetIterator? Many DataSetIterators do support resetting, but some don't- Specified by:
resetSupported
in interfaceDataSetIterator
- Returns:
- true if reset method is supported; false otherwise
-
asyncSupported
public boolean asyncSupported()
Description copied from interface:DataSetIterator
Does this DataSetIterator support asynchronous prefetching of multiple DataSet objects? Most DataSetIterators do, but in some cases it may not make sense to wrap this iterator in an iterator that does asynchronous prefetching. For example, it would not make sense to use asynchronous prefetching for the following types of iterators: (a) Iterators that store their full contents in memory already (b) Iterators that re-use features/labels arrays (as future next() calls will overwrite past contents) (c) Iterators that already implement some level of asynchronous prefetching (d) Iterators that may return different data depending on when the next() method is called- Specified by:
asyncSupported
in interfaceDataSetIterator
- Returns:
- true if asynchronous prefetching from this iterator is OK; false if asynchronous prefetching should not be used with this iterator
-
reset
public void reset()
Shuffles the dataset and resets to the first fold- Specified by:
reset
in interfaceDataSetIterator
-
batch
public int batch()
The number of examples in every fold is (N / k), except when (N % k) > 0, when the first (N % k) folds contain (N / k) + 1 examples- Specified by:
batch
in interfaceDataSetIterator
- Returns:
- examples in a fold
-
setPreProcessor
public void setPreProcessor(DataSetPreProcessor preProcessor)
Description copied from interface:DataSetIterator
Set a pre processor- Specified by:
setPreProcessor
in interfaceDataSetIterator
- Parameters:
preProcessor
- a pre processor to set
-
getPreProcessor
public DataSetPreProcessor getPreProcessor()
Description copied from interface:DataSetIterator
Returns preprocessors, if defined- Specified by:
getPreProcessor
in interfaceDataSetIterator
- Returns:
-
getLabels
public List<String> getLabels()
Description copied from interface:DataSetIterator
Get dataset iterator class labels, if any. Note that implementations are not required to implement this, and can simply return null- Specified by:
getLabels
in interfaceDataSetIterator
-
nextFold
protected void nextFold()
-
testFold
public DataSet testFold()
- Returns:
- the held out fold as a dataset
-
-