org.deeplearning4j.iterator.provider

Class FileLabeledSentenceProvider

java.lang.Object

org.deeplearning4j.iterator.provider.FileLabeledSentenceProvider

All Implemented Interfaces:

LabeledSentenceProvider

public class FileLabeledSentenceProvider
extends Object
implements LabeledSentenceProvider

Iterate over a set of sentences/documents, where the sentences are to be loaded (as required) from the provided files.

Author:

Alex Black

Constructor Summary

Constructors

Constructor and Description
FileLabeledSentenceProvider(Map<String,List<File>> filesByLabel)
FileLabeledSentenceProvider(Map<String,List<File>> filesByLabel, Random rng)

Method Summary

Modifier and Type	Method and Description
List<String>	allLabels() Return the list of labels - this also defines the class/integer label assignment order
boolean	hasNext() Are there more sentences/documents available?
org.nd4j.linalg.primitives.Pair<String,String>	nextSentence()
int	numLabelClasses() Equivalent to allLabels().size()
void	reset() Reset the iterator - including shuffling the order, if necessary/appropriate
int	totalNumSentences() Return the total number of sentences, or -1 if not available

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FileLabeledSentenceProvider

public FileLabeledSentenceProvider(Map<String,List<File>> filesByLabel)

Parameters:

filesByLabel - Key: label. Value: list of files for that label

FileLabeledSentenceProvider

public FileLabeledSentenceProvider(@NonNull
                                   Map<String,List<File>> filesByLabel,
                                   Random rng)

Parameters:

filesByLabel - Key: label. Value: list of files for that label

rng - Random number generator. May be null.

Method Detail

hasNext

public boolean hasNext()

Description copied from interface: LabeledSentenceProvider

Are there more sentences/documents available?

Specified by:

hasNext in interface LabeledSentenceProvider

nextSentence

public org.nd4j.linalg.primitives.Pair<String,String> nextSentence()

Specified by:

nextSentence in interface LabeledSentenceProvider

Returns:

Pair: sentence/document text and label

reset

public void reset()

Description copied from interface: LabeledSentenceProvider

Reset the iterator - including shuffling the order, if necessary/appropriate

Specified by:

reset in interface LabeledSentenceProvider

totalNumSentences

public int totalNumSentences()

Description copied from interface: LabeledSentenceProvider

Return the total number of sentences, or -1 if not available

Specified by:

totalNumSentences in interface LabeledSentenceProvider

allLabels

public List<String> allLabels()

Description copied from interface: LabeledSentenceProvider

Return the list of labels - this also defines the class/integer label assignment order

Specified by:

allLabels in interface LabeledSentenceProvider

numLabelClasses

public int numLabelClasses()

Description copied from interface: LabeledSentenceProvider

Equivalent to allLabels().size()

Specified by:

numLabelClasses in interface LabeledSentenceProvider