Class SVMLightRecordReader
- java.lang.Object
-
- org.datavec.api.records.reader.BaseRecordReader
-
- org.datavec.api.records.reader.impl.LineRecordReader
-
- org.datavec.api.records.reader.impl.misc.SVMLightRecordReader
-
- All Implemented Interfaces:
Closeable
,Serializable
,AutoCloseable
,Configurable
,RecordReader
- Direct Known Subclasses:
LibSvmRecordReader
public class SVMLightRecordReader extends LineRecordReader
- See Also:
- Serialized Form
-
-
Field Summary
Fields Modifier and Type Field Description static String
ALLOWED_DELIMITERS
protected boolean
appendLabel
static String
COMMENT_CHAR
static String
FEATURE_DELIMITER
static String
LABEL_DELIMITER
static Writable
LABEL_ONE
static Writable
LABEL_ZERO
protected boolean
multilabel
static String
MULTILABEL
static String
NAME_SPACE
static String
NUM_ATTRIBUTES
static String
NUM_FEATURES
static String
NUM_LABELS
protected int
numFeatures
protected int
numLabels
static Writable
ONE
static String
PREFERRED_DELIMITER
static String
QID_PREFIX
protected Writable
recordLookahead
static Writable
ZERO
static String
ZERO_BASED_INDEXING
static String
ZERO_BASED_LABEL_INDEXING
protected boolean
zeroBasedIndexing
protected boolean
zeroBasedLabelIndexing
-
Fields inherited from class org.datavec.api.records.reader.impl.LineRecordReader
charset, conf, initialized, lineIndex, locations, splitIndex
-
Fields inherited from class org.datavec.api.records.reader.BaseRecordReader
inputSplit, listeners, streamCreatorFn
-
Fields inherited from interface org.datavec.api.records.reader.RecordReader
APPEND_LABEL, LABELS
-
-
Constructor Summary
Constructors Constructor Description SVMLightRecordReader()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description protected Writable
getNextRecord()
Helper function to help detect lines that are commented out.boolean
hasNext()
Whether there are anymore recordsvoid
initialize(Configuration conf, InputSplit split)
Must be called before attempting to read records.List<Writable>
next()
Return next record as list of Writables.Record
nextRecord()
Return next Record.protected void
onLocationOpen(URI location)
List<Writable>
record(URI uri, DataInputStream dataInputStream)
Load the record from the given DataInputStream UnlikeRecordReader.next()
the internal state of the RecordReader is not modified Implementations of this method should not close the DataInputStreamvoid
reset()
Reset record reader iteratorvoid
setConf(Configuration conf)
Set configuration.-
Methods inherited from class org.datavec.api.records.reader.impl.LineRecordReader
close, closeIfRequired, getConf, getIterator, getLabels, initialize, loadFromMetaData, loadFromMetaData, resetSupported
-
Methods inherited from class org.datavec.api.records.reader.BaseRecordReader
batchesSupported, getListeners, invokeListeners, next, setListeners, setListeners
-
-
-
-
Field Detail
-
NAME_SPACE
public static final String NAME_SPACE
-
NUM_FEATURES
public static final String NUM_FEATURES
-
ZERO_BASED_INDEXING
public static final String ZERO_BASED_INDEXING
-
ZERO_BASED_LABEL_INDEXING
public static final String ZERO_BASED_LABEL_INDEXING
-
MULTILABEL
public static final String MULTILABEL
-
NUM_LABELS
public static final String NUM_LABELS
-
COMMENT_CHAR
public static final String COMMENT_CHAR
- See Also:
- Constant Field Values
-
ALLOWED_DELIMITERS
public static final String ALLOWED_DELIMITERS
- See Also:
- Constant Field Values
-
PREFERRED_DELIMITER
public static final String PREFERRED_DELIMITER
- See Also:
- Constant Field Values
-
FEATURE_DELIMITER
public static final String FEATURE_DELIMITER
- See Also:
- Constant Field Values
-
LABEL_DELIMITER
public static final String LABEL_DELIMITER
- See Also:
- Constant Field Values
-
QID_PREFIX
public static final String QID_PREFIX
- See Also:
- Constant Field Values
-
ZERO
public static final Writable ZERO
-
ONE
public static final Writable ONE
-
LABEL_ZERO
public static final Writable LABEL_ZERO
-
LABEL_ONE
public static final Writable LABEL_ONE
-
numFeatures
protected int numFeatures
-
zeroBasedIndexing
protected boolean zeroBasedIndexing
-
zeroBasedLabelIndexing
protected boolean zeroBasedLabelIndexing
-
appendLabel
protected boolean appendLabel
-
multilabel
protected boolean multilabel
-
numLabels
protected int numLabels
-
recordLookahead
protected Writable recordLookahead
-
NUM_ATTRIBUTES
public static final String NUM_ATTRIBUTES
-
-
Method Detail
-
initialize
public void initialize(Configuration conf, InputSplit split) throws IOException, InterruptedException
Must be called before attempting to read records.- Specified by:
initialize
in interfaceRecordReader
- Overrides:
initialize
in classLineRecordReader
- Parameters:
conf
- DataVec configurationsplit
- FileSplit- Throws:
IOException
InterruptedException
-
setConf
public void setConf(Configuration conf)
Set configuration.- Specified by:
setConf
in interfaceConfigurable
- Overrides:
setConf
in classLineRecordReader
- Parameters:
conf
- DataVec configuration- Throws:
IOException
InterruptedException
-
getNextRecord
protected Writable getNextRecord()
Helper function to help detect lines that are commented out. May read ahead and cache a line.- Returns:
-
hasNext
public boolean hasNext()
Description copied from interface:RecordReader
Whether there are anymore records- Specified by:
hasNext
in interfaceRecordReader
- Overrides:
hasNext
in classLineRecordReader
- Returns:
-
next
public List<Writable> next()
Return next record as list of Writables.- Specified by:
next
in interfaceRecordReader
- Overrides:
next
in classLineRecordReader
- Returns:
-
nextRecord
public Record nextRecord()
Return next Record.- Specified by:
nextRecord
in interfaceRecordReader
- Overrides:
nextRecord
in classLineRecordReader
- Returns:
-
record
public List<Writable> record(URI uri, DataInputStream dataInputStream) throws IOException
Description copied from interface:RecordReader
Load the record from the given DataInputStream UnlikeRecordReader.next()
the internal state of the RecordReader is not modified Implementations of this method should not close the DataInputStream- Specified by:
record
in interfaceRecordReader
- Overrides:
record
in classLineRecordReader
- Throws:
IOException
- if error occurs during reading from the input stream
-
reset
public void reset()
Description copied from interface:RecordReader
Reset record reader iterator- Specified by:
reset
in interfaceRecordReader
- Overrides:
reset
in classLineRecordReader
-
onLocationOpen
protected void onLocationOpen(URI location)
- Overrides:
onLocationOpen
in classLineRecordReader
-
-