org.apache.flink.api.java.io

Class CsvInputFormat<OUT extends Tuple>

java.lang.Object

org.apache.flink.api.common.io.FileInputFormat<OT>

org.apache.flink.api.common.io.DelimitedInputFormat<OT>

org.apache.flink.api.common.io.GenericCsvInputFormat<OUT>

org.apache.flink.api.java.io.CsvInputFormat<OUT>

All Implemented Interfaces:

Serializable, InputFormat<OUT,FileInputSplit>, InputSplitSource<FileInputSplit>

public class CsvInputFormat<OUT extends Tuple>
extends GenericCsvInputFormat<OUT>

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.DelimitedInputFormat

DelimitedInputFormat.AbstractConfigBuilder<T>, DelimitedInputFormat.ConfigBuilder

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.FileInputFormat

FileInputFormat.FileBaseStatistics, FileInputFormat.InputSplitOpenThread

Field Summary

Fields

Modifier and Type	Field and Description
static char	DEFAULT_FIELD_DELIMITER
static String	DEFAULT_LINE_DELIMITER

Fields inherited from class org.apache.flink.api.common.io.DelimitedInputFormat

RECORD_DELIMITER

Fields inherited from class org.apache.flink.api.common.io.FileInputFormat

DEFLATE_SUFFIX, enumerateNestedFiles, filePath, minSplitSize, numSplits, openTimeout, READ_WHOLE_SPLIT_FLAG, splitLength, splitStart, stream, unsplittable

Constructor Summary

Constructors

Constructor and Description
CsvInputFormat(Path filePath)
CsvInputFormat(Path filePath, Class<?>... types)
CsvInputFormat(Path filePath, String lineDelimiter, char fieldDelimiter, Class<?>... types)

Method Summary

Modifier and Type	Method and Description
void	close()
byte[]	getCommentPrefix()
Class<?>[]	getFieldTypes()
OUT	nextRecord(OUT record)
void	open(FileInputSplit split)
OUT	readRecord(OUT reuse, byte[] bytes, int offset, int numBytes)
void	setCommentPrefix(byte[] commentPrefix)
void	setCommentPrefix(char commentPrefix)
void	setCommentPrefix(String commentPrefix)
void	setCommentPrefix(String commentPrefix, Charset charset)
void	setCommentPrefix(String commentPrefix, String charsetName)
void	setFields(boolean[] sourceFieldMask, Class<?>[] fieldTypes)
void	setFields(int[] sourceFieldIndices, Class<?>[] fieldTypes)
void	setFieldTypes(Class<?>... fieldTypes)
String	toString()

Methods inherited from class org.apache.flink.api.common.io.GenericCsvInputFormat

getFieldDelimiter, getFieldParsers, getGenericFieldTypes, getNumberOfFieldsTotal, getNumberOfNonNullFields, isLenient, isSkippingFirstLineAsHeader, parseRecord, setFieldDelimiter, setFieldsGeneric, setFieldsGeneric, setFieldTypesGeneric, setLenient, setSkipFirstLineAsHeader, skipFields

Methods inherited from class org.apache.flink.api.common.io.DelimitedInputFormat

configure, configureDelimitedFormat, getBufferSize, getDelimiter, getLineLengthLimit, getNumLineSamples, getStatistics, loadGloablConfigParams, reachedEnd, readLine, setBufferSize, setDelimiter, setDelimiter, setDelimiter, setDelimiter, setDelimiter, setLineLengthLimit, setNumLineSamples

Methods inherited from class org.apache.flink.api.common.io.FileInputFormat

acceptFile, configureFileFormat, createInputSplits, getFilePath, getFileStats, getInputSplitAssigner, getMinSplitSize, getNumSplits, getOpenTimeout, getSplitLength, getSplitStart, setFilePath, setFilePath, setMinSplitSize, setNumSplits, setOpenTimeout

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

DEFAULT_LINE_DELIMITER

public static final String DEFAULT_LINE_DELIMITER

See Also:

Constant Field Values

DEFAULT_FIELD_DELIMITER

public static final char DEFAULT_FIELD_DELIMITER

See Also:

Constant Field Values

Constructor Detail

CsvInputFormat

public CsvInputFormat(Path filePath)

CsvInputFormat

public CsvInputFormat(Path filePath,
                      Class<?>... types)

CsvInputFormat

public CsvInputFormat(Path filePath,
                      String lineDelimiter,
                      char fieldDelimiter,
                      Class<?>... types)

Method Detail

getCommentPrefix

public byte[] getCommentPrefix()

setCommentPrefix

public void setCommentPrefix(byte[] commentPrefix)

setCommentPrefix

public void setCommentPrefix(char commentPrefix)

setCommentPrefix

public void setCommentPrefix(String commentPrefix)

setCommentPrefix

public void setCommentPrefix(String commentPrefix,
                             String charsetName)
                      throws IllegalCharsetNameException,
                             UnsupportedCharsetException

Throws:

IllegalCharsetNameException

UnsupportedCharsetException

setCommentPrefix

public void setCommentPrefix(String commentPrefix,
                             Charset charset)

setFieldTypes

public void setFieldTypes(Class<?>... fieldTypes)

setFields

public void setFields(int[] sourceFieldIndices,
                      Class<?>[] fieldTypes)

setFields

public void setFields(boolean[] sourceFieldMask,
                      Class<?>[] fieldTypes)

getFieldTypes

public Class<?>[] getFieldTypes()

open

public void open(FileInputSplit split)
          throws IOException

Specified by:

open in interface InputFormat<OUT extends Tuple,FileInputSplit>

Overrides:

open in class GenericCsvInputFormat<OUT extends Tuple>

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface InputFormat<OUT extends Tuple,FileInputSplit>

Overrides:

close in class DelimitedInputFormat<OUT extends Tuple>

Throws:

IOException

nextRecord

public OUT nextRecord(OUT record)
               throws IOException

Specified by:

nextRecord in interface InputFormat<OUT extends Tuple,FileInputSplit>

Overrides:

nextRecord in class DelimitedInputFormat<OUT extends Tuple>

Throws:

IOException

readRecord

public OUT readRecord(OUT reuse,
                      byte[] bytes,
                      int offset,
                      int numBytes)
               throws IOException

Specified by:

readRecord in class DelimitedInputFormat<OUT extends Tuple>

Throws:

IOException

toString

public String toString()

Overrides:

toString in class FileInputFormat<OUT extends Tuple>