org.apache.flink.api.java.io

Class RowCsvInputFormat

java.lang.Object

org.apache.flink.api.common.io.RichInputFormat<OT,org.apache.flink.core.fs.FileInputSplit>

org.apache.flink.api.common.io.FileInputFormat<OT>

org.apache.flink.api.common.io.DelimitedInputFormat<OT>

org.apache.flink.api.common.io.GenericCsvInputFormat<OUT>

org.apache.flink.api.java.io.CsvInputFormat<org.apache.flink.types.Row>

org.apache.flink.api.java.io.RowCsvInputFormat

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.io.CheckpointableInputFormat<org.apache.flink.core.fs.FileInputSplit,Long>, org.apache.flink.api.common.io.InputFormat<org.apache.flink.types.Row,org.apache.flink.core.fs.FileInputSplit>, org.apache.flink.api.java.typeutils.ResultTypeQueryable<org.apache.flink.types.Row>, org.apache.flink.core.io.InputSplitSource<org.apache.flink.core.fs.FileInputSplit>

@PublicEvolving
public class RowCsvInputFormat
extends CsvInputFormat<org.apache.flink.types.Row>
implements org.apache.flink.api.java.typeutils.ResultTypeQueryable<org.apache.flink.types.Row>

Input format that reads csv into Row.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.FileInputFormat

org.apache.flink.api.common.io.FileInputFormat.FileBaseStatistics, org.apache.flink.api.common.io.FileInputFormat.InputSplitOpenThread

Field Summary

Fields inherited from class org.apache.flink.api.java.io.CsvInputFormat

DEFAULT_FIELD_DELIMITER, DEFAULT_LINE_DELIMITER, parsedValues

Fields inherited from class org.apache.flink.api.common.io.GenericCsvInputFormat

commentCount, commentPrefix, fieldIncluded, invalidLineCount, lineDelimiterIsLinebreak

Fields inherited from class org.apache.flink.api.common.io.DelimitedInputFormat

currBuffer, currLen, currOffset, RECORD_DELIMITER

Fields inherited from class org.apache.flink.api.common.io.FileInputFormat

currentSplit, ENUMERATE_NESTED_FILES_FLAG, enumerateNestedFiles, filePath, INFLATER_INPUT_STREAM_FACTORIES, minSplitSize, numSplits, openTimeout, READ_WHOLE_SPLIT_FLAG, splitLength, splitStart, stream, unsplittable

Constructor Summary

Constructors

Constructor and Description
RowCsvInputFormat(org.apache.flink.core.fs.Path filePath, org.apache.flink.api.common.typeinfo.TypeInformation[] fieldTypes)
RowCsvInputFormat(org.apache.flink.core.fs.Path filePath, org.apache.flink.api.common.typeinfo.TypeInformation[] fieldTypes, boolean emptyColumnAsNull)
RowCsvInputFormat(org.apache.flink.core.fs.Path filePath, org.apache.flink.api.common.typeinfo.TypeInformation[] fieldTypes, int[] selectedFields)
RowCsvInputFormat(org.apache.flink.core.fs.Path filePath, org.apache.flink.api.common.typeinfo.TypeInformation[] fieldTypes, String lineDelimiter, String fieldDelimiter)
RowCsvInputFormat(org.apache.flink.core.fs.Path filePath, org.apache.flink.api.common.typeinfo.TypeInformation[] fieldTypes, String lineDelimiter, String fieldDelimiter, int[] selectedFields)
RowCsvInputFormat(org.apache.flink.core.fs.Path filePath, org.apache.flink.api.common.typeinfo.TypeInformation[] fieldTypeInfos, String lineDelimiter, String fieldDelimiter, int[] selectedFields, boolean emptyColumnAsNull)

Method Summary

Modifier and Type	Method and Description
protected org.apache.flink.types.Row	fillRecord(org.apache.flink.types.Row reuse, Object[] parsedValues)
org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.types.Row>	getProducedType()
protected boolean	parseRecord(Object[] holders, byte[] bytes, int offset, int numBytes)

Methods inherited from class org.apache.flink.api.java.io.CsvInputFormat

createDefaultMask, getFieldTypes, initializeSplit, nextRecord, readRecord, toBooleanMask, toString

Methods inherited from class org.apache.flink.api.common.io.GenericCsvInputFormat

checkAndCoSort, checkForMonotonousOrder, close, enableQuotedStringParsing, getCommentPrefix, getFieldDelimiter, getFieldParsers, getGenericFieldTypes, getNumberOfFieldsTotal, getNumberOfNonNullFields, isLenient, isSkippingFirstLineAsHeader, setCharset, setCommentPrefix, setFieldDelimiter, setFieldsGeneric, setFieldsGeneric, setFieldTypesGeneric, setLenient, setSkipFirstLineAsHeader, skipFields, supportsMultiPaths

Methods inherited from class org.apache.flink.api.common.io.DelimitedInputFormat

configure, getBufferSize, getCharset, getCurrentState, getDelimiter, getLineLengthLimit, getNumLineSamples, getStatistics, loadConfigParameters, loadGlobalConfigParams, open, reachedEnd, readLine, reopen, setBufferSize, setDelimiter, setDelimiter, setDelimiter, setLineLengthLimit, setNumLineSamples

Methods inherited from class org.apache.flink.api.common.io.FileInputFormat

acceptFile, createInputSplits, decorateInputStream, extractFileExtension, getFilePath, getFilePaths, getFileStats, getFileStats, getInflaterInputStreamFactory, getInputSplitAssigner, getMinSplitSize, getNestedFileEnumeration, getNumSplits, getOpenTimeout, getSplitLength, getSplitStart, registerInflaterInputStreamFactory, setFilePath, setFilePath, setFilePaths, setFilePaths, setFilesFilter, setMinSplitSize, setNestedFileEnumeration, setNumSplits, setOpenTimeout, testForUnsplittable

Methods inherited from class org.apache.flink.api.common.io.RichInputFormat

closeInputFormat, getRuntimeContext, openInputFormat, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

RowCsvInputFormat

public RowCsvInputFormat(org.apache.flink.core.fs.Path filePath,
                         org.apache.flink.api.common.typeinfo.TypeInformation[] fieldTypeInfos,
                         String lineDelimiter,
                         String fieldDelimiter,
                         int[] selectedFields,
                         boolean emptyColumnAsNull)

RowCsvInputFormat

public RowCsvInputFormat(org.apache.flink.core.fs.Path filePath,
                         org.apache.flink.api.common.typeinfo.TypeInformation[] fieldTypes,
                         String lineDelimiter,
                         String fieldDelimiter,
                         int[] selectedFields)

RowCsvInputFormat

public RowCsvInputFormat(org.apache.flink.core.fs.Path filePath,
                         org.apache.flink.api.common.typeinfo.TypeInformation[] fieldTypes,
                         String lineDelimiter,
                         String fieldDelimiter)

RowCsvInputFormat

public RowCsvInputFormat(org.apache.flink.core.fs.Path filePath,
                         org.apache.flink.api.common.typeinfo.TypeInformation[] fieldTypes,
                         int[] selectedFields)

RowCsvInputFormat

public RowCsvInputFormat(org.apache.flink.core.fs.Path filePath,
                         org.apache.flink.api.common.typeinfo.TypeInformation[] fieldTypes,
                         boolean emptyColumnAsNull)

RowCsvInputFormat

public RowCsvInputFormat(org.apache.flink.core.fs.Path filePath,
                         org.apache.flink.api.common.typeinfo.TypeInformation[] fieldTypes)

Method Detail

fillRecord

protected org.apache.flink.types.Row fillRecord(org.apache.flink.types.Row reuse,
                                                Object[] parsedValues)

Specified by:

fillRecord in class CsvInputFormat<org.apache.flink.types.Row>

parseRecord

protected boolean parseRecord(Object[] holders,
                              byte[] bytes,
                              int offset,
                              int numBytes)
                       throws org.apache.flink.api.common.io.ParseException

Overrides:

parseRecord in class org.apache.flink.api.common.io.GenericCsvInputFormat<org.apache.flink.types.Row>

Throws:

org.apache.flink.api.common.io.ParseException

getProducedType

public org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.types.Row> getProducedType()

Specified by:

getProducedType in interface org.apache.flink.api.java.typeutils.ResultTypeQueryable<org.apache.flink.types.Row>