org.apache.flink.api.java.operators

Class DataSource<OUT>

java.lang.Object

org.apache.flink.api.java.DataSet<OUT>

org.apache.flink.api.java.operators.Operator<OUT,DataSource<OUT>>

org.apache.flink.api.java.operators.DataSource<OUT>

Type Parameters:

OUT - The type of the elements produced by this data source.

public class DataSource<OUT>
extends Operator<OUT,DataSource<OUT>>

An operation that creates a new data set (data source). The operation acts as the data set on which to apply further transformations. It encapsulates additional configuration parameters, to customize the execution.

Field Summary

Fields inherited from class org.apache.flink.api.java.operators.Operator

name, parallelism

Fields inherited from class org.apache.flink.api.java.DataSet

context

Constructor Summary

Constructors

Constructor and Description
DataSource(ExecutionEnvironment context, InputFormat<OUT,?> inputFormat, TypeInformation<OUT> type, String dataSourceLocationName) Creates a new data source.

Method Summary

Methods

Modifier and Type	Method and Description
InputFormat<OUT,?>	getInputFormat() Gets the input format that is executed by this data source.
Configuration	getParameters()
SplitDataProperties<OUT>	getSplitDataProperties() Returns the SplitDataProperties for the InputSplits of this DataSource for configurations.
protected GenericDataSourceBase<OUT,?>	translateToDataFlow()
DataSource<OUT>	withParameters(Configuration parameters) Pass a configuration to the InputFormat

Methods inherited from class org.apache.flink.api.java.operators.Operator

getName, getParallelism, getResultType, name, setParallelism

Methods inherited from class org.apache.flink.api.java.DataSet

aggregate, checkSameExecutionContext, clean, coGroup, collect, combineGroup, count, cross, crossWithHuge, crossWithTiny, distinct, distinct, distinct, distinct, fillInType, filter, first, flatMap, fullOuterJoin, fullOuterJoin, getExecutionEnvironment, getType, groupBy, groupBy, groupBy, iterate, iterateDelta, join, join, joinWithHuge, joinWithTiny, leftOuterJoin, leftOuterJoin, map, mapPartition, max, maxBy, min, minBy, output, partitionByHash, partitionByHash, partitionByHash, partitionCustom, partitionCustom, partitionCustom, print, print, printOnTaskManager, printToErr, printToErr, project, rebalance, reduce, reduceGroup, rightOuterJoin, rightOuterJoin, runOperation, sortPartition, sortPartition, sum, union, write, write, writeAsCsv, writeAsCsv, writeAsCsv, writeAsCsv, writeAsFormattedText, writeAsFormattedText, writeAsText, writeAsText

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DataSource

public DataSource(ExecutionEnvironment context,
          InputFormat<OUT,?> inputFormat,
          TypeInformation<OUT> type,
          String dataSourceLocationName)

Creates a new data source.

Parameters:

context - The environment in which the data source gets executed.

inputFormat - The input format that the data source executes.

type - The type of the elements produced by this input format.

Method Detail

getInputFormat

public InputFormat<OUT,?> getInputFormat()

Gets the input format that is executed by this data source.

Returns:

The input format that is executed by this data source.

withParameters

public DataSource<OUT> withParameters(Configuration parameters)

Pass a configuration to the InputFormat

Parameters:

parameters - Configuration parameters

getParameters

public Configuration getParameters()

Returns:

Configuration for the InputFormat.

getSplitDataProperties

public SplitDataProperties<OUT> getSplitDataProperties()

Returns the SplitDataProperties for the InputSplits of this DataSource for configurations. SplitDataProperties can help to generate more efficient execution plans.
IMPORTANT: Incorrect configuration of SplitDataProperties can cause wrong results!

Returns:

The SplitDataProperties for the InputSplits of this DataSource.

translateToDataFlow

protected GenericDataSourceBase<OUT,?> translateToDataFlow()