| Package | Description |
|---|---|
| org.apache.flink.api.java | |
| org.apache.flink.api.java.io | |
| org.apache.flink.api.java.operators |
| Modifier and Type | Method and Description |
|---|---|
<X> DataSet<X> |
DataSet.runOperation(CustomUnaryOperation<T,X> operation)
Runs a
CustomUnaryOperation on the data set. |
| Modifier and Type | Method and Description |
|---|---|
protected static void |
DataSet.checkSameExecutionContext(DataSet<?> set1,
DataSet<?> set2) |
protected static void |
DataSet.checkSameExecutionContext(DataSet<?> set1,
DataSet<?> set2) |
<R> CoGroupOperator.CoGroupOperatorSets<T,R> |
DataSet.coGroup(DataSet<R> other)
Initiates a CoGroup transformation.
A CoGroup transformation combines the elements of two DataSets into one DataSet. |
<R> CrossOperator.DefaultCross<T,R> |
DataSet.cross(DataSet<R> other)
Initiates a Cross transformation.
A Cross transformation combines the elements of two DataSets into one DataSet. |
<R> CrossOperator.DefaultCross<T,R> |
DataSet.crossWithHuge(DataSet<R> other)
Initiates a Cross transformation.
A Cross transformation combines the elements of two DataSets into one DataSet. |
<R> CrossOperator.DefaultCross<T,R> |
DataSet.crossWithTiny(DataSet<R> other)
Initiates a Cross transformation.
A Cross transformation combines the elements of two DataSets into one DataSet. |
<R> DeltaIteration<T,R> |
DataSet.iterateDelta(DataSet<R> workset,
int maxIterations,
int... keyPositions)
Initiates a delta iteration.
|
<R> JoinOperator.JoinOperatorSets<T,R> |
DataSet.join(DataSet<R> other)
Initiates a Join transformation.
|
<R> JoinOperator.JoinOperatorSets<T,R> |
DataSet.join(DataSet<R> other,
JoinOperatorBase.JoinHint strategy)
Initiates a Join transformation.
|
<R> JoinOperator.JoinOperatorSets<T,R> |
DataSet.joinWithHuge(DataSet<R> other)
Initiates a Join transformation.
A Join transformation joins the elements of two DataSets on key equality and provides multiple ways to combine
joining elements into one DataSet.
This method also gives the hint to the optimizer that the second DataSet to join is much
larger than the first one.
This method returns a JoinOperator.JoinOperatorSets on which one of the where methods
can be called to define the join key of the first joining (i.e., this) DataSet. |
<R> JoinOperator.JoinOperatorSets<T,R> |
DataSet.joinWithTiny(DataSet<R> other)
Initiates a Join transformation.
|
UnionOperator<T> |
DataSet.union(DataSet<T> other)
Creates a union of this DataSet with an other DataSet.
|
| Modifier and Type | Method and Description |
|---|---|
static <T> void |
RemoteCollectorImpl.collectLocal(DataSet<T> source,
Collection<T> collection)
Writes a DataSet to a local
Collection through an
RemoteCollector and a standard RemoteCollectorConsumer
implementation remotely called from the
RemoteCollectorOutputFormat. |
static <T> DataSink<T> |
RemoteCollectorImpl.collectLocal(DataSet<T> source,
RemoteCollectorConsumer<T> consumer)
Writes a DataSet to a
RemoteCollectorConsumer through an
RemoteCollector remotely called from the
RemoteCollectorOutputFormat. |
| Modifier and Type | Class and Description |
|---|---|
class |
AggregateOperator<IN>
This operator represents the application of a "aggregate" operation on a data set, and the
result data set produced by the function.
|
class |
BulkIterationResultSet<T> |
class |
CoGroupOperator<I1,I2,OUT>
A
DataSet that is the result of a CoGroup transformation. |
class |
CrossOperator<I1,I2,OUT>
A
DataSet that is the result of a Cross transformation. |
static class |
CrossOperator.DefaultCross<I1,I2>
|
static class |
CrossOperator.ProjectCross<I1,I2,OUT extends Tuple>
|
class |
DataSource<OUT>
An operation that creates a new data set (data source).
|
static class |
DeltaIteration.SolutionSetPlaceHolder<ST>
A
DataSet that acts as a placeholder for the solution set during the iteration. |
static class |
DeltaIteration.WorksetPlaceHolder<WT>
A
DataSet that acts as a placeholder for the workset during the iteration. |
class |
DeltaIterationResultSet<ST,WT> |
class |
DistinctOperator<T>
This operator represents the application of a "distinct" function on a data set, and the
result data set produced by the function.
|
class |
FilterOperator<T>
This operator represents the application of a "filter" function on a data set, and the
result data set produced by the function.
|
class |
FlatMapOperator<IN,OUT>
This operator represents the application of a "flatMap" function on a data set, and the
result data set produced by the function.
|
class |
GroupReduceOperator<IN,OUT>
This operator represents the application of a "reduceGroup" function on a data set, and the
result data set produced by the function.
|
class |
IterativeDataSet<T>
The IterativeDataSet represents the start of an iteration.
|
class |
JoinOperator<I1,I2,OUT>
A
DataSet that is the result of a Join transformation. |
static class |
JoinOperator.DefaultJoin<I1,I2>
|
static class |
JoinOperator.EquiJoin<I1,I2,OUT>
A Join transformation that applies a
JoinFunction on each pair of joining elements.It also represents the DataSet that is the result of a Join transformation. |
static class |
JoinOperator.ProjectJoin<I1,I2,OUT extends Tuple>
|
class |
MapOperator<IN,OUT>
This operator represents the application of a "map" function on a data set, and the
result data set produced by the function.
|
class |
MapPartitionOperator<IN,OUT>
This operator represents the application of a "mapPartition" function on a data set, and the
result data set produced by the function.
|
class |
Operator<OUT,O extends Operator<OUT,O>>
Base class of all operators in the Java API.
|
class |
PartitionOperator<T>
This operator represents a partitioning.
|
class |
ProjectOperator<IN,OUT extends Tuple>
This operator represents the application of a projection operation on a data set, and the
result data set produced by the function.
|
class |
ReduceOperator<IN>
This operator represents the application of a "reduce" function on a data set, and the
result data set produced by the function.
|
class |
SingleInputOperator<IN,OUT,O extends SingleInputOperator<IN,OUT,O>>
Base class for operations that operates on a single input data set.
|
class |
SingleInputUdfOperator<IN,OUT,O extends SingleInputUdfOperator<IN,OUT,O>>
The SingleInputUdfOperator is the base class of all unary operators that execute
user-defined functions (UDFs).
|
class |
TwoInputOperator<IN1,IN2,OUT,O extends TwoInputOperator<IN1,IN2,OUT,O>>
Base class for operations that operates on two input data sets.
|
class |
TwoInputUdfOperator<IN1,IN2,OUT,O extends TwoInputUdfOperator<IN1,IN2,OUT,O>>
The TwoInputUdfOperator is the base class of all binary operators that execute
user-defined functions (UDFs).
|
class |
UnionOperator<T>
Java API operator for union of two data sets
|
| Modifier and Type | Field and Description |
|---|---|
protected DataSet<T> |
Grouping.dataSet |
| Modifier and Type | Method and Description |
|---|---|
DataSet<ST> |
DeltaIteration.closeWith(DataSet<ST> solutionSetDelta,
DataSet<WT> newWorkset)
Closes the delta iteration.
|
DataSet<T> |
IterativeDataSet.closeWith(DataSet<T> iterationResult)
Closes the iteration.
|
DataSet<T> |
IterativeDataSet.closeWith(DataSet<T> iterationResult,
DataSet<?> terminationCriterion)
Closes the iteration and specifies a termination criterion.
|
DataSet<OUT> |
CustomUnaryOperation.createResult() |
DataSet<T> |
Grouping.getDataSet() |
DataSet<T> |
DataSink.getDataSet() |
DataSet<ST> |
DeltaIteration.getInitialSolutionSet()
Gets the initial solution set.
|
DataSet<WT> |
DeltaIteration.getInitialWorkset()
Gets the initial workset.
|
DataSet<IN> |
SingleInputOperator.getInput()
Gets the data set that this operation uses as its input.
|
DataSet<IN1> |
TwoInputOperator.getInput1()
Gets the data set that this operation uses as its first input.
|
DataSet<IN2> |
TwoInputOperator.getInput2()
Gets the data set that this operation uses as its second input.
|
DataSet<T> |
BulkIterationResultSet.getNextPartialSolution() |
DataSet<ST> |
DeltaIterationResultSet.getNextSolutionSet() |
DataSet<WT> |
DeltaIterationResultSet.getNextWorkset() |
DataSet<?> |
BulkIterationResultSet.getTerminationCriterion() |
| Modifier and Type | Method and Description |
|---|---|
Map<String,DataSet<?>> |
TwoInputUdfOperator.getBroadcastSets() |
Map<String,DataSet<?>> |
SingleInputUdfOperator.getBroadcastSets() |
Map<String,DataSet<?>> |
UdfOperator.getBroadcastSets()
Gets the broadcast sets (name and data set) that have been added to context of the UDF.
|
| Modifier and Type | Method and Description |
|---|---|
DataSet<ST> |
DeltaIteration.closeWith(DataSet<ST> solutionSetDelta,
DataSet<WT> newWorkset)
Closes the delta iteration.
|
DataSet<ST> |
DeltaIteration.closeWith(DataSet<ST> solutionSetDelta,
DataSet<WT> newWorkset)
Closes the delta iteration.
|
DataSet<T> |
IterativeDataSet.closeWith(DataSet<T> iterationResult)
Closes the iteration.
|
DataSet<T> |
IterativeDataSet.closeWith(DataSet<T> iterationResult,
DataSet<?> terminationCriterion)
Closes the iteration and specifies a termination criterion.
|
DataSet<T> |
IterativeDataSet.closeWith(DataSet<T> iterationResult,
DataSet<?> terminationCriterion)
Closes the iteration and specifies a termination criterion.
|
void |
CustomUnaryOperation.setInput(DataSet<IN> inputData) |
O |
TwoInputUdfOperator.withBroadcastSet(DataSet<?> data,
String name) |
O |
SingleInputUdfOperator.withBroadcastSet(DataSet<?> data,
String name) |
O |
UdfOperator.withBroadcastSet(DataSet<?> data,
String name)
Adds a certain data set as a broadcast set to this operator.
|
| Constructor and Description |
|---|
AggregateOperator(DataSet<IN> input,
Aggregations function,
int field,
String aggregateLocationName)
Non grouped aggregation
|
CoGroupOperator.CoGroupOperatorSets(DataSet<I1> input1,
DataSet<I2> input2) |
CoGroupOperator.CoGroupOperatorSets(DataSet<I1> input1,
DataSet<I2> input2) |
CoGroupOperator(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
CoGroupFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
List<org.apache.commons.lang3.tuple.Pair<Integer,Order>> groupSortKeyOrderFirst,
List<org.apache.commons.lang3.tuple.Pair<Integer,Order>> groupSortKeyOrderSecond,
Partitioner<?> customPartitioner,
String defaultName) |
CoGroupOperator(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
CoGroupFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
List<org.apache.commons.lang3.tuple.Pair<Integer,Order>> groupSortKeyOrderFirst,
List<org.apache.commons.lang3.tuple.Pair<Integer,Order>> groupSortKeyOrderSecond,
Partitioner<?> customPartitioner,
String defaultName) |
CoGroupOperator(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
CoGroupFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
Partitioner<?> customPartitioner,
String defaultName) |
CoGroupOperator(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
CoGroupFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
Partitioner<?> customPartitioner,
String defaultName) |
CrossOperator.CrossProjection(DataSet<I1> ds1,
DataSet<I2> ds2,
int[] firstFieldIndexes,
int[] secondFieldIndexes) |
CrossOperator.CrossProjection(DataSet<I1> ds1,
DataSet<I2> ds2,
int[] firstFieldIndexes,
int[] secondFieldIndexes) |
CrossOperator.DefaultCross(DataSet<I1> input1,
DataSet<I2> input2,
String defaultName) |
CrossOperator.DefaultCross(DataSet<I1> input1,
DataSet<I2> input2,
String defaultName) |
CrossOperator.ProjectCross(DataSet<I1> input1,
DataSet<I2> input2,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType) |
CrossOperator.ProjectCross(DataSet<I1> input1,
DataSet<I2> input2,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType) |
CrossOperator.ProjectCross(DataSet<I1> input1,
DataSet<I2> input2,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType,
CrossOperator.CrossProjection<I1,I2> crossProjection) |
CrossOperator.ProjectCross(DataSet<I1> input1,
DataSet<I2> input2,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType,
CrossOperator.CrossProjection<I1,I2> crossProjection) |
CrossOperator(DataSet<I1> input1,
DataSet<I2> input2,
CrossFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
String defaultName) |
CrossOperator(DataSet<I1> input1,
DataSet<I2> input2,
CrossFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
String defaultName) |
DataSink(DataSet<T> data,
OutputFormat<T> format,
TypeInformation<T> type) |
DeltaIteration(ExecutionEnvironment context,
TypeInformation<ST> type,
DataSet<ST> solutionSet,
DataSet<WT> workset,
Keys<ST> keys,
int maxIterations) |
DeltaIteration(ExecutionEnvironment context,
TypeInformation<ST> type,
DataSet<ST> solutionSet,
DataSet<WT> workset,
Keys<ST> keys,
int maxIterations) |
DistinctOperator(DataSet<T> input,
Keys<T> keys,
String distinctLocationName) |
FilterOperator(DataSet<T> input,
FilterFunction<T> function,
String defaultName) |
FlatMapOperator(DataSet<IN> input,
TypeInformation<OUT> resultType,
FlatMapFunction<IN,OUT> function,
String defaultName) |
Grouping(DataSet<T> set,
Keys<T> keys) |
GroupReduceOperator(DataSet<IN> input,
TypeInformation<OUT> resultType,
GroupReduceFunction<IN,OUT> function,
String defaultName)
Constructor for a non-grouped reduce (all reduce).
|
IterativeDataSet(ExecutionEnvironment context,
TypeInformation<T> type,
DataSet<T> input,
int maxIterations) |
JoinOperator.DefaultJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
JoinOperatorBase.JoinHint hint,
String joinLocationName) |
JoinOperator.DefaultJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
JoinOperatorBase.JoinHint hint,
String joinLocationName) |
JoinOperator.EquiJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
FlatJoinFunction<I1,I2,OUT> generatedFunction,
JoinFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
String joinLocationName) |
JoinOperator.EquiJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
FlatJoinFunction<I1,I2,OUT> generatedFunction,
JoinFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
String joinLocationName) |
JoinOperator.EquiJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
FlatJoinFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
String joinLocationName) |
JoinOperator.EquiJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
FlatJoinFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
String joinLocationName) |
JoinOperator.JoinOperatorSets(DataSet<I1> input1,
DataSet<I2> input2) |
JoinOperator.JoinOperatorSets(DataSet<I1> input1,
DataSet<I2> input2) |
JoinOperator.JoinOperatorSets(DataSet<I1> input1,
DataSet<I2> input2,
JoinOperatorBase.JoinHint hint) |
JoinOperator.JoinOperatorSets(DataSet<I1> input1,
DataSet<I2> input2,
JoinOperatorBase.JoinHint hint) |
JoinOperator.JoinProjection(DataSet<I1> ds1,
DataSet<I2> ds2,
Keys<I1> keys1,
Keys<I2> keys2,
JoinOperatorBase.JoinHint hint,
int[] firstFieldIndexes,
int[] secondFieldIndexes) |
JoinOperator.JoinProjection(DataSet<I1> ds1,
DataSet<I2> ds2,
Keys<I1> keys1,
Keys<I2> keys2,
JoinOperatorBase.JoinHint hint,
int[] firstFieldIndexes,
int[] secondFieldIndexes) |
JoinOperator.ProjectJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
JoinOperatorBase.JoinHint hint,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType) |
JoinOperator.ProjectJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
JoinOperatorBase.JoinHint hint,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType) |
JoinOperator.ProjectJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
JoinOperatorBase.JoinHint hint,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType,
JoinOperator.JoinProjection<I1,I2> joinProj) |
JoinOperator.ProjectJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
JoinOperatorBase.JoinHint hint,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType,
JoinOperator.JoinProjection<I1,I2> joinProj) |
JoinOperator(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint) |
JoinOperator(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint) |
MapOperator(DataSet<IN> input,
TypeInformation<OUT> resultType,
MapFunction<IN,OUT> function,
String defaultName) |
MapPartitionOperator(DataSet<IN> input,
TypeInformation<OUT> resultType,
MapPartitionFunction<IN,OUT> function,
String defaultName) |
PartitionOperator(DataSet<T> input,
Keys<T> pKeys,
Partitioner<?> customPartitioner,
String partitionLocationName) |
PartitionOperator(DataSet<T> input,
Keys<T> pKeys,
Partitioner<P> customPartitioner,
TypeInformation<P> partitionerTypeInfo,
String partitionLocationName) |
PartitionOperator(DataSet<T> input,
PartitionOperatorBase.PartitionMethod pMethod,
Keys<T> pKeys,
String partitionLocationName) |
PartitionOperator(DataSet<T> input,
PartitionOperatorBase.PartitionMethod pMethod,
String partitionLocationName) |
ProjectOperator.Projection(DataSet<T> ds,
int[] fieldIndexes) |
ProjectOperator(DataSet<IN> input,
int[] fields,
TupleTypeInfo<OUT> returnType) |
ProjectOperator(DataSet<IN> input,
int[] fields,
TupleTypeInfo<OUT> returnType,
ProjectOperator.Projection<IN> proj) |
ReduceOperator(DataSet<IN> input,
ReduceFunction<IN> function,
String defaultName)
This is the case for a reduce-all case (in contrast to the reduce-per-group case).
|
SingleInputOperator(DataSet<IN> input,
TypeInformation<OUT> resultType) |
SingleInputUdfOperator(DataSet<IN> input,
TypeInformation<OUT> resultType)
Creates a new operators with the given data set as input.
|
SortedGrouping(DataSet<T> set,
Keys<T> keys,
int field,
Order order) |
SortedGrouping(DataSet<T> set,
Keys<T> keys,
String field,
Order order) |
TwoInputOperator(DataSet<IN1> input1,
DataSet<IN2> input2,
TypeInformation<OUT> resultType) |
TwoInputOperator(DataSet<IN1> input1,
DataSet<IN2> input2,
TypeInformation<OUT> resultType) |
TwoInputUdfOperator(DataSet<IN1> input1,
DataSet<IN2> input2,
TypeInformation<OUT> resultType)
Creates a new operators with the two given data sets as inputs.
|
TwoInputUdfOperator(DataSet<IN1> input1,
DataSet<IN2> input2,
TypeInformation<OUT> resultType)
Creates a new operators with the two given data sets as inputs.
|
UnionOperator(DataSet<T> input1,
DataSet<T> input2,
String unionLocationName)
Create an operator that produces the union of the two given data sets.
|
UnionOperator(DataSet<T> input1,
DataSet<T> input2,
String unionLocationName)
Create an operator that produces the union of the two given data sets.
|
UnsortedGrouping(DataSet<T> set,
Keys<T> keys) |
Copyright © 2015 The Apache Software Foundation. All rights reserved.