Modifier and Type | Method and Description |
---|---|
<K,V> DataSource<Tuple2<K,V>> |
ExecutionEnvironment.createHadoopInput(org.apache.hadoop.mapreduce.InputFormat<K,V> mapreduceInputFormat,
Class<K> key,
Class<V> value,
org.apache.hadoop.mapreduce.Job job)
Creates a
DataSet from the given InputFormat . |
<K,V> DataSource<Tuple2<K,V>> |
ExecutionEnvironment.createHadoopInput(org.apache.hadoop.mapred.InputFormat<K,V> mapredInputFormat,
Class<K> key,
Class<V> value,
org.apache.hadoop.mapred.JobConf job)
Creates a
DataSet from the given InputFormat . |
<K,V> DataSource<Tuple2<K,V>> |
ExecutionEnvironment.readHadoopFile(org.apache.hadoop.mapred.FileInputFormat<K,V> mapredInputFormat,
Class<K> key,
Class<V> value,
String inputPath)
Creates a
DataSet from the given FileInputFormat . |
<K,V> DataSource<Tuple2<K,V>> |
ExecutionEnvironment.readHadoopFile(org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V> mapreduceInputFormat,
Class<K> key,
Class<V> value,
String inputPath)
Creates a
DataSet from the given FileInputFormat . |
<K,V> DataSource<Tuple2<K,V>> |
ExecutionEnvironment.readHadoopFile(org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V> mapreduceInputFormat,
Class<K> key,
Class<V> value,
String inputPath,
org.apache.hadoop.mapreduce.Job job)
Creates a
DataSet from the given FileInputFormat . |
<K,V> DataSource<Tuple2<K,V>> |
ExecutionEnvironment.readHadoopFile(org.apache.hadoop.mapred.FileInputFormat<K,V> mapredInputFormat,
Class<K> key,
Class<V> value,
String inputPath,
org.apache.hadoop.mapred.JobConf job)
Creates a
DataSet from the given FileInputFormat . |
<K,V> DataSource<Tuple2<K,V>> |
ExecutionEnvironment.readSequenceFile(Class<K> key,
Class<V> value,
String inputPath)
|
Modifier and Type | Method and Description |
---|---|
Tuple2<K,V> |
HadoopInputFormat.nextRecord(Tuple2<K,V> record) |
Modifier and Type | Method and Description |
---|---|
TypeInformation<Tuple2<K,V>> |
HadoopInputFormat.getProducedType() |
Modifier and Type | Method and Description |
---|---|
Tuple2<K,V> |
HadoopInputFormat.nextRecord(Tuple2<K,V> record) |
void |
HadoopOutputFormat.writeRecord(Tuple2<K,V> record) |
Modifier and Type | Method and Description |
---|---|
Tuple2<K,V> |
HadoopInputFormat.nextRecord(Tuple2<K,V> record) |
Modifier and Type | Method and Description |
---|---|
TypeInformation<Tuple2<K,V>> |
HadoopInputFormat.getProducedType() |
Modifier and Type | Method and Description |
---|---|
Tuple2<K,V> |
HadoopInputFormat.nextRecord(Tuple2<K,V> record) |
void |
HadoopOutputFormat.writeRecord(Tuple2<K,V> record) |
Modifier and Type | Method and Description |
---|---|
<T0,T1> DataSource<Tuple2<T0,T1>> |
CsvReader.types(Class<T0> type0,
Class<T1> type1)
Specifies the types for the CSV fields.
|
Modifier and Type | Method and Description |
---|---|
Tuple2<T1,T2> |
CrossOperator.DefaultCrossFunction.cross(T1 first,
T2 second) |
Modifier and Type | Method and Description |
---|---|
<T0,T1> ProjectOperator<T,Tuple2<T0,T1>> |
ProjectOperator.Projection.projectTuple2()
|
<T0,T1> CrossOperator.ProjectCross<I1,I2,Tuple2<T0,T1>> |
CrossOperator.CrossProjection.projectTuple2()
Projects a pair of crossed elements to a
Tuple with the previously selected fields. |
<T0,T1> JoinOperator.ProjectJoin<I1,I2,Tuple2<T0,T1>> |
JoinOperator.JoinProjection.projectTuple2()
Projects a pair of joined elements to a
Tuple with the previously selected fields. |
Modifier and Type | Method and Description |
---|---|
void |
JoinOperator.DefaultFlatJoinFunction.join(T1 first,
T2 second,
Collector<Tuple2<T1,T2>> out) |
Modifier and Type | Method and Description |
---|---|
Tuple2<K,T> |
KeyExtractingMapper.map(T value) |
Tuple2<K,T> |
PlanUnwrappingReduceOperator.ReduceWrapper.reduce(Tuple2<K,T> value1,
Tuple2<K,T> value2) |
Modifier and Type | Method and Description |
---|---|
void |
TupleRightUnwrappingJoiner.join(I1 value1,
Tuple2<K,I2> value2,
Collector<OUT> collector) |
void |
TupleLeftUnwrappingJoiner.join(Tuple2<K,I1> value1,
I2 value2,
Collector<OUT> collector) |
void |
TupleUnwrappingJoiner.join(Tuple2<K,I1> value1,
Tuple2<K,I2> value2,
Collector<OUT> collector) |
void |
TupleUnwrappingJoiner.join(Tuple2<K,I1> value1,
Tuple2<K,I2> value2,
Collector<OUT> collector) |
T |
KeyRemovingMapper.map(Tuple2<K,T> value) |
Tuple2<K,T> |
PlanUnwrappingReduceOperator.ReduceWrapper.reduce(Tuple2<K,T> value1,
Tuple2<K,T> value2) |
Tuple2<K,T> |
PlanUnwrappingReduceOperator.ReduceWrapper.reduce(Tuple2<K,T> value1,
Tuple2<K,T> value2) |
Modifier and Type | Method and Description |
---|---|
void |
PlanRightUnwrappingCoGroupOperator.TupleRightUnwrappingCoGrouper.coGroup(Iterable<I1> records1,
Iterable<Tuple2<K,I2>> records2,
Collector<OUT> out) |
void |
PlanLeftUnwrappingCoGroupOperator.TupleLeftUnwrappingCoGrouper.coGroup(Iterable<Tuple2<K,I1>> records1,
Iterable<I2> records2,
Collector<OUT> out) |
void |
PlanBothUnwrappingCoGroupOperator.TupleBothUnwrappingCoGrouper.coGroup(Iterable<Tuple2<K,I1>> records1,
Iterable<Tuple2<K,I2>> records2,
Collector<OUT> out) |
void |
PlanBothUnwrappingCoGroupOperator.TupleBothUnwrappingCoGrouper.coGroup(Iterable<Tuple2<K,I1>> records1,
Iterable<Tuple2<K,I2>> records2,
Collector<OUT> out) |
void |
PlanUnwrappingGroupCombineOperator.TupleUnwrappingGroupCombiner.combine(Iterable<Tuple2<K,IN>> values,
Collector<OUT> out) |
void |
PlanUnwrappingReduceGroupOperator.TupleUnwrappingGroupCombinableGroupReducer.combine(Iterable<Tuple2<K,IN>> values,
Collector<Tuple2<K,IN>> out) |
void |
PlanUnwrappingReduceGroupOperator.TupleUnwrappingGroupCombinableGroupReducer.combine(Iterable<Tuple2<K,IN>> values,
Collector<Tuple2<K,IN>> out) |
void |
PlanUnwrappingReduceGroupOperator.TupleUnwrappingGroupCombinableGroupReducer.reduce(Iterable<Tuple2<K,IN>> values,
Collector<OUT> out) |
void |
PlanUnwrappingReduceGroupOperator.TupleUnwrappingNonCombinableGroupReducer.reduce(Iterable<Tuple2<K,IN>> values,
Collector<OUT> out) |
void |
TupleWrappingCollector.set(Collector<Tuple2<K,IN>> wrappedCollector) |
void |
TupleUnwrappingIterator.set(Iterator<Tuple2<K,T>> iterator) |
Modifier and Type | Method and Description |
---|---|
Tuple2<T0,T1> |
Tuple2.copy()
Shallow tuple copy.
|
static <T0,T1> Tuple2<T0,T1> |
Tuple2.of(T0 value0,
T1 value1)
Creates a new tuple and assigns the given values to the tuple's fields.
|
Tuple2<T1,T0> |
Tuple2.swap()
Returns a shallow copy of the tuple with swapped values.
|
Modifier and Type | Method and Description |
---|---|
Tuple2<T0,T1>[] |
Tuple2Builder.build() |
Modifier and Type | Method and Description |
---|---|
static <T> DataSet<Tuple2<Long,T>> |
DataSetUtils.zipWithIndex(DataSet<T> input)
Method that assigns a unique
Long value to all elements in the input data set. |
static <T> DataSet<Tuple2<Long,T>> |
DataSetUtils.zipWithUniqueId(DataSet<T> input)
Method that assigns a unique
Long value to all elements in the input data set in the following way:
a map function is applied to the input data set
each map task holds a counter c which is increased for each record
c is shifted by n bits where n = log2(number of parallel tasks)
to create a unique ID among all tasks, the task id is added to the counter
for each record, the resulting counter is collected
|
Copyright © 2014–2016 The Apache Software Foundation. All rights reserved.