org.apache.flink.streaming.api.datastream

Class ConnectedStreams<IN1,IN2>

java.lang.Object

org.apache.flink.streaming.api.datastream.ConnectedStreams<IN1,IN2>

Type Parameters:

IN1 - Type of the first input data steam.

IN2 - Type of the second input data stream.

Direct Known Subclasses:

IterativeStream.ConnectedIterativeStreams

public class ConnectedStreams<IN1,IN2>
extends Object

ConnectedStreams represents two connected streams of (possible) different data types. It can be used to apply transformations such as CoMapFunction on two DataStreams

Field Summary

Fields

Modifier and Type	Field and Description
protected StreamExecutionEnvironment	environment
protected DataStream<IN1>	inputStream1
protected DataStream<IN2>	inputStream2

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	ConnectedStreams(StreamExecutionEnvironment env, DataStream<IN1> input1, DataStream<IN2> input2)

Method Summary

Modifier and Type	Method and Description
<OUT> SingleOutputStreamOperator<OUT,?>	flatMap(CoFlatMapFunction<IN1,IN2,OUT> coFlatMapper) Applies a CoFlatMap transformation on a ConnectedStreams and maps the output to a common type.
StreamExecutionEnvironment	getExecutionEnvironment()
DataStream<IN1>	getFirstInput() Returns the first DataStream.
DataStream<IN2>	getSecondInput() Returns the second DataStream.
org.apache.flink.api.common.typeinfo.TypeInformation<IN1>	getType1() Gets the type of the first input
org.apache.flink.api.common.typeinfo.TypeInformation<IN2>	getType2() Gets the type of the second input
ConnectedStreams<IN1,IN2>	keyBy(int[] keyPositions1, int[] keyPositions2) KeyBy operation for connected data stream.
ConnectedStreams<IN1,IN2>	keyBy(int keyPosition1, int keyPosition2) KeyBy operation for connected data stream.
ConnectedStreams<IN1,IN2>	keyBy(org.apache.flink.api.java.functions.KeySelector<IN1,?> keySelector1, org.apache.flink.api.java.functions.KeySelector<IN2,?> keySelector2) KeyBy operation for connected data stream.
ConnectedStreams<IN1,IN2>	keyBy(String[] fields1, String[] fields2) KeyBy operation for connected data stream using key expressions.
ConnectedStreams<IN1,IN2>	keyBy(String field1, String field2) KeyBy operation for connected data stream using key expressions.
<OUT> SingleOutputStreamOperator<OUT,?>	map(CoMapFunction<IN1,IN2,OUT> coMapper) Applies a CoMap transformation on a ConnectedStreams and maps the output to a common type.
ConnectedStreams<IN1,IN2>	partitionByHash(int[] keyPositions1, int[] keyPositions2) PartitionBy operation for connected data stream.
ConnectedStreams<IN1,IN2>	partitionByHash(int keyPosition1, int keyPosition2) PartitionBy operation for connected data stream.
ConnectedStreams<IN1,IN2>	partitionByHash(org.apache.flink.api.java.functions.KeySelector<IN1,?> keySelector1, org.apache.flink.api.java.functions.KeySelector<IN2,?> keySelector2) PartitionBy operation for connected data stream.
ConnectedStreams<IN1,IN2>	partitionByHash(String[] fields1, String[] fields2) PartitionBy operation for connected data stream using key expressions.
ConnectedStreams<IN1,IN2>	partitionByHash(String field1, String field2) PartitionBy operation for connected data stream using key expressions.
<OUT> SingleOutputStreamOperator<OUT,?>	transform(String functionName, org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outTypeInfo, TwoInputStreamOperator<IN1,IN2,OUT> operator)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

environment

protected StreamExecutionEnvironment environment

inputStream1

protected DataStream<IN1> inputStream1

inputStream2

protected DataStream<IN2> inputStream2

Constructor Detail

ConnectedStreams

protected ConnectedStreams(StreamExecutionEnvironment env,
                           DataStream<IN1> input1,
                           DataStream<IN2> input2)

Method Detail

getExecutionEnvironment

public StreamExecutionEnvironment getExecutionEnvironment()

getFirstInput

public DataStream<IN1> getFirstInput()

Returns the first DataStream.

Returns:

The first DataStream.

getSecondInput

public DataStream<IN2> getSecondInput()

Returns the second DataStream.

Returns:

The second DataStream.

getType1

public org.apache.flink.api.common.typeinfo.TypeInformation<IN1> getType1()

Gets the type of the first input

Returns:

The type of the first input

getType2

public org.apache.flink.api.common.typeinfo.TypeInformation<IN2> getType2()

Gets the type of the second input

Returns:

The type of the second input

keyBy

public ConnectedStreams<IN1,IN2> keyBy(int keyPosition1,
                                       int keyPosition2)

KeyBy operation for connected data stream. Assigns keys to the elements of input1 and input2 according to keyPosition1 and keyPosition2.

Parameters:

keyPosition1 - The field used to compute the hashcode of the elements in the first input stream.

keyPosition2 - The field used to compute the hashcode of the elements in the second input stream.

Returns:

The grouped ConnectedStreams

keyBy

public ConnectedStreams<IN1,IN2> keyBy(int[] keyPositions1,
                                       int[] keyPositions2)

KeyBy operation for connected data stream. Assigns keys to the elements of input1 and input2 according to keyPositions1 and keyPositions2.

Parameters:

keyPositions1 - The fields used to group the first input stream.

keyPositions2 - The fields used to group the second input stream.

Returns:

The grouped ConnectedStreams

keyBy

public ConnectedStreams<IN1,IN2> keyBy(String field1,
                                       String field2)

KeyBy operation for connected data stream using key expressions. Assigns keys to the elements of input1 and input2 according to field1 and field2. A field expression is either the name of a public field or a getter method with parentheses of the DataStreamS underlying type. A dot can be used to drill down into objects, as in "field1.getInnerField2()" .

Parameters:

field1 - The grouping expression for the first input

field2 - The grouping expression for the second input

Returns:

The grouped ConnectedStreams

keyBy

public ConnectedStreams<IN1,IN2> keyBy(String[] fields1,
                                       String[] fields2)

KeyBy operation for connected data stream using key expressions. the elements of input1 and input2 according to fields1 and fields2. A field expression is either the name of a public field or a getter method with parentheses of the DataStreamS underlying type. A dot can be used to drill down into objects, as in "field1.getInnerField2()" .

Parameters:

fields1 - The grouping expressions for the first input

fields2 - The grouping expressions for the second input

Returns:

The grouped ConnectedStreams

keyBy

public ConnectedStreams<IN1,IN2> keyBy(org.apache.flink.api.java.functions.KeySelector<IN1,?> keySelector1,
                                       org.apache.flink.api.java.functions.KeySelector<IN2,?> keySelector2)

KeyBy operation for connected data stream. Assigns keys to the elements of input1 and input2 using keySelector1 and keySelector2.

Parameters:

keySelector1 - The KeySelector used for grouping the first input

keySelector2 - The KeySelector used for grouping the second input

Returns:

The partitioned ConnectedStreams

partitionByHash

public ConnectedStreams<IN1,IN2> partitionByHash(int keyPosition1,
                                                 int keyPosition2)

PartitionBy operation for connected data stream. Partitions the elements of input1 and input2 according to keyPosition1 and keyPosition2.

Parameters:

keyPosition1 - The field used to compute the hashcode of the elements in the first input stream.

keyPosition2 - The field used to compute the hashcode of the elements in the second input stream.

Returns:

The partitioned ConnectedStreams

partitionByHash

public ConnectedStreams<IN1,IN2> partitionByHash(int[] keyPositions1,
                                                 int[] keyPositions2)

PartitionBy operation for connected data stream. Partitions the elements of input1 and input2 according to keyPositions1 and keyPositions2.

Parameters:

keyPositions1 - The fields used to group the first input stream.

keyPositions2 - The fields used to group the second input stream.

Returns:

The partitioned ConnectedStreams

partitionByHash

public ConnectedStreams<IN1,IN2> partitionByHash(String field1,
                                                 String field2)

PartitionBy operation for connected data stream using key expressions. Partitions the elements of input1 and input2 according to field1 and field2. A field expression is either the name of a public field or a getter method with parentheses of the DataStreams underlying type. A dot can be used to drill down into objects, as in "field1.getInnerField2()"

Parameters:

field1 - The partitioning expressions for the first input

field2 - The partitioning expressions for the second input

Returns:

The partitioned ConnectedStreams

partitionByHash

public ConnectedStreams<IN1,IN2> partitionByHash(String[] fields1,
                                                 String[] fields2)

PartitionBy operation for connected data stream using key expressions. Partitions the elements of input1 and input2 according to fields1 and fields2. A field expression is either the name of a public field or a getter method with parentheses of the DataStreams underlying type. A dot can be used to drill down into objects, as in "field1.getInnerField2()"

Parameters:

fields1 - The partitioning expressions for the first input

fields2 - The partitioning expressions for the second input

Returns:

The partitioned ConnectedStreams

partitionByHash

public ConnectedStreams<IN1,IN2> partitionByHash(org.apache.flink.api.java.functions.KeySelector<IN1,?> keySelector1,
                                                 org.apache.flink.api.java.functions.KeySelector<IN2,?> keySelector2)

PartitionBy operation for connected data stream. Partitions the elements of input1 and input2 using keySelector1 and keySelector2.

Parameters:

keySelector1 - The KeySelector used for partitioning the first input

keySelector2 - The KeySelector used for partitioning the second input

Returns:

@return The partitioned ConnectedStreams

map

public <OUT> SingleOutputStreamOperator<OUT,?> map(CoMapFunction<IN1,IN2,OUT> coMapper)

Applies a CoMap transformation on a ConnectedStreams and maps the output to a common type. The transformation calls a CoMapFunction.map1(IN1) for each element of the first input and CoMapFunction.map2(IN2) for each element of the second input. Each CoMapFunction call returns exactly one element.

Parameters:

coMapper - The CoMapFunction used to jointly transform the two input DataStreams

Returns:

The transformed DataStream

flatMap

public <OUT> SingleOutputStreamOperator<OUT,?> flatMap(CoFlatMapFunction<IN1,IN2,OUT> coFlatMapper)

Applies a CoFlatMap transformation on a ConnectedStreams and maps the output to a common type. The transformation calls a CoFlatMapFunction.flatMap1(IN1, org.apache.flink.util.Collector<OUT>) for each element of the first input and CoFlatMapFunction.flatMap2(IN2, org.apache.flink.util.Collector<OUT>) for each element of the second input. Each CoFlatMapFunction call returns any number of elements including none.

Parameters:

coFlatMapper - The CoFlatMapFunction used to jointly transform the two input DataStreams

Returns:

The transformed DataStream

transform

public <OUT> SingleOutputStreamOperator<OUT,?> transform(String functionName,
                                                         org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outTypeInfo,
                                                         TwoInputStreamOperator<IN1,IN2,OUT> operator)