Package org.apache.flink.streaming.api.datastream

Class ConnectedStreams<IN1,​IN2>

  • Type Parameters:
    IN1 - Type of the first input data steam.
    IN2 - Type of the second input data stream.
    @Public
public class ConnectedStreams<IN1,​IN2>
extends Object
    ConnectedStreams represent two connected streams of (possibly) different data types. Connected streams are useful for cases where operations on one stream directly affect the operations on the other stream, usually via shared state between the streams.

    An example for the use of connected streams would be to apply rules that change over time onto another stream. One of the connected streams has the rules, the other stream the elements to apply the rules to. The operation on the connected stream maintains the current set of rules in the state. It may receive either a rule update and update the state or a data element and apply the rules in the state to the element.

    The connected stream can be conceptually viewed as a union stream of an Either type, that holds either the first stream's type or the second stream's type.

    • Method Detail

      • getFirstInput

        public DataStream<IN1> getFirstInput()
        Returns the first DataStream.
        Returns:
        The first DataStream.

      • getSecondInput

        public DataStream<IN2> getSecondInput()
        Returns the second DataStream.
        Returns:
        The second DataStream.

      • getType1

        public org.apache.flink.api.common.typeinfo.TypeInformation<IN1> getType1()
        Gets the type of the first input.
        Returns:
        The type of the first input

      • getType2

        public org.apache.flink.api.common.typeinfo.TypeInformation<IN2> getType2()
        Gets the type of the second input.
        Returns:
        The type of the second input

      • keyBy

        @Deprecated
public ConnectedStreams<IN1,​IN2> keyBy​(int keyPosition1,
                                             int keyPosition2)
        Deprecated.
        use keyBy(KeySelector, KeySelector)
        KeyBy operation for connected data stream. Assigns keys to the elements of input1 and input2 according to keyPosition1 and keyPosition2.
        Parameters:
        keyPosition1 - The field used to compute the hashcode of the elements in the first input stream.
        keyPosition2 - The field used to compute the hashcode of the elements in the second input stream.
        Returns:
        The grouped ConnectedStreams

      • keyBy

        @Deprecated
public ConnectedStreams<IN1,​IN2> keyBy​(int[] keyPositions1,
                                             int[] keyPositions2)
        Deprecated.
        use keyBy(KeySelector, KeySelector)
        KeyBy operation for connected data stream. Assigns keys to the elements of input1 and input2 according to keyPositions1 and keyPositions2.
        Parameters:
        keyPositions1 - The fields used to group the first input stream.
        keyPositions2 - The fields used to group the second input stream.
        Returns:
        The grouped ConnectedStreams

      • keyBy

        @Deprecated
public ConnectedStreams<IN1,​IN2> keyBy​(String field1,
                                             String field2)
        Deprecated.
        use keyBy(KeySelector, KeySelector)
        KeyBy operation for connected data stream using key expressions. Assigns keys to the elements of input1 and input2 according to field1 and field2. A field expression is either the name of a public field or a getter method with parentheses of the DataStreamS underlying type. A dot can be used to drill down into objects, as in "field1.getInnerField2()" .
        Parameters:
        field1 - The grouping expression for the first input
        field2 - The grouping expression for the second input
        Returns:
        The grouped ConnectedStreams

      • keyBy

        @Deprecated
public ConnectedStreams<IN1,​IN2> keyBy​(String[] fields1,
                                             String[] fields2)
        Deprecated.
        use keyBy(KeySelector, KeySelector)
        KeyBy operation for connected data stream using key expressions. the elements of input1 and input2 according to fields1 and fields2. A field expression is either the name of a public field or a getter method with parentheses of the DataStreamS underlying type. A dot can be used to drill down into objects, as in "field1.getInnerField2()" .
        Parameters:
        fields1 - The grouping expressions for the first input
        fields2 - The grouping expressions for the second input
        Returns:
        The grouped ConnectedStreams

      • keyBy

        public <KEY> ConnectedStreams<IN1,​IN2> keyBy​(org.apache.flink.api.java.functions.KeySelector<IN1,​KEY> keySelector1,
                                                   org.apache.flink.api.java.functions.KeySelector<IN2,​KEY> keySelector2)
        KeyBy operation for connected data stream. Assigns keys to the elements of input1 and input2 using keySelector1 and keySelector2.
        Parameters:
        keySelector1 - The KeySelector used for grouping the first input
        keySelector2 - The KeySelector used for grouping the second input
        Returns:
        The partitioned ConnectedStreams

      • keyBy

        public <KEY> ConnectedStreams<IN1,​IN2> keyBy​(org.apache.flink.api.java.functions.KeySelector<IN1,​KEY> keySelector1,
                                                   org.apache.flink.api.java.functions.KeySelector<IN2,​KEY> keySelector2,
                                                   org.apache.flink.api.common.typeinfo.TypeInformation<KEY> keyType)
        KeyBy operation for connected data stream. Assigns keys to the elements of input1 and input2 using keySelector1 and keySelector2 with explicit type information for the common key type.
        Parameters:
        keySelector1 - The KeySelector used for grouping the first input
        keySelector2 - The KeySelector used for grouping the second input
        keyType - The type information of the common key type.
        Returns:
        The partitioned ConnectedStreams

      • map

        public <R> SingleOutputStreamOperator<R> map​(CoMapFunction<IN1,​IN2,​R> coMapper,
                                             org.apache.flink.api.common.typeinfo.TypeInformation<R> outputType)
        Applies a CoMap transformation on a ConnectedStreams and maps the output to a common type. The transformation calls a CoMapFunction.map1(IN1) for each element of the first input and CoMapFunction.map2(IN2) for each element of the second input. Each CoMapFunction call returns exactly one element.
        Parameters:
        coMapper - The CoMapFunction used to jointly transform the two input DataStreams
        outputType - TypeInformation for the result type of the function.
        Returns:
        The transformed DataStream

      • process

        @PublicEvolving
public <R> SingleOutputStreamOperator<R> process​(CoProcessFunction<IN1,​IN2,​R> coProcessFunction)
        Applies the given CoProcessFunction on the connected input streams, thereby creating a transformed output stream.

        The function will be called for every element in the input streams and can produce zero or more output elements. Contrary to the flatMap(CoFlatMapFunction) function, this function can also query the time and set timers. When reacting to the firing of set timers the function can directly emit elements and/or register yet more timers.

        Type Parameters:
        R - The type of elements emitted by the CoProcessFunction.
        Parameters:
        coProcessFunction - The CoProcessFunction that is called for each element in the stream.
        Returns:
        The transformed DataStream.

      • process

        @Internal
public <R> SingleOutputStreamOperator<R> process​(CoProcessFunction<IN1,​IN2,​R> coProcessFunction,
                                                 org.apache.flink.api.common.typeinfo.TypeInformation<R> outputType)
        Applies the given CoProcessFunction on the connected input streams, thereby creating a transformed output stream.

        The function will be called for every element in the input streams and can produce zero or more output elements. Contrary to the flatMap(CoFlatMapFunction) function, this function can also query the time and set timers. When reacting to the firing of set timers the function can directly emit elements and/or register yet more timers.

        Type Parameters:
        R - The type of elements emitted by the CoProcessFunction.
        Parameters:
        coProcessFunction - The CoProcessFunction that is called for each element in the stream.
        Returns:
        The transformed DataStream.

      • process

        @PublicEvolving
public <K,​R> SingleOutputStreamOperator<R> process​(KeyedCoProcessFunction<K,​IN1,​IN2,​R> keyedCoProcessFunction)
        Applies the given KeyedCoProcessFunction on the connected input keyed streams, thereby creating a transformed output stream.

        The function will be called for every element in the input keyed streams and can produce zero or more output elements. Contrary to the flatMap(CoFlatMapFunction) function, this function can also query the time and set timers. When reacting to the firing of set timers the function can directly emit elements and/or register yet more timers.

        Type Parameters:
        R - The type of elements emitted by the CoProcessFunction.
        Parameters:
        keyedCoProcessFunction - The KeyedCoProcessFunction that is called for each element in the stream.
        Returns:
        The transformed DataStream.

      • process

        @Internal
public <K,​R> SingleOutputStreamOperator<R> process​(KeyedCoProcessFunction<K,​IN1,​IN2,​R> keyedCoProcessFunction,
                                                         org.apache.flink.api.common.typeinfo.TypeInformation<R> outputType)
        Applies the given KeyedCoProcessFunction on the connected input streams, thereby creating a transformed output stream.

        The function will be called for every element in the input streams and can produce zero or more output elements. Contrary to the flatMap(CoFlatMapFunction) function, this function can also query the time and set timers. When reacting to the firing of set timers the function can directly emit elements and/or register yet more timers.

        Type Parameters:
        R - The type of elements emitted by the CoProcessFunction.
        Parameters:
        keyedCoProcessFunction - The KeyedCoProcessFunction that is called for each element in the stream.
        Returns:
        The transformed DataStream.