Package org.apache.flink.streaming.api.datastream

Class AsyncDataStream

java.lang.Object

org.apache.flink.streaming.api.datastream.AsyncDataStream

@PublicEvolving
public class AsyncDataStream
extends Object

A helper class to apply AsyncFunction to a data stream.

 DataStream<String> input = ...
 AsyncFunction<String, Tuple<String, String>> asyncFunc = ...

 AsyncDataStream.orderedWait(input, asyncFunc, timeout, TimeUnit.MILLISECONDS, 100);
 

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	AsyncDataStream.OutputMode	Output mode for asynchronous operations.

Constructor Summary

Constructors

Constructor	Description
AsyncDataStream()

Method Summary

Modifier and Type	Method	Description
static <IN,OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT>	orderedWait(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit)	Adds an AsyncWaitOperator.
static <IN,OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT>	orderedWait(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit, int capacity)	Adds an AsyncWaitOperator.
static <IN,OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT>	orderedWaitWithRetry(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit, int capacity, AsyncRetryStrategy<OUT> asyncRetryStrategy)	Adds an AsyncWaitOperator with an AsyncRetryStrategy to support retry of AsyncFunction.
static <IN,OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT>	orderedWaitWithRetry(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit, AsyncRetryStrategy<OUT> asyncRetryStrategy)	Adds an AsyncWaitOperator with an AsyncRetryStrategy to support retry of AsyncFunction.
static <IN,OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT>	unorderedWait(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit)	Adds an AsyncWaitOperator.
static <IN,OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT>	unorderedWait(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit, int capacity)	Adds an AsyncWaitOperator.
static <IN,OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT>	unorderedWaitWithRetry(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit, int capacity, AsyncRetryStrategy<OUT> asyncRetryStrategy)	Adds an AsyncWaitOperator with an AsyncRetryStrategy to support retry of AsyncFunction.
static <IN,OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT>	unorderedWaitWithRetry(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit, AsyncRetryStrategy<OUT> asyncRetryStrategy)	Adds an AsyncWaitOperator with an AsyncRetryStrategy to support retry of AsyncFunction.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AsyncDataStream

public AsyncDataStream()

Method Detail

unorderedWait

public static <IN,OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT> unorderedWait(org.apache.flink.streaming.api.datastream.DataStream<IN> in,
                                                                                                                     AsyncFunction<IN,OUT> func,
                                                                                                                     long timeout,
                                                                                                                     TimeUnit timeUnit,
                                                                                                                     int capacity)

Adds an AsyncWaitOperator. The order of output stream records may be reordered.

Type Parameters:

IN - Type of input record

OUT - Type of output record

Parameters:

in - Input DataStream

func - AsyncFunction

timeout - for the asynchronous operation to complete

timeUnit - of the given timeout

capacity - The max number of async i/o operation that can be triggered

Returns:

A new SingleOutputStreamOperator.

unorderedWait

public static <IN,OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT> unorderedWait(org.apache.flink.streaming.api.datastream.DataStream<IN> in,
                                                                                                                     AsyncFunction<IN,OUT> func,
                                                                                                                     long timeout,
                                                                                                                     TimeUnit timeUnit)

Adds an AsyncWaitOperator. The order of output stream records may be reordered.

Type Parameters:

IN - Type of input record

OUT - Type of output record

Parameters:

in - Input DataStream

func - AsyncFunction

timeout - for the asynchronous operation to complete

timeUnit - of the given timeout

Returns:

A new SingleOutputStreamOperator.

orderedWait

public static <IN,OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT> orderedWait(org.apache.flink.streaming.api.datastream.DataStream<IN> in,
                                                                                                                   AsyncFunction<IN,OUT> func,
                                                                                                                   long timeout,
                                                                                                                   TimeUnit timeUnit,
                                                                                                                   int capacity)

Adds an AsyncWaitOperator. The order to process input records is guaranteed to be the same as input ones.

Type Parameters:

IN - Type of input record

OUT - Type of output record

Parameters:

in - Input DataStream

func - AsyncFunction

timeout - for the asynchronous operation to complete

timeUnit - of the given timeout

capacity - The max number of async i/o operation that can be triggered

Returns:

A new SingleOutputStreamOperator.

orderedWait

public static <IN,OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT> orderedWait(org.apache.flink.streaming.api.datastream.DataStream<IN> in,
                                                                                                                   AsyncFunction<IN,OUT> func,
                                                                                                                   long timeout,
                                                                                                                   TimeUnit timeUnit)

Adds an AsyncWaitOperator. The order to process input records is guaranteed to be the same as input ones.

Type Parameters:

IN - Type of input record

OUT - Type of output record

Parameters:

in - Input DataStream

func - AsyncFunction

timeout - for the asynchronous operation to complete

timeUnit - of the given timeout

Returns:

A new SingleOutputStreamOperator.

unorderedWaitWithRetry

public static <IN,OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT> unorderedWaitWithRetry(org.apache.flink.streaming.api.datastream.DataStream<IN> in,
                                                                                                                              AsyncFunction<IN,OUT> func,
                                                                                                                              long timeout,
                                                                                                                              TimeUnit timeUnit,
                                                                                                                              AsyncRetryStrategy<OUT> asyncRetryStrategy)

Adds an AsyncWaitOperator with an AsyncRetryStrategy to support retry of AsyncFunction. The order of output stream records may be reordered.

Type Parameters:

IN - Type of input record

OUT - Type of output record

Parameters:

in - Input DataStream

func - AsyncFunction

timeout - from first invoke to final completion of asynchronous operation, may include multiple retries, and will be reset in case of restart

timeUnit - of the given timeout

asyncRetryStrategy - The strategy of reattempt async i/o operation that can be triggered

Returns:

A new SingleOutputStreamOperator.

unorderedWaitWithRetry

public static <IN,OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT> unorderedWaitWithRetry(org.apache.flink.streaming.api.datastream.DataStream<IN> in,
                                                                                                                              AsyncFunction<IN,OUT> func,
                                                                                                                              long timeout,
                                                                                                                              TimeUnit timeUnit,
                                                                                                                              int capacity,
                                                                                                                              AsyncRetryStrategy<OUT> asyncRetryStrategy)

Adds an AsyncWaitOperator with an AsyncRetryStrategy to support retry of AsyncFunction. The order of output stream records may be reordered.

Type Parameters:

IN - Type of input record

OUT - Type of output record

Parameters:

in - Input DataStream

func - AsyncFunction

timeout - from first invoke to final completion of asynchronous operation, may include multiple retries, and will be reset in case of restart

timeUnit - of the given timeout

capacity - The max number of async i/o operation that can be triggered

asyncRetryStrategy - The strategy of reattempt async i/o operation that can be triggered

Returns:

A new SingleOutputStreamOperator.

orderedWaitWithRetry

public static <IN,OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT> orderedWaitWithRetry(org.apache.flink.streaming.api.datastream.DataStream<IN> in,
                                                                                                                            AsyncFunction<IN,OUT> func,
                                                                                                                            long timeout,
                                                                                                                            TimeUnit timeUnit,
                                                                                                                            AsyncRetryStrategy<OUT> asyncRetryStrategy)

Adds an AsyncWaitOperator with an AsyncRetryStrategy to support retry of AsyncFunction. The order to process input records is guaranteed to be the same as * input ones.

Type Parameters:

IN - Type of input record

OUT - Type of output record

Parameters:

in - Input DataStream

func - AsyncFunction

timeout - from first invoke to final completion of asynchronous operation, may include multiple retries, and will be reset in case of restart

timeUnit - of the given timeout

asyncRetryStrategy - The strategy of reattempt async i/o operation that can be triggered

Returns:

A new SingleOutputStreamOperator.

orderedWaitWithRetry

public static <IN,OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT> orderedWaitWithRetry(org.apache.flink.streaming.api.datastream.DataStream<IN> in,
                                                                                                                            AsyncFunction<IN,OUT> func,
                                                                                                                            long timeout,
                                                                                                                            TimeUnit timeUnit,
                                                                                                                            int capacity,
                                                                                                                            AsyncRetryStrategy<OUT> asyncRetryStrategy)

Adds an AsyncWaitOperator with an AsyncRetryStrategy to support retry of AsyncFunction. The order to process input records is guaranteed to be the same as * input ones.

Type Parameters:

IN - Type of input record

OUT - Type of output record

Parameters:

in - Input DataStream

func - AsyncFunction

timeout - from first invoke to final completion of asynchronous operation, may include multiple retries, and will be reset in case of restart

timeUnit - of the given timeout

capacity - The max number of async i/o operation that can be triggered

asyncRetryStrategy - The strategy of reattempt async i/o operation that can be triggered

Returns:

A new SingleOutputStreamOperator.