AsyncDataStream

@PublicEvolving

A helper class to apply AsyncFunction to a data stream.

Example:

 val input: DataStream[String] = ...
 val asyncFunction: (String, ResultFuture[String]) => Unit = ...

 AsyncDataStream.orderedWait(input, asyncFunction, timeout, TimeUnit.MILLISECONDS, 100)
class Object
trait Matchable
class Any

Value members

Concrete methods

def orderedWait[IN, OUT : TypeInformation](input: DataStream[IN], asyncFunction: AsyncFunction[IN, OUT], timeout: Long, timeUnit: TimeUnit, capacity: Int): DataStream[OUT]

Apply an asynchronous function on the input data stream. The output order is the same as the input order of the elements.

Apply an asynchronous function on the input data stream. The output order is the same as the input order of the elements.

Type parameters:
IN

Type of the input record

OUT

Type of the output record

Value parameters:
asyncFunction

to use

capacity

of the operator which is equivalent to the number of concurrent asynchronous operations

input

to apply the async function on

timeUnit

of the timeout

timeout

for the asynchronous operation to complete

Returns:

the resulting stream containing the asynchronous results

def orderedWait[IN, OUT : TypeInformation](input: DataStream[IN], asyncFunction: AsyncFunction[IN, OUT], timeout: Long, timeUnit: TimeUnit): DataStream[OUT]

Apply an asynchronous function on the input data stream. The output order is the same as the input order of the elements.

Apply an asynchronous function on the input data stream. The output order is the same as the input order of the elements.

Type parameters:
IN

Type of the input record

OUT

Type of the output record

Value parameters:
asyncFunction

to use

input

to apply the async function on

timeUnit

of the timeout

timeout

for the asynchronous operation to complete

Returns:

the resulting stream containing the asynchronous results

def orderedWait[IN, OUT : TypeInformation](input: DataStream[IN], timeout: Long, timeUnit: TimeUnit, capacity: Int)(asyncFunction: (IN, ResultFuture[OUT]) => Unit): DataStream[OUT]

Apply an asynchronous function on the input data stream. The output order is the same as the input order of the elements.

Apply an asynchronous function on the input data stream. The output order is the same as the input order of the elements.

Type parameters:
IN

Type of the input record

OUT

Type of the output record

Value parameters:
asyncFunction

to use

capacity

of the operator which is equivalent to the number of concurrent asynchronous operations

input

to apply the async function on

timeUnit

of the timeout

timeout

for the asynchronous operation to complete

Returns:

the resulting stream containing the asynchronous results

def orderedWait[IN, OUT : TypeInformation](input: DataStream[IN], timeout: Long, timeUnit: TimeUnit)(asyncFunction: (IN, ResultFuture[OUT]) => Unit): DataStream[OUT]

Apply an asynchronous function on the input data stream. The output order is the same as the input order of the elements.

Apply an asynchronous function on the input data stream. The output order is the same as the input order of the elements.

Type parameters:
IN

Type of the input record

OUT

Type of the output record

Value parameters:
asyncFunction

to use

input

to apply the async function on

timeUnit

of the timeout

timeout

for the asynchronous operation to complete

Returns:

the resulting stream containing the asynchronous results

def unorderedWait[IN, OUT : TypeInformation](input: DataStream[IN], asyncFunction: AsyncFunction[IN, OUT], timeout: Long, timeUnit: TimeUnit, capacity: Int): DataStream[OUT]

Apply an asynchronous function on the input data stream. The output order is only maintained with respect to watermarks. Stream records which lie between the same two watermarks, can be re-ordered.

Apply an asynchronous function on the input data stream. The output order is only maintained with respect to watermarks. Stream records which lie between the same two watermarks, can be re-ordered.

Type parameters:
IN

Type of the input record

OUT

Type of the output record

Value parameters:
asyncFunction

to use

capacity

of the operator which is equivalent to the number of concurrent asynchronous operations

input

to apply the async function on

timeUnit

of the timeout

timeout

for the asynchronous operation to complete

Returns:

the resulting stream containing the asynchronous results

def unorderedWait[IN, OUT : TypeInformation](input: DataStream[IN], asyncFunction: AsyncFunction[IN, OUT], timeout: Long, timeUnit: TimeUnit): DataStream[OUT]

Apply an asynchronous function on the input data stream. The output order is only maintained with respect to watermarks. Stream records which lie between the same two watermarks, can be re-ordered.

Apply an asynchronous function on the input data stream. The output order is only maintained with respect to watermarks. Stream records which lie between the same two watermarks, can be re-ordered.

Type parameters:
IN

Type of the input record

OUT

Type of the output record

Value parameters:
asyncFunction

to use

input

to apply the async function on

timeUnit

of the timeout

timeout

for the asynchronous operation to complete

Returns:

the resulting stream containing the asynchronous results

def unorderedWait[IN, OUT : TypeInformation](input: DataStream[IN], timeout: Long, timeUnit: TimeUnit, capacity: Int)(asyncFunction: (IN, ResultFuture[OUT]) => Unit): DataStream[OUT]

Apply an asynchronous function on the input data stream. The output order is only maintained with respect to watermarks. Stream records which lie between the same two watermarks, can be re-ordered.

Apply an asynchronous function on the input data stream. The output order is only maintained with respect to watermarks. Stream records which lie between the same two watermarks, can be re-ordered.

Type parameters:
IN

Type of the input record

OUT

Type of the output record

Value parameters:
asyncFunction

to use

capacity

of the operator which is equivalent to the number of concurrent asynchronous operations

input

to apply the async function on

timeUnit

of the timeout

timeout

for the asynchronous operation to complete

Returns:

the resulting stream containing the asynchronous results

def unorderedWait[IN, OUT : TypeInformation](input: DataStream[IN], timeout: Long, timeUnit: TimeUnit)(asyncFunction: (IN, ResultFuture[OUT]) => Unit): DataStream[OUT]

Apply an asynchronous function on the input data stream. The output order is only maintained with respect to watermarks. Stream records which lie between the same two watermarks, can be re-ordered.

Apply an asynchronous function on the input data stream. The output order is only maintained with respect to watermarks. Stream records which lie between the same two watermarks, can be re-ordered.

Type parameters:
IN

Type of the input record

OUT

Type of the output record

Value parameters:
asyncFunction

to use

input

to apply the async function on

timeUnit

of the timeout

timeout

for the asynchronous operation to complete

Returns:

the resulting stream containing the asynchronous results