AllWindowedStream

@Public
class AllWindowedStream[T, W <: Window](javaStream: AllWindowedStream[T, W])

A AllWindowedStream represents a data stream where the stream of elements is split into windows based on a org.apache.flink.streaming.api.windowing.assigners.WindowAssigner. Window emission is triggered based on a Trigger.

If an Evictor is specified it will be used to evict elements from the window after evaluation was triggered by the Trigger but before the actual evaluation of the window. When using an evictor window performance will degrade significantly, since pre-aggregation of window results cannot be used.

Note that the AllWindowedStream is purely and API construct, during runtime the AllWindowedStream will be collapsed together with the operation over the window into one single operation.

Type parameters:
T

The type of elements in the stream.

W

The type of Window that the org.apache.flink.streaming.api.windowing.assigners.WindowAssigner assigns the elements to.

class Object
trait Matchable
class Any

Value members

Concrete methods

@PublicEvolving
def aggregate[ACC : TypeInformation, R : TypeInformation](aggregateFunction: AggregateFunction[T, ACC, R]): DataStream[R]

Applies the given aggregation function to each window. The aggregation function is called for each element, aggregating values incrementally and keeping the state to one accumulator per window.

Applies the given aggregation function to each window. The aggregation function is called for each element, aggregating values incrementally and keeping the state to one accumulator per window.

Value parameters:
aggregateFunction

The aggregation function.

Returns:

The data stream that is the result of applying the aggregate function to the window.

@PublicEvolving
def aggregate[ACC : TypeInformation, V : TypeInformation, R : TypeInformation](preAggregator: AggregateFunction[T, ACC, V], windowFunction: AllWindowFunction[V, R, W]): DataStream[R]

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Arriving data is pre-aggregated using the given aggregation function.

Value parameters:
preAggregator

The aggregation function that is used for pre-aggregation

windowFunction

The window function.

Returns:

The data stream that is the result of applying the window function to the window.

@PublicEvolving
def aggregate[ACC : TypeInformation, V : TypeInformation, R : TypeInformation](preAggregator: AggregateFunction[T, ACC, V], windowFunction: ProcessAllWindowFunction[V, R, W]): DataStream[R]

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Arriving data is pre-aggregated using the given aggregation function.

Value parameters:
preAggregator

The aggregation function that is used for pre-aggregation

windowFunction

The process window function.

Returns:

The data stream that is the result of applying the window function to the window.

@PublicEvolving
def aggregate[ACC : TypeInformation, V : TypeInformation, R : TypeInformation](preAggregator: AggregateFunction[T, ACC, V], windowFunction: (W, Iterable[V], Collector[R]) => Unit): DataStream[R]

Applies the given window function to each window. The window function is called for each evaluation of the window. The output of the window function is interpreted as a regular non-windowed stream.

Applies the given window function to each window. The window function is called for each evaluation of the window. The output of the window function is interpreted as a regular non-windowed stream.

Arriving data is pre-aggregated using the given aggregation function.

Value parameters:
preAggregator

The aggregation function that is used for pre-aggregation

windowFunction

The window function.

Returns:

The data stream that is the result of applying the window function to the window.

def aggregate(aggregationType: AggregationType, position: Int): DataStream[T]
@PublicEvolving
def allowedLateness(lateness: Time): AllWindowedStream[T, W]

Sets the allowed lateness to a user-specified value. If not explicitly set, the allowed lateness is 0L. Setting the allowed lateness is only valid for event-time windows. If a value different than 0 is provided with a processing-time org.apache.flink.streaming.api.windowing.assigners.WindowAssigner, then an exception is thrown.

Sets the allowed lateness to a user-specified value. If not explicitly set, the allowed lateness is 0L. Setting the allowed lateness is only valid for event-time windows. If a value different than 0 is provided with a processing-time org.apache.flink.streaming.api.windowing.assigners.WindowAssigner, then an exception is thrown.

def apply[R : TypeInformation](function: AllWindowFunction[T, R, W]): DataStream[R]

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Note that this function requires that all data in the windows is buffered until the window is evaluated, as the function provides no means of pre-aggregation.

Value parameters:
function

The window function.

Returns:

The data stream that is the result of applying the window function to the window.

def apply[R : TypeInformation](function: (W, Iterable[T], Collector[R]) => Unit): DataStream[R]

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Note that this function requires that all data in the windows is buffered until the window is evaluated, as the function provides no means of pre-aggregation.

Value parameters:
function

The window function.

Returns:

The data stream that is the result of applying the window function to the window.

@PublicEvolving
def evictor(evictor: Evictor[_ >: T, _ >: W]): AllWindowedStream[T, W]

Sets the Evictor that should be used to evict elements from a window before emission.

Sets the Evictor that should be used to evict elements from a window before emission.

Note: When using an evictor window performance will degrade significantly, since pre-aggregation of window results cannot be used.

def max(position: Int): DataStream[T]

Applies an aggregation that that gives the maximum of the elements in the window at the given position.

Applies an aggregation that that gives the maximum of the elements in the window at the given position.

def max(field: String): DataStream[T]

Applies an aggregation that that gives the maximum of the elements in the window at the given field.

Applies an aggregation that that gives the maximum of the elements in the window at the given field.

def maxBy(position: Int): DataStream[T]

Applies an aggregation that that gives the maximum element of the window by the given position. When equality, returns the first.

Applies an aggregation that that gives the maximum element of the window by the given position. When equality, returns the first.

def maxBy(field: String): DataStream[T]

Applies an aggregation that that gives the maximum element of the window by the given field. When equality, returns the first.

Applies an aggregation that that gives the maximum element of the window by the given field. When equality, returns the first.

def min(position: Int): DataStream[T]

Applies an aggregation that that gives the minimum of the elements in the window at the given position.

Applies an aggregation that that gives the minimum of the elements in the window at the given position.

def min(field: String): DataStream[T]

Applies an aggregation that that gives the minimum of the elements in the window at the given field.

Applies an aggregation that that gives the minimum of the elements in the window at the given field.

def minBy(position: Int): DataStream[T]

Applies an aggregation that that gives the minimum element of the window by the given position. When equality, returns the first.

Applies an aggregation that that gives the minimum element of the window by the given position. When equality, returns the first.

def minBy(field: String): DataStream[T]

Applies an aggregation that that gives the minimum element of the window by the given field. When equality, returns the first.

Applies an aggregation that that gives the minimum element of the window by the given field. When equality, returns the first.

@PublicEvolving
def process[R : TypeInformation](function: ProcessAllWindowFunction[T, R, W]): DataStream[R]

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Note that this function requires that all data in the windows is buffered until the window is evaluated, as the function provides no means of pre-aggregation.

Value parameters:
function

The process window function.

Returns:

The data stream that is the result of applying the window function to the window.

def reduce(function: ReduceFunction[T]): DataStream[T]

Applies a reduce function to the window. The window function is called for each evaluation of the window for each key individually. The output of the reduce function is interpreted as a regular non-windowed stream.

Applies a reduce function to the window. The window function is called for each evaluation of the window for each key individually. The output of the reduce function is interpreted as a regular non-windowed stream.

This window will try and pre-aggregate data as much as the window policies permit. For example, tumbling time windows can perfectly pre-aggregate the data, meaning that only one element per key is stored. Sliding time windows will pre-aggregate on the granularity of the slide interval, so a few elements are stored per key (one per slide interval). Custom windows may not be able to pre-aggregate, or may need to store extra values in an aggregation tree.

Value parameters:
function

The reduce function.

Returns:

The data stream that is the result of applying the reduce function to the window.

def reduce(function: (T, T) => T): DataStream[T]

Applies a reduce function to the window. The window function is called for each evaluation of the window for each key individually. The output of the reduce function is interpreted as a regular non-windowed stream.

Applies a reduce function to the window. The window function is called for each evaluation of the window for each key individually. The output of the reduce function is interpreted as a regular non-windowed stream.

This window will try and pre-aggregate data as much as the window policies permit. For example, tumbling time windows can perfectly pre-aggregate the data, meaning that only one element per key is stored. Sliding time windows will pre-aggregate on the granularity of the slide interval, so a few elements are stored per key (one per slide interval). Custom windows may not be able to pre-aggregate, or may need to store extra values in an aggregation tree.

Value parameters:
function

The reduce function.

Returns:

The data stream that is the result of applying the reduce function to the window.

def reduce[R : TypeInformation](preAggregator: ReduceFunction[T], windowFunction: AllWindowFunction[T, R, W]): DataStream[R]

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Arriving data is pre-aggregated using the given pre-aggregation reducer.

Value parameters:
preAggregator

The reduce function that is used for pre-aggregation

windowFunction

The window function.

Returns:

The data stream that is the result of applying the window function to the window.

def reduce[R : TypeInformation](preAggregator: (T, T) => T, windowFunction: (W, Iterable[T], Collector[R]) => Unit): DataStream[R]

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Arriving data is pre-aggregated using the given pre-aggregation reducer.

Value parameters:
preAggregator

The reduce function that is used for pre-aggregation

windowFunction

The window function.

Returns:

The data stream that is the result of applying the window function to the window.

@PublicEvolving
def reduce[R : TypeInformation](preAggregator: ReduceFunction[T], windowFunction: ProcessAllWindowFunction[T, R, W]): DataStream[R]

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Arriving data is pre-aggregated using the given pre-aggregation reducer.

Value parameters:
preAggregator

The reduce function that is used for pre-aggregation

windowFunction

The process window function.

Returns:

The data stream that is the result of applying the window function to the window.

@PublicEvolving
def reduce[R : TypeInformation](preAggregator: (T, T) => T, windowFunction: ProcessAllWindowFunction[T, R, W]): DataStream[R]

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Arriving data is pre-aggregated using the given pre-aggregation reducer.

Value parameters:
preAggregator

The reduce function that is used for pre-aggregation

windowFunction

The process window function.

Returns:

The data stream that is the result of applying the window function to the window.

@PublicEvolving

Send late arriving data to the side output identified by the given OutputTag. Data is considered late after the watermark has passed the end of the window plus the allowed lateness set using allowedLateness.

Send late arriving data to the side output identified by the given OutputTag. Data is considered late after the watermark has passed the end of the window plus the allowed lateness set using allowedLateness.

You can get the stream of late data using DataStream.getSideOutput on the DataStream resulting from the windowed operation with the same OutputTag.

def sum(position: Int): DataStream[T]

Applies an aggregation that sums the elements in the window at the given position.

Applies an aggregation that sums the elements in the window at the given position.

def sum(field: String): DataStream[T]

Applies an aggregation that sums the elements in the window at the given field.

Applies an aggregation that sums the elements in the window at the given field.

@PublicEvolving
def trigger(trigger: Trigger[_ >: T, _ >: W]): AllWindowedStream[T, W]

Sets the Trigger that should be used to trigger window emission.

Sets the Trigger that should be used to trigger window emission.