org.apache.flink.api.java.functions

Class RichGroupReduceFunction<IN,OUT>

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.flink.api.java.functions.RichGroupReduceFunction<IN,OUT>

Type Parameters:

IN - Type of the elements that this function processes.

OUT - The type of the elements returned by the user-defined function.

All Implemented Interfaces:

Serializable, FlatCombineFunction<IN>, Function, GroupReduceFunction<IN,OUT>, RichFunction

Direct Known Subclasses:

AggregateOperator.AggregatingUdf, DistinctOperator.DistinctFunction, GroupReduceIterator

public abstract class RichGroupReduceFunction<IN,OUT>
extends AbstractRichFunction
implements GroupReduceFunction<IN,OUT>, FlatCombineFunction<IN>

Rich variant of the GroupReduceFunction. As a RichFunction, it gives access to the RuntimeContext and provides setup and teardown methods: RichFunction.open(org.apache.flink.configuration.Configuration) and RichFunction.close().

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	RichGroupReduceFunction.Combinable This annotation can be added to classes that extend RichGroupReduceFunction, in oder to mark them as "combinable".

Constructor Summary

Constructors

Constructor and Description
RichGroupReduceFunction()

Method Summary

Methods

Modifier and Type	Method and Description
void	combine(Iterable<IN> values, Collector<IN> out) The combine methods pre-reduces elements.
abstract void	reduce(Iterable<IN> values, Collector<OUT> out)

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

close, getIterationRuntimeContext, getRuntimeContext, open, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RichGroupReduceFunction

public RichGroupReduceFunction()

Method Detail

reduce

public abstract void reduce(Iterable<IN> values,
          Collector<OUT> out)
                     throws Exception

Specified by:

reduce in interface GroupReduceFunction<IN,OUT>

Throws:

Exception

combine

public void combine(Iterable<IN> values,
           Collector<IN> out)
             throws Exception

The combine methods pre-reduces elements. It may be called on subsets of the data before the actual reduce function. This is often helpful to lower data volume prior to reorganizing the data in an expensive way, as might be required for the final reduce function.

This method is only ever invoked when the subclass of RichGroupReduceFunction adds the RichGroupReduceFunction.Combinable annotation, or if the combinable flag is set when defining the reduceGroup operation via GroupReduceOperator.setCombinable(boolean).

Since the reduce function will be called on the result of this method, it is important that this method returns the same data type as it consumes. By default, this method only calls the reduce(Iterable, Collector) method. If the behavior in the pre-reducing is different from the final reduce function (for example because the reduce function changes the data type), this method must be overwritten, or the execution will fail.

Specified by:

combine in interface FlatCombineFunction<IN>

Parameters:

values - The iterator returning the group of values to be reduced.

out - The collector to emit the returned values.

Throws:

Exception - This method may throw exceptions. Throwing an exception will cause the operation to fail and may trigger recovery.