org.apache.flink.api.java.operators

Class DeltaIteration<ST,WT>

java.lang.Object

org.apache.flink.api.java.operators.DeltaIteration<ST,WT>

Type Parameters:

ST - The data type of the solution set.

WT - The data type of the workset (the feedback data set).

public class DeltaIteration<ST,WT>
extends Object

The DeltaIteration represents the start of a delta iteration. It is created from the DataSet that represents the initial solution set via the DataSet.iterateDelta(DataSet, int, int...) method.

See Also:

DataSet.iterateDelta(DataSet, int, int...), DataSet.iterateDelta(DataSet, int, int[])

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DeltaIteration.SolutionSetPlaceHolder<ST> A DataSet that acts as a placeholder for the solution set during the iteration.
static class	DeltaIteration.WorksetPlaceHolder<WT> A DataSet that acts as a placeholder for the workset during the iteration.

Constructor Summary

Constructors

Constructor and Description
DeltaIteration(ExecutionEnvironment context, org.apache.flink.api.common.typeinfo.TypeInformation<ST> type, DataSet<ST> solutionSet, DataSet<WT> workset, Keys<ST> keys, int maxIterations)

Method Summary

Modifier and Type	Method and Description
DataSet<ST>	closeWith(DataSet<ST> solutionSetDelta, DataSet<WT> newWorkset) Closes the delta iteration.
org.apache.flink.api.common.aggregators.AggregatorRegistry	getAggregators() Gets the registry for aggregators for the iteration.
DataSet<ST>	getInitialSolutionSet() Gets the initial solution set.
DataSet<WT>	getInitialWorkset() Gets the initial workset.
String	getName() Gets the name from this iteration.
int	getParallelism() Gets the iteration's parallelism.
DeltaIteration.SolutionSetPlaceHolder<ST>	getSolutionSet() Gets the solution set of the delta iteration.
DeltaIteration.WorksetPlaceHolder<WT>	getWorkset() Gets the working set of the delta iteration.
boolean	isSolutionSetUnManaged() gets whether the solution set is in managed or unmanaged memory.
DeltaIteration<ST,WT>	name(String name) Sets the name for the iteration.
DeltaIteration<ST,WT>	parallelism(int parallelism) Sets the parallelism for the iteration.
DeltaIteration<ST,WT>	registerAggregator(String name, org.apache.flink.api.common.aggregators.Aggregator<?> aggregator) Registers an Aggregator for the iteration.
void	setSolutionSetUnManaged(boolean solutionSetUnManaged) Sets whether to keep the solution set in managed memory (safe against heap exhaustion) or unmanaged memory (objects on heap).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DeltaIteration

public DeltaIteration(ExecutionEnvironment context,
                      org.apache.flink.api.common.typeinfo.TypeInformation<ST> type,
                      DataSet<ST> solutionSet,
                      DataSet<WT> workset,
                      Keys<ST> keys,
                      int maxIterations)

Method Detail

closeWith

public DataSet<ST> closeWith(DataSet<ST> solutionSetDelta,
                             DataSet<WT> newWorkset)

Closes the delta iteration. This method defines the end of the delta iteration's function.

Parameters:

solutionSetDelta - The delta for the solution set. The delta will be merged into the solution set at the end of each iteration.

newWorkset - The new workset (feedback data set) that will be fed back to the next iteration.

Returns:

The DataSet that represents the result of the iteration, after the computation has terminated.

See Also:

DataSet.iterateDelta(DataSet, int, int...)

getInitialSolutionSet

public DataSet<ST> getInitialSolutionSet()

Gets the initial solution set. This is the data set on which the delta iteration was started.

Consider the following example:

 
 DataSet<MyType> solutionSetData = ...;
 DataSet<AnotherType> worksetData = ...;
 
 DeltaIteration<MyType, AnotherType> iteration = solutionSetData.iteratorDelta(worksetData, 10, ...);
 
 

The solutionSetData would be the data set returned by iteration.getInitialSolutionSet();.

Returns:

The data set that forms the initial solution set.

getInitialWorkset

public DataSet<WT> getInitialWorkset()

Gets the initial workset. This is the data set passed to the method that starts the delta iteration.

Consider the following example:

 
 DataSet<MyType> solutionSetData = ...;
 DataSet<AnotherType> worksetData = ...;
 
 DeltaIteration<MyType, AnotherType> iteration = solutionSetData.iteratorDelta(worksetData, 10, ...);
 
 

The worksetData would be the data set returned by iteration.getInitialWorkset();.

Returns:

The data set that forms the initial workset.

getSolutionSet

public DeltaIteration.SolutionSetPlaceHolder<ST> getSolutionSet()

Gets the solution set of the delta iteration. The solution set represents the state that is kept across iterations.

Returns:

The solution set of the delta iteration.

getWorkset

public DeltaIteration.WorksetPlaceHolder<WT> getWorkset()

Gets the working set of the delta iteration. The working set is constructed by the previous iteration.

Returns:

The working set of the delta iteration.

name

public DeltaIteration<ST,WT> name(String name)

Sets the name for the iteration. The name is displayed in logs and messages.

Parameters:

name - The name for the iteration.

Returns:

The iteration object, for function call chaining.

getName

public String getName()

Gets the name from this iteration.

Returns:

The name of the iteration.

parallelism

public DeltaIteration<ST,WT> parallelism(int parallelism)

Sets the parallelism for the iteration.

Parameters:

parallelism - The parallelism.

Returns:

The iteration object, for function call chaining.

getParallelism

public int getParallelism()

Gets the iteration's parallelism.

Returns:

The iterations parallelism, or -1, if not set.

registerAggregator

public DeltaIteration<ST,WT> registerAggregator(String name,
                                                org.apache.flink.api.common.aggregators.Aggregator<?> aggregator)

Registers an Aggregator for the iteration. Aggregators can be used to maintain simple statistics during the iteration, such as number of elements processed. The aggregators compute global aggregates: After each iteration step, the values are globally aggregated to produce one aggregate that represents statistics across all parallel instances. The value of an aggregator can be accessed in the next iteration.

Aggregators can be accessed inside a function via the AbstractRichFunction.getIterationRuntimeContext() method.

Parameters:

name - The name under which the aggregator is registered.

aggregator - The aggregator class.

Returns:

The DeltaIteration itself, to allow chaining function calls.

getAggregators

public org.apache.flink.api.common.aggregators.AggregatorRegistry getAggregators()

Gets the registry for aggregators for the iteration.

Returns:

The registry with all aggregators.

setSolutionSetUnManaged

public void setSolutionSetUnManaged(boolean solutionSetUnManaged)

Sets whether to keep the solution set in managed memory (safe against heap exhaustion) or unmanaged memory (objects on heap).

Parameters:

solutionSetUnManaged - True to keep the solution set in unmanaged memory, false to keep it in managed memory.

See Also:

isSolutionSetUnManaged()

isSolutionSetUnManaged

public boolean isSolutionSetUnManaged()

gets whether the solution set is in managed or unmanaged memory.

Returns:

True, if the solution set is in unmanaged memory (object heap), false if in managed memory.

See Also:

setSolutionSetUnManaged(boolean)