org.apache.flink.streaming.runtime.tasks

Class StreamTask<OUT,Operator extends StreamOperator<OUT>>

java.lang.Object

org.apache.flink.runtime.jobgraph.tasks.AbstractInvokable

org.apache.flink.streaming.runtime.tasks.StreamTask<OUT,Operator>

Type Parameters:

OUT -

Operator -

All Implemented Interfaces:

org.apache.flink.runtime.jobgraph.tasks.StatefulTask<StreamTaskStateList>

Direct Known Subclasses:

OneInputStreamTask, SourceStreamTask, TwoInputStreamTask

@Internal
public abstract class StreamTask<OUT,Operator extends StreamOperator<OUT>>
extends org.apache.flink.runtime.jobgraph.tasks.AbstractInvokable
implements org.apache.flink.runtime.jobgraph.tasks.StatefulTask<StreamTaskStateList>

Base class for all streaming tasks. A task is the unit of local processing that is deployed and executed by the TaskManagers. Each task runs one or more StreamOperators which form the Task's operator chain. Operators that are chained together execute synchronously in the same thread and hence on the same stream partition. A common case for these chaines are successive map/flatmap/filter tasks.

The task chain contains one "head" operator and multiple chained operators. The StreamTask is specialized for the type of the head operator: one-input and two-input tasks, as well as for sources, iteration heads and iteration tails.

The Task class deals with the setup of the streams read by the head operator, and the streams produced by the operators at the ends of the operator chain. Note that the chain may fork and thus have multiple ends. The life cycle of the task is set up as follows:

  -- restoreState() -> restores state of all operators in the chain
  
  -- invoke()
        |
        +----> Create basic utils (config, etc) and load the chain of operators
        +----> operators.setup()
        +----> task specific init()
        +----> open-operators()
        +----> run()
        +----> close-operators()
        +----> dispose-operators()
        +----> common cleanup
        +----> task specific cleanup()
 

The StreamTask has a lock object called lock. All calls to methods on a StreamOperator must be synchronized on this lock object to ensure that no methods are called concurrently.

Field Summary

Fields

Modifier and Type	Field and Description
protected Operator	headOperator the head operator that consumes the input streams of this task
protected static org.slf4j.Logger	LOG The logger used by the StreamTask and its subclasses
static ThreadGroup	TRIGGER_THREAD_GROUP The thread group that holds all trigger timer threads

Constructor Summary

Constructors

Constructor and Description
StreamTask()

Method Summary

Modifier and Type	Method and Description
void	cancel()
protected abstract void	cancelTask()
void	checkTimerException()
protected abstract void	cleanup()
org.apache.flink.runtime.state.AbstractStateBackend	createStateBackend(String operatorIdentifier, org.apache.flink.api.common.typeutils.TypeSerializer<?> keySerializer)
protected void	finalize() The finalize method shuts down the timer.
Map<String,org.apache.flink.api.common.accumulators.Accumulator<?,?>>	getAccumulatorMap()
protected org.apache.flink.runtime.util.event.EventListener<org.apache.flink.runtime.io.network.api.CheckpointBarrier>	getCheckpointBarrierListener()
Object	getCheckpointLock() Gets the lock object on which all operations that involve data and state mutation have to lock.
StreamConfig	getConfiguration()
Output<StreamRecord<OUT>>	getHeadOutput()
String	getName() Gets the name of the task, in the form "taskname (2/5)".
RecordWriterOutput<?>[]	getStreamOutputs()
protected abstract void	init()
void	invoke()
boolean	isCanceled()
boolean	isRunning()
protected boolean	isSerializingTimestamps()
void	notifyCheckpointComplete(long checkpointId)
protected boolean	performCheckpoint(long checkpointId, long timestamp)
void	registerTimer(long timestamp, Triggerable target) Registers a timer.
protected abstract void	run()
void	setInitialState(StreamTaskStateList initialState, long recoveryTimestamp)
String	toString()
boolean	triggerCheckpoint(long checkpointId, long timestamp)

Methods inherited from class org.apache.flink.runtime.jobgraph.tasks.AbstractInvokable

getCurrentNumberOfSubtasks, getEnvironment, getExecutionConfig, getIndexInSubtaskGroup, getJobConfiguration, getTaskConfiguration, getUserCodeClassLoader, setEnvironment

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

TRIGGER_THREAD_GROUP

public static final ThreadGroup TRIGGER_THREAD_GROUP

The thread group that holds all trigger timer threads

LOG

protected static final org.slf4j.Logger LOG

The logger used by the StreamTask and its subclasses

headOperator

protected Operator extends StreamOperator<OUT> headOperator

the head operator that consumes the input streams of this task

Constructor Detail

StreamTask

public StreamTask()

Method Detail

init

protected abstract void init()
                      throws Exception

Throws:

Exception

run

protected abstract void run()
                     throws Exception

Throws:

Exception

cleanup

protected abstract void cleanup()
                         throws Exception

Throws:

Exception

cancelTask

protected abstract void cancelTask()
                            throws Exception

Throws:

Exception

invoke

public final void invoke()
                  throws Exception

Specified by:

invoke in class org.apache.flink.runtime.jobgraph.tasks.AbstractInvokable

Throws:

Exception

cancel

public final void cancel()
                  throws Exception

Overrides:

cancel in class org.apache.flink.runtime.jobgraph.tasks.AbstractInvokable

Throws:

Exception

isRunning

public final boolean isRunning()

isCanceled

public final boolean isCanceled()

finalize

protected void finalize()
                 throws Throwable

The finalize method shuts down the timer. This is a fail-safe shutdown, in case the original shutdown method was never called.

This should not be relied upon! It will cause shutdown to happen much later than if manual shutdown is attempted, and cause threads to linger for longer than needed.

Overrides:

finalize in class Object

Throws:

Throwable

isSerializingTimestamps

protected boolean isSerializingTimestamps()

getName

public String getName()

Gets the name of the task, in the form "taskname (2/5)".

Returns:

The name of the task.

getCheckpointLock

public Object getCheckpointLock()

Gets the lock object on which all operations that involve data and state mutation have to lock.

Returns:

The checkpoint lock object.

getConfiguration

public StreamConfig getConfiguration()

getAccumulatorMap

public Map<String,org.apache.flink.api.common.accumulators.Accumulator<?,?>> getAccumulatorMap()

getHeadOutput

public Output<StreamRecord<OUT>> getHeadOutput()

getStreamOutputs

public RecordWriterOutput<?>[] getStreamOutputs()

setInitialState

public void setInitialState(StreamTaskStateList initialState,
                            long recoveryTimestamp)

Specified by:

setInitialState in interface org.apache.flink.runtime.jobgraph.tasks.StatefulTask<StreamTaskStateList>

triggerCheckpoint

public boolean triggerCheckpoint(long checkpointId,
                                 long timestamp)
                          throws Exception

Specified by:

triggerCheckpoint in interface org.apache.flink.runtime.jobgraph.tasks.StatefulTask<StreamTaskStateList>

Throws:

Exception

performCheckpoint

protected boolean performCheckpoint(long checkpointId,
                                    long timestamp)
                             throws Exception

Throws:

Exception

notifyCheckpointComplete

public void notifyCheckpointComplete(long checkpointId)
                              throws Exception

Specified by:

notifyCheckpointComplete in interface org.apache.flink.runtime.jobgraph.tasks.StatefulTask<StreamTaskStateList>

Throws:

Exception

createStateBackend

public org.apache.flink.runtime.state.AbstractStateBackend createStateBackend(String operatorIdentifier,
                                                                              org.apache.flink.api.common.typeutils.TypeSerializer<?> keySerializer)
                                                                       throws Exception

Throws:

Exception

registerTimer

public void registerTimer(long timestamp,
                          Triggerable target)

Registers a timer.

checkTimerException

public void checkTimerException()
                         throws TimerException

Throws:

TimerException

toString

public String toString()

Overrides:

toString in class Object

getCheckpointBarrierListener

protected final org.apache.flink.runtime.util.event.EventListener<org.apache.flink.runtime.io.network.api.CheckpointBarrier> getCheckpointBarrierListener()