org.apache.flink.streaming.runtime.tasks

类 StreamTask<OUT,OP extends StreamOperator<OUT>>

java.lang.Object

org.apache.flink.streaming.runtime.tasks.StreamTask<OUT,OP>

类型参数:

OUT -

OP -

所有已实现的接口:

org.apache.flink.runtime.jobgraph.tasks.CheckpointableTask, org.apache.flink.runtime.jobgraph.tasks.CoordinatedTask, org.apache.flink.runtime.jobgraph.tasks.TaskInvokable, AsyncExceptionHandler, ContainingTaskDetails

直接已知子类:

AbstractTwoInputStreamTask, MultipleInputStreamTask, OneInputStreamTask, SourceOperatorStreamTask, SourceStreamTask

@Internal
public abstract class StreamTask<OUT,OP extends StreamOperator<OUT>>
extends Object
implements org.apache.flink.runtime.jobgraph.tasks.TaskInvokable, org.apache.flink.runtime.jobgraph.tasks.CheckpointableTask, org.apache.flink.runtime.jobgraph.tasks.CoordinatedTask, AsyncExceptionHandler, ContainingTaskDetails

Base class for all streaming tasks. A task is the unit of local processing that is deployed and executed by the TaskManagers. Each task runs one or more StreamOperators which form the Task's operator chain. Operators that are chained together execute synchronously in the same thread and hence on the same stream partition. A common case for these chains are successive map/flatmap/filter tasks.

The task chain contains one "head" operator and multiple chained operators. The StreamTask is specialized for the type of the head operator: one-input and two-input tasks, as well as for sources, iteration heads and iteration tails.

The Task class deals with the setup of the streams read by the head operator, and the streams produced by the operators at the ends of the operator chain. Note that the chain may fork and thus have multiple ends.

The life cycle of the task is set up as follows:

 -- setInitialState -> provides state of all operators in the chain

 -- invoke()
       |
       +----> Create basic utils (config, etc) and load the chain of operators
       +----> operators.setup()
       +----> task specific init()
       +----> initialize-operator-states()
       +----> open-operators()
       +----> run()
       +----> finish-operators()
       +----> close-operators()
       +----> common cleanup
       +----> task specific cleanup()
 

The StreamTask has a lock object called lock. All calls to methods on a StreamOperator must be synchronized on this lock object to ensure that no methods are called concurrently.

字段概要

字段

限定符和类型	字段和说明
protected org.apache.flink.runtime.state.CheckpointStorage	checkpointStorage Our checkpoint storage.
protected StreamConfig	configuration The configuration of this streaming task.
protected StreamInputProcessor	inputProcessor The input processor.
protected static org.slf4j.Logger	LOG The logger used by the StreamTask and its subclasses.
protected MailboxProcessor	mailboxProcessor
protected OP	mainOperator the main operator that consumes the input streams of this task.
protected OperatorChain<OUT,OP>	operatorChain The chain of operators executed by this task.
protected org.apache.flink.runtime.state.StateBackend	stateBackend Our state backend.
protected TimerService	systemTimerService In contrast to timerService we should not register any user timers here.
protected TimerService	timerService The internal TimerService used to define the current processing time (default = System.currentTimeMillis()) and register timers for tasks to be executed in the future.
static ThreadGroup	TRIGGER_THREAD_GROUP The thread group that holds all trigger timer threads.

构造器概要

构造器

限定符	构造器和说明
protected	StreamTask(org.apache.flink.runtime.execution.Environment env) Constructor for initialization, possibly with initial state (recovery / savepoint / etc).
protected	StreamTask(org.apache.flink.runtime.execution.Environment env, TimerService timerService) Constructor for initialization, possibly with initial state (recovery / savepoint / etc).
protected	StreamTask(org.apache.flink.runtime.execution.Environment environment, TimerService timerService, Thread.UncaughtExceptionHandler uncaughtExceptionHandler)
protected	StreamTask(org.apache.flink.runtime.execution.Environment environment, TimerService timerService, Thread.UncaughtExceptionHandler uncaughtExceptionHandler, StreamTaskActionExecutor actionExecutor) Constructor for initialization, possibly with initial state (recovery / savepoint / etc).
protected	StreamTask(org.apache.flink.runtime.execution.Environment environment, TimerService timerService, Thread.UncaughtExceptionHandler uncaughtExceptionHandler, StreamTaskActionExecutor actionExecutor, TaskMailbox mailbox)

方法概要

限定符和类型	方法和说明
void	abortCheckpointOnBarrier(long checkpointId, org.apache.flink.runtime.checkpoint.CheckpointException cause)
protected void	advanceToEndOfEventTime() Emits the MAX_WATERMARK so that all registered timers are fired.
protected void	afterInvoke()
protected CompletableFuture<Boolean>	assertTriggeringCheckpointExceptions(CompletableFuture<Boolean> triggerFuture, long checkpointId)
Future<Void>	cancel()
protected void	cancelTask()
void	cleanUp(Throwable throwable)
protected void	cleanUpInternal()
static <OUT> org.apache.flink.runtime.io.network.api.writer.RecordWriterDelegate<org.apache.flink.runtime.plugable.SerializationDelegate<StreamRecord<OUT>>>	createRecordWriterDelegate(StreamConfig configuration, org.apache.flink.runtime.execution.Environment environment)
StreamTaskStateInitializer	createStreamTaskStateInitializer()
protected void	declineCheckpoint(long checkpointId)
void	dispatchOperatorEvent(org.apache.flink.runtime.jobgraph.OperatorID operator, org.apache.flink.util.SerializedValue<org.apache.flink.runtime.operators.coordination.OperatorEvent> event)
protected void	endData()
protected void	finalize() The finalize method shuts down the timer.
protected void	finishTask() Instructs the task to go through its normal termination routine, i.e. exit the run-loop and call StreamOperator.finish() and StreamOperator.close() on its operators.
protected long	getAsyncCheckpointStartDelayNanos()
ExecutorService	getAsyncOperationsThreadPool()
org.apache.flink.core.fs.CloseableRegistry	getCancelables()
protected Optional<CheckpointBarrierHandler>	getCheckpointBarrierHandler() Acquires the optional CheckpointBarrierHandler associated with this stream task.
org.apache.flink.runtime.state.CheckpointStorageWorkerView	getCheckpointStorage()
protected CompletableFuture<Void>	getCompletionFuture()
StreamConfig	getConfiguration()
org.apache.flink.runtime.execution.Environment	getEnvironment()
MailboxExecutorFactory	getMailboxExecutorFactory()
String	getName() Gets the name of the task, in the form "taskname (2/5)".
ProcessingTimeServiceFactory	getProcessingTimeServiceFactory()
void	handleAsyncException(String message, Throwable exception) Handles an exception thrown by another thread (e.g. a TriggerTask), other than the one executing the main task by failing the task entirely.
protected abstract void	init()
void	invoke()
boolean	isCanceled()
boolean	isFailing()
boolean	isMailboxLoopRunning()
boolean	isRunning()
boolean	isUsingNonBlockingInput()
void	maybeInterruptOnCancel(Thread toInterrupt, String taskName, Long timeout)
Future<Void>	notifyCheckpointAbortAsync(long checkpointId, long latestCompletedCheckpointId)
Future<Void>	notifyCheckpointCompleteAsync(long checkpointId)
protected void	processInput(MailboxDefaultAction.Controller controller) This method implements the default action of the task (e.g. processing one event from the input).
void	restore()
void	runMailboxLoop()
boolean	runMailboxStep()
protected void	setSynchronousSavepoint(long checkpointId, boolean isDrain)
protected org.apache.flink.metrics.Counter	setupNumRecordsInCounter(StreamOperator streamOperator)
String	toString()
CompletableFuture<Boolean>	triggerCheckpointAsync(org.apache.flink.runtime.checkpoint.CheckpointMetaData checkpointMetaData, org.apache.flink.runtime.checkpoint.CheckpointOptions checkpointOptions)
void	triggerCheckpointOnBarrier(org.apache.flink.runtime.checkpoint.CheckpointMetaData checkpointMetaData, org.apache.flink.runtime.checkpoint.CheckpointOptions checkpointOptions, org.apache.flink.runtime.checkpoint.CheckpointMetricsBuilder checkpointMetrics)

从类继承的方法 java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

从接口继承的方法 org.apache.flink.streaming.runtime.tasks.ContainingTaskDetails

getExecutionConfig, getIndexInSubtaskGroup, getJobConfiguration, getUserCodeClassLoader

字段详细资料

TRIGGER_THREAD_GROUP

public static final ThreadGroup TRIGGER_THREAD_GROUP

The thread group that holds all trigger timer threads.

LOG

protected static final org.slf4j.Logger LOG

The logger used by the StreamTask and its subclasses.

inputProcessor

@Nullable
protected StreamInputProcessor inputProcessor

The input processor. Initialized in init() method.

mainOperator

protected OP extends StreamOperator<OUT> mainOperator

the main operator that consumes the input streams of this task.

operatorChain

protected OperatorChain<OUT,OP extends StreamOperator<OUT>> operatorChain

The chain of operators executed by this task.

configuration

protected final StreamConfig configuration

The configuration of this streaming task.

stateBackend

protected final org.apache.flink.runtime.state.StateBackend stateBackend

Our state backend. We use this to create a keyed state backend.

checkpointStorage

protected final org.apache.flink.runtime.state.CheckpointStorage checkpointStorage

Our checkpoint storage. We use this to create checkpoint streams.

timerService

protected final TimerService timerService

The internal TimerService used to define the current processing time (default = System.currentTimeMillis()) and register timers for tasks to be executed in the future.

systemTimerService

protected final TimerService systemTimerService

In contrast to timerService we should not register any user timers here. It should be used only for system level timers.

mailboxProcessor

protected final MailboxProcessor mailboxProcessor

构造器详细资料

StreamTask

protected StreamTask(org.apache.flink.runtime.execution.Environment env)
              throws Exception

Constructor for initialization, possibly with initial state (recovery / savepoint / etc).

参数:

env - The task environment for this task.

抛出:

Exception

StreamTask

protected StreamTask(org.apache.flink.runtime.execution.Environment env,
                     @Nullable
                     TimerService timerService)
              throws Exception

Constructor for initialization, possibly with initial state (recovery / savepoint / etc).

参数:

env - The task environment for this task.

timerService - Optionally, a specific timer service to use.

抛出:

Exception

StreamTask

protected StreamTask(org.apache.flink.runtime.execution.Environment environment,
                     @Nullable
                     TimerService timerService,
                     Thread.UncaughtExceptionHandler uncaughtExceptionHandler)
              throws Exception

抛出:

Exception

StreamTask

protected StreamTask(org.apache.flink.runtime.execution.Environment environment,
                     @Nullable
                     TimerService timerService,
                     Thread.UncaughtExceptionHandler uncaughtExceptionHandler,
                     StreamTaskActionExecutor actionExecutor)
              throws Exception

Constructor for initialization, possibly with initial state (recovery / savepoint / etc).

This constructor accepts a special TimerService. By default (and if null is passes for the timer service) a DefaultTimerService will be used.

参数:

environment - The task environment for this task.

timerService - Optionally, a specific timer service to use.

uncaughtExceptionHandler - to handle uncaught exceptions in the async operations thread pool

actionExecutor - a mean to wrap all actions performed by this task thread. Currently, only SynchronizedActionExecutor can be used to preserve locking semantics.

抛出:

Exception

StreamTask

protected StreamTask(org.apache.flink.runtime.execution.Environment environment,
                     @Nullable
                     TimerService timerService,
                     Thread.UncaughtExceptionHandler uncaughtExceptionHandler,
                     StreamTaskActionExecutor actionExecutor,
                     TaskMailbox mailbox)
              throws Exception

抛出:

Exception

方法详细资料

init

protected abstract void init()
                      throws Exception

抛出:

Exception

cancelTask

protected void cancelTask()
                   throws Exception

抛出:

Exception

processInput

protected void processInput(MailboxDefaultAction.Controller controller)
                     throws Exception

This method implements the default action of the task (e.g. processing one event from the input). Implementations should (in general) be non-blocking.

参数:

controller - controller object for collaborative interaction between the action and the stream task.

抛出:

Exception - on any problems in the action.

endData

protected void endData()
                throws Exception

抛出:

Exception

setSynchronousSavepoint

protected void setSynchronousSavepoint(long checkpointId,
                                       boolean isDrain)

advanceToEndOfEventTime

protected void advanceToEndOfEventTime()
                                throws Exception

Emits the MAX_WATERMARK so that all registered timers are fired.

This is used by the source task when the job is TERMINATED. In the case, we want all the timers registered throughout the pipeline to fire and the related state (e.g. windows) to be flushed.

For tasks other than the source task, this method does nothing.

抛出:

Exception

finishTask

protected void finishTask()
                   throws Exception

Instructs the task to go through its normal termination routine, i.e. exit the run-loop and call StreamOperator.finish() and StreamOperator.close() on its operators.

This is used by the source task to get out of the run-loop when the job is stopped with a savepoint.

For tasks other than the source task, this method does nothing.

抛出:

Exception

createStreamTaskStateInitializer

public StreamTaskStateInitializer createStreamTaskStateInitializer()

setupNumRecordsInCounter

protected org.apache.flink.metrics.Counter setupNumRecordsInCounter(StreamOperator streamOperator)

restore

public final void restore()
                   throws Exception

指定者:

restore 在接口中 org.apache.flink.runtime.jobgraph.tasks.TaskInvokable

抛出:

Exception

invoke

public final void invoke()
                  throws Exception

指定者:

invoke 在接口中 org.apache.flink.runtime.jobgraph.tasks.TaskInvokable

抛出:

Exception

runMailboxStep

@VisibleForTesting
public boolean runMailboxStep()
                                          throws Exception

抛出:

Exception

isMailboxLoopRunning

@VisibleForTesting
public boolean isMailboxLoopRunning()

runMailboxLoop

public void runMailboxLoop()
                    throws Exception

抛出:

Exception

afterInvoke

protected void afterInvoke()
                    throws Exception

抛出:

Exception

cleanUp

public final void cleanUp(Throwable throwable)
                   throws Exception

指定者:

cleanUp 在接口中 org.apache.flink.runtime.jobgraph.tasks.TaskInvokable

抛出:

Exception

cleanUpInternal

protected void cleanUpInternal()
                        throws Exception

抛出:

Exception

getCompletionFuture

protected CompletableFuture<Void> getCompletionFuture()

cancel

public final Future<Void> cancel()
                          throws Exception

指定者:

cancel 在接口中 org.apache.flink.runtime.jobgraph.tasks.TaskInvokable

抛出:

Exception

getMailboxExecutorFactory

public MailboxExecutorFactory getMailboxExecutorFactory()

isRunning

public final boolean isRunning()

isCanceled

public final boolean isCanceled()

isFailing

public final boolean isFailing()

finalize

protected void finalize()
                 throws Throwable

The finalize method shuts down the timer. This is a fail-safe shutdown, in case the original shutdown method was never called.

This should not be relied upon! It will cause shutdown to happen much later than if manual shutdown is attempted, and cause threads to linger for longer than needed.

覆盖:

finalize 在类中 Object

抛出:

Throwable

getName

public final String getName()

Gets the name of the task, in the form "taskname (2/5)".

返回:

The name of the task.

getCheckpointStorage

public org.apache.flink.runtime.state.CheckpointStorageWorkerView getCheckpointStorage()

getConfiguration

public StreamConfig getConfiguration()

triggerCheckpointAsync

public CompletableFuture<Boolean> triggerCheckpointAsync(org.apache.flink.runtime.checkpoint.CheckpointMetaData checkpointMetaData,
                                                         org.apache.flink.runtime.checkpoint.CheckpointOptions checkpointOptions)

指定者:

triggerCheckpointAsync 在接口中 org.apache.flink.runtime.jobgraph.tasks.CheckpointableTask

assertTriggeringCheckpointExceptions

protected final CompletableFuture<Boolean> assertTriggeringCheckpointExceptions(CompletableFuture<Boolean> triggerFuture,
                                                                                long checkpointId)

getCheckpointBarrierHandler

protected Optional<CheckpointBarrierHandler> getCheckpointBarrierHandler()

Acquires the optional CheckpointBarrierHandler associated with this stream task. The CheckpointBarrierHandler should exist if the task has data inputs and requires to align the barriers.

triggerCheckpointOnBarrier

public void triggerCheckpointOnBarrier(org.apache.flink.runtime.checkpoint.CheckpointMetaData checkpointMetaData,
                                       org.apache.flink.runtime.checkpoint.CheckpointOptions checkpointOptions,
                                       org.apache.flink.runtime.checkpoint.CheckpointMetricsBuilder checkpointMetrics)
                                throws IOException

指定者:

triggerCheckpointOnBarrier 在接口中 org.apache.flink.runtime.jobgraph.tasks.CheckpointableTask

抛出:

IOException

abortCheckpointOnBarrier

public void abortCheckpointOnBarrier(long checkpointId,
                                     org.apache.flink.runtime.checkpoint.CheckpointException cause)
                              throws IOException

指定者:

abortCheckpointOnBarrier 在接口中 org.apache.flink.runtime.jobgraph.tasks.CheckpointableTask

抛出:

IOException

declineCheckpoint

protected void declineCheckpoint(long checkpointId)

getAsyncOperationsThreadPool

public final ExecutorService getAsyncOperationsThreadPool()

notifyCheckpointCompleteAsync

public Future<Void> notifyCheckpointCompleteAsync(long checkpointId)

指定者:

notifyCheckpointCompleteAsync 在接口中 org.apache.flink.runtime.jobgraph.tasks.CheckpointableTask

notifyCheckpointAbortAsync

public Future<Void> notifyCheckpointAbortAsync(long checkpointId,
                                               long latestCompletedCheckpointId)

指定者:

notifyCheckpointAbortAsync 在接口中 org.apache.flink.runtime.jobgraph.tasks.CheckpointableTask

dispatchOperatorEvent

public void dispatchOperatorEvent(org.apache.flink.runtime.jobgraph.OperatorID operator,
                                  org.apache.flink.util.SerializedValue<org.apache.flink.runtime.operators.coordination.OperatorEvent> event)
                           throws org.apache.flink.util.FlinkException

指定者:

dispatchOperatorEvent 在接口中 org.apache.flink.runtime.jobgraph.tasks.CoordinatedTask

抛出:

org.apache.flink.util.FlinkException

getProcessingTimeServiceFactory

public ProcessingTimeServiceFactory getProcessingTimeServiceFactory()

handleAsyncException

public void handleAsyncException(String message,
                                 Throwable exception)

Handles an exception thrown by another thread (e.g. a TriggerTask), other than the one executing the main task by failing the task entirely.

In more detail, it marks task execution failed for an external reason (a reason other than the task code itself throwing an exception). If the task is already in a terminal state (such as FINISHED, CANCELED, FAILED), or if the task is already canceling this does nothing. Otherwise it sets the state to FAILED, and, if the invokable code is running, starts an asynchronous thread that aborts that code.

This method never blocks.

指定者:

handleAsyncException 在接口中 AsyncExceptionHandler

toString

public String toString()

覆盖:

toString 在类中 Object

getCancelables

public final org.apache.flink.core.fs.CloseableRegistry getCancelables()

createRecordWriterDelegate

@VisibleForTesting
public static <OUT> org.apache.flink.runtime.io.network.api.writer.RecordWriterDelegate<org.apache.flink.runtime.plugable.SerializationDelegate<StreamRecord<OUT>>> createRecordWriterDelegate(StreamConfig configuration,
                                                                                                                                                                                                                  org.apache.flink.runtime.execution.Environment environment)

getAsyncCheckpointStartDelayNanos

protected long getAsyncCheckpointStartDelayNanos()

isUsingNonBlockingInput

public boolean isUsingNonBlockingInput()

指定者:

isUsingNonBlockingInput 在接口中 org.apache.flink.runtime.jobgraph.tasks.TaskInvokable

maybeInterruptOnCancel

public void maybeInterruptOnCancel(Thread toInterrupt,
                                   @Nullable
                                   String taskName,
                                   @Nullable
                                   Long timeout)

指定者:

maybeInterruptOnCancel 在接口中 org.apache.flink.runtime.jobgraph.tasks.TaskInvokable

getEnvironment

public final org.apache.flink.runtime.execution.Environment getEnvironment()