org.apache.flink.iteration.operator

类 HeadOperator

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<IterationRecord<?>>

org.apache.flink.iteration.operator.HeadOperator

所有已实现的接口:

Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.runtime.operators.coordination.OperatorEventHandler, org.apache.flink.statefun.flink.core.feedback.FeedbackConsumer<org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IterationRecord<?>>>, org.apache.flink.streaming.api.operators.BoundedOneInput, org.apache.flink.streaming.api.operators.Input<IterationRecord<?>>, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.OneInputStreamOperator<IterationRecord<?>,IterationRecord<?>>, org.apache.flink.streaming.api.operators.SetupableStreamOperator<IterationRecord<?>>, org.apache.flink.streaming.api.operators.StreamOperator<IterationRecord<?>>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator

public class HeadOperator
extends org.apache.flink.streaming.api.operators.AbstractStreamOperator<IterationRecord<?>>
implements org.apache.flink.streaming.api.operators.OneInputStreamOperator<IterationRecord<?>,IterationRecord<?>>, org.apache.flink.statefun.flink.core.feedback.FeedbackConsumer<org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IterationRecord<?>>>, org.apache.flink.runtime.operators.coordination.OperatorEventHandler, org.apache.flink.streaming.api.operators.BoundedOneInput

The head operator unions the initialized variable stream and the feedback stream, synchronize the epoch watermark (round) and taking care of the checkpoints.

Specially for checkpoint, the head operator would like to

• Ensures the exactly-once for processing elements.
• Ensures the exactly-once for IterationListener.onEpochWatermarkIncremented(int, IterationListener.Context, Collector).

To implement the first target, the head operator also need to include the records between alignment and received barrier from the feed-back edge into the snapshot. To implement the second target, the head operator would also wait for the notification from the OperatorCoordinator in additional to the task inputs. This ensures the GloballyAlignedEvent would not interleave with the epoch watermarks and all the tasks inside the iteration would be notified with the same epochs, which facility the rescaling in the future.

另请参阅:

序列化表格

字段概要

字段

限定符和类型	字段和说明
static org.apache.flink.util.OutputTag<IterationRecord<Void>>	ALIGN_NOTIFY_OUTPUT_TAG

从类继承的字段 org.apache.flink.streaming.api.operators.AbstractStreamOperator

chainingStrategy, config, latencyStats, LOG, metrics, output, processingTimeService

构造器概要

构造器

构造器和说明
HeadOperator(IterationID iterationId, int feedbackIndex, boolean isCriteriaStream, org.apache.flink.api.common.operators.MailboxExecutor mailboxExecutor, org.apache.flink.runtime.operators.coordination.OperatorEventGateway operatorEventGateway, org.apache.flink.streaming.runtime.tasks.ProcessingTimeService processingTimeService)

方法概要

限定符和类型	方法和说明
void	close()
void	endInput()
org.apache.flink.runtime.operators.coordination.OperatorEventGateway	getOperatorEventGateway()
org.apache.flink.iteration.operator.HeadOperator.HeadOperatorStatus	getStatus()
void	handleOperatorEvent(org.apache.flink.runtime.operators.coordination.OperatorEvent operatorEvent)
void	initializeState(org.apache.flink.runtime.state.StateInitializationContext context)
void	notifyCheckpointAborted(long checkpointId)
void	prepareSnapshotPreBarrier(long checkpointId)
void	processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IterationRecord<?>> element)
void	processFeedback(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IterationRecord<?>> iterationRecord)
void	setup(org.apache.flink.streaming.runtime.tasks.StreamTask<?,?> containingTask, org.apache.flink.streaming.api.graph.StreamConfig config, org.apache.flink.streaming.api.operators.Output<org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IterationRecord<?>>> output)
void	snapshotState(org.apache.flink.runtime.state.StateSnapshotContext context)

从类继承的方法 org.apache.flink.streaming.api.operators.AbstractStreamOperator

finish, getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, initializeState, isUsingCustomRawKeyedState, notifyCheckpointComplete, open, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark, processWatermark1, processWatermark2, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, snapshotState

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.flink.streaming.api.operators.OneInputStreamOperator

setKeyContextElement

从接口继承的方法 org.apache.flink.streaming.api.operators.StreamOperator

finish, getMetricGroup, getOperatorID, initializeState, open, setKeyContextElement1, setKeyContextElement2, snapshotState

从接口继承的方法 org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointComplete

从接口继承的方法 org.apache.flink.streaming.api.operators.KeyContext

getCurrentKey, setCurrentKey

从接口继承的方法 org.apache.flink.streaming.api.operators.Input

processLatencyMarker, processWatermark, processWatermarkStatus

字段详细资料

ALIGN_NOTIFY_OUTPUT_TAG

public static final org.apache.flink.util.OutputTag<IterationRecord<Void>> ALIGN_NOTIFY_OUTPUT_TAG

构造器详细资料

HeadOperator

public HeadOperator(IterationID iterationId,
                    int feedbackIndex,
                    boolean isCriteriaStream,
                    org.apache.flink.api.common.operators.MailboxExecutor mailboxExecutor,
                    org.apache.flink.runtime.operators.coordination.OperatorEventGateway operatorEventGateway,
                    org.apache.flink.streaming.runtime.tasks.ProcessingTimeService processingTimeService)

方法详细资料

setup

public void setup(org.apache.flink.streaming.runtime.tasks.StreamTask<?,?> containingTask,
                  org.apache.flink.streaming.api.graph.StreamConfig config,
                  org.apache.flink.streaming.api.operators.Output<org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IterationRecord<?>>> output)

指定者:

setup 在接口中 org.apache.flink.streaming.api.operators.SetupableStreamOperator<IterationRecord<?>>

覆盖:

setup 在类中 org.apache.flink.streaming.api.operators.AbstractStreamOperator<IterationRecord<?>>

initializeState

public void initializeState(org.apache.flink.runtime.state.StateInitializationContext context)
                     throws Exception

指定者:

initializeState 在接口中 org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator

覆盖:

initializeState 在类中 org.apache.flink.streaming.api.operators.AbstractStreamOperator<IterationRecord<?>>

抛出:

Exception

prepareSnapshotPreBarrier

public void prepareSnapshotPreBarrier(long checkpointId)
                               throws Exception

指定者:

prepareSnapshotPreBarrier 在接口中 org.apache.flink.streaming.api.operators.StreamOperator<IterationRecord<?>>

覆盖:

prepareSnapshotPreBarrier 在类中 org.apache.flink.streaming.api.operators.AbstractStreamOperator<IterationRecord<?>>

抛出:

Exception

snapshotState

public void snapshotState(org.apache.flink.runtime.state.StateSnapshotContext context)
                   throws Exception

指定者:

snapshotState 在接口中 org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator

覆盖:

snapshotState 在类中 org.apache.flink.streaming.api.operators.AbstractStreamOperator<IterationRecord<?>>

抛出:

Exception

notifyCheckpointAborted

public void notifyCheckpointAborted(long checkpointId)
                             throws Exception

指定者:

notifyCheckpointAborted 在接口中 org.apache.flink.api.common.state.CheckpointListener

覆盖:

notifyCheckpointAborted 在类中 org.apache.flink.streaming.api.operators.AbstractStreamOperator<IterationRecord<?>>

抛出:

Exception

processElement

public void processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IterationRecord<?>> element)
                    throws Exception

指定者:

processElement 在接口中 org.apache.flink.streaming.api.operators.Input<IterationRecord<?>>

抛出:

Exception

processFeedback

public void processFeedback(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IterationRecord<?>> iterationRecord)
                     throws Exception

指定者:

processFeedback 在接口中 org.apache.flink.statefun.flink.core.feedback.FeedbackConsumer<org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IterationRecord<?>>>

抛出:

Exception

handleOperatorEvent

public void handleOperatorEvent(org.apache.flink.runtime.operators.coordination.OperatorEvent operatorEvent)

指定者:

handleOperatorEvent 在接口中 org.apache.flink.runtime.operators.coordination.OperatorEventHandler

endInput

public void endInput()
              throws Exception

指定者:

endInput 在接口中 org.apache.flink.streaming.api.operators.BoundedOneInput

抛出:

Exception

close

public void close()
           throws Exception

指定者:

close 在接口中 org.apache.flink.streaming.api.operators.StreamOperator<IterationRecord<?>>

覆盖:

close 在类中 org.apache.flink.streaming.api.operators.AbstractStreamOperator<IterationRecord<?>>

抛出:

Exception

getOperatorEventGateway

@VisibleForTesting
public org.apache.flink.runtime.operators.coordination.OperatorEventGateway getOperatorEventGateway()

getStatus

@VisibleForTesting
public org.apache.flink.iteration.operator.HeadOperator.HeadOperatorStatus getStatus()