@Internal public class StreamTaskSourceInput<T> extends Object implements StreamTaskInput<T>, org.apache.flink.runtime.io.network.partition.consumer.CheckpointableInput
StreamTaskInput
that reads data from the SourceOperator
and
returns the DataInputStatus
to indicate whether the source state is available,
unavailable or finished.PushingAsyncDataInput.DataOutput<T>
UNSPECIFIED
构造器和说明 |
---|
StreamTaskSourceInput(SourceOperator<T,?> operator,
int inputGateIndex,
int inputIndex) |
限定符和类型 | 方法和说明 |
---|---|
void |
blockConsumption(org.apache.flink.runtime.checkpoint.channel.InputChannelInfo channelInfo) |
void |
checkpointStarted(org.apache.flink.runtime.io.network.api.CheckpointBarrier barrier)
This method is used with unaligned checkpoints to mark the arrival of a first
CheckpointBarrier . |
void |
checkpointStopped(long cancelledCheckpointId) |
void |
close() |
void |
convertToPriorityEvent(int channelIndex,
int sequenceNumber) |
DataInputStatus |
emitNext(PushingAsyncDataInput.DataOutput<T> output)
Pushes the next element to the output from current data input, and returns the input status
to indicate whether there are more available data in current input.
|
CompletableFuture<?> |
getAvailableFuture() |
List<org.apache.flink.runtime.checkpoint.channel.InputChannelInfo> |
getChannelInfos() |
int |
getInputGateIndex() |
int |
getInputIndex()
Returns the input index of this input.
|
int |
getNumberOfInputChannels() |
SourceOperator<T,?> |
getOperator() |
org.apache.flink.runtime.jobgraph.OperatorID |
getOperatorID() |
CompletableFuture<Void> |
prepareSnapshot(org.apache.flink.runtime.checkpoint.channel.ChannelStateWriter channelStateWriter,
long checkpointId)
Prepares to spill the in-flight input buffers as checkpoint snapshot.
|
void |
resumeConsumption(org.apache.flink.runtime.checkpoint.channel.InputChannelInfo channelInfo) |
public StreamTaskSourceInput(SourceOperator<T,?> operator, int inputGateIndex, int inputIndex)
public DataInputStatus emitNext(PushingAsyncDataInput.DataOutput<T> output) throws Exception
PushingAsyncDataInput
This method should be non blocking.
emitNext
在接口中 PushingAsyncDataInput<T>
Exception
public CompletableFuture<?> getAvailableFuture()
getAvailableFuture
在接口中 org.apache.flink.runtime.io.AvailabilityProvider
public void blockConsumption(org.apache.flink.runtime.checkpoint.channel.InputChannelInfo channelInfo)
blockConsumption
在接口中 org.apache.flink.runtime.io.network.partition.consumer.CheckpointableInput
public void resumeConsumption(org.apache.flink.runtime.checkpoint.channel.InputChannelInfo channelInfo)
resumeConsumption
在接口中 org.apache.flink.runtime.io.network.partition.consumer.CheckpointableInput
public List<org.apache.flink.runtime.checkpoint.channel.InputChannelInfo> getChannelInfos()
getChannelInfos
在接口中 org.apache.flink.runtime.io.network.partition.consumer.CheckpointableInput
public int getNumberOfInputChannels()
getNumberOfInputChannels
在接口中 org.apache.flink.runtime.io.network.partition.consumer.CheckpointableInput
public void checkpointStarted(org.apache.flink.runtime.io.network.api.CheckpointBarrier barrier)
CheckpointBarrier
. For chained sources, there is no CheckpointBarrier
per se flowing
through the job graph. We can assume that an imaginary CheckpointBarrier
was produced
by the source, at any point of time of our choosing.
We are choosing to interpret it, that CheckpointBarrier
for sources was received
immediately as soon as we receive either checkpoint start RPC, or CheckpointBarrier
from a network input. So that we can checkpoint state of the source and all of the other
operators at the same time.
Also we are choosing to block the source, as a best effort optimisation as: - either there is no backpressure and the checkpoint "alignment" will happen very quickly anyway - or there is a backpressure, and it's better to prioritize processing data from the network to speed up checkpointing. From the cluster resource utilisation perspective, by blocking chained source doesn't block any resources from being used, as this task running the source has a backlog of buffered input data waiting to be processed.
However from the correctness point of view, checkpointStarted(CheckpointBarrier)
and checkpointStopped(long)
methods could be empty no-op.
checkpointStarted
在接口中 org.apache.flink.runtime.io.network.partition.consumer.CheckpointableInput
public void checkpointStopped(long cancelledCheckpointId)
checkpointStopped
在接口中 org.apache.flink.runtime.io.network.partition.consumer.CheckpointableInput
public int getInputGateIndex()
getInputGateIndex
在接口中 org.apache.flink.runtime.io.network.partition.consumer.CheckpointableInput
public void convertToPriorityEvent(int channelIndex, int sequenceNumber) throws IOException
convertToPriorityEvent
在接口中 org.apache.flink.runtime.io.network.partition.consumer.CheckpointableInput
IOException
public int getInputIndex()
StreamTaskInput
getInputIndex
在接口中 StreamTaskInput<T>
public void close()
close
在接口中 Closeable
close
在接口中 AutoCloseable
public CompletableFuture<Void> prepareSnapshot(org.apache.flink.runtime.checkpoint.channel.ChannelStateWriter channelStateWriter, long checkpointId) throws org.apache.flink.runtime.checkpoint.CheckpointException
StreamTaskInput
prepareSnapshot
在接口中 StreamTaskInput<T>
org.apache.flink.runtime.checkpoint.CheckpointException
public org.apache.flink.runtime.jobgraph.OperatorID getOperatorID()
public SourceOperator<T,?> getOperator()
Copyright © 2014–2022 The Apache Software Foundation. All rights reserved.