task class
context class
user config
ActorRefProvider.actorOf
ActorRefProvider.actorOf
The actorRef of AppMaster
message send by upstream tasks
startTime that can be used to decide from when a source producer task should replay the data source, or from when a processor task should recover its checkpoint data in to in-memory state.
This can be used to cleanup resource when the application finished.
This can be used to cleanup resource when the application finished.
This can be used to output messages to downstream tasks.
This can be used to output messages to downstream tasks.
The data shuffling rule can be decided by Partitioner.
Use with caution, output unmanaged message to target tasks
Use with caution, output unmanaged message to target tasks
The task parallelism
The task parallelism
For example, we can create 3 source tasks, and 3 sink tasks, the task parallelism is 3 for each.
This can be useful when reading from partitioned data source. For example, for kafka, there may be 10 partitions, if we have parallelism of 2 for this task, then each task will be responsible to read data from 5 partitions.
handler for unmanaged message
ActorRefProducer.schedule
ActorRefProvider.scheduleOnce
Please don't use this if possible.
For managed message(type of Message), the sender only serve as a unique Id, It's address is not something meaningful, you should not use this directly
For managed message(type of Message), the sender only serve as a unique Id, It's address is not something meaningful, you should not use this directly
For unmanaged message, the sender represent the sender ActorRef
Please don't use this if possible
retrieve upstream min clock from TaskActor
This provides TaskContext for user defined tasks