Package org.apache.flink.runtime.scheduler

Class DefaultScheduler

java.lang.Object

org.apache.flink.runtime.scheduler.SchedulerBase

org.apache.flink.runtime.scheduler.DefaultScheduler

All Implemented Interfaces:

AutoCloseable, CheckpointScheduling, GlobalFailureHandler, SchedulerNG, SchedulerOperations, org.apache.flink.util.AutoCloseableAsync

Direct Known Subclasses:

AdaptiveBatchScheduler

public class DefaultScheduler
extends SchedulerBase
implements SchedulerOperations

The future default scheduler.

Field Summary

Fields

Modifier and Type	Field	Description
protected ExecutionDeployer	executionDeployer
protected ExecutionSlotAllocator	executionSlotAllocator
protected FailoverStrategy	failoverStrategy
protected org.slf4j.Logger	log
protected SchedulingStrategy	schedulingStrategy
protected ShuffleMaster<?>	shuffleMaster

Fields inherited from class org.apache.flink.runtime.scheduler.SchedulerBase

executionVertexVersioner, inputsLocationsRetriever, jobInfo, jobManagerJobMetricGroup, operatorCoordinatorHandler, stateLocationRetriever

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	DefaultScheduler(org.slf4j.Logger log, JobGraph jobGraph, Executor ioExecutor, org.apache.flink.configuration.Configuration jobMasterConfiguration, Consumer<org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor> startUpAction, org.apache.flink.util.concurrent.ScheduledExecutor delayExecutor, ClassLoader userCodeLoader, CheckpointsCleaner checkpointsCleaner, CheckpointRecoveryFactory checkpointRecoveryFactory, JobManagerJobMetricGroup jobManagerJobMetricGroup, SchedulingStrategyFactory schedulingStrategyFactory, FailoverStrategy.Factory failoverStrategyFactory, RestartBackoffTimeStrategy restartBackoffTimeStrategy, ExecutionOperations executionOperations, ExecutionVertexVersioner executionVertexVersioner, ExecutionSlotAllocatorFactory executionSlotAllocatorFactory, long initializationTimestamp, org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor mainThreadExecutor, JobStatusListener jobStatusListener, Collection<org.apache.flink.core.failure.FailureEnricher> failureEnrichers, ExecutionGraphFactory executionGraphFactory, ShuffleMaster<?> shuffleMaster, Duration rpcTimeout, VertexParallelismStore vertexParallelismStore, ExecutionDeployer.Factory executionDeployerFactory, ExecutionPlanSchedulingContext executionPlanSchedulingContext)

Method Summary

Modifier and Type	Method	Description
void	allocateSlotsAndDeploy(List<ExecutionVertexID> verticesToDeploy)	Allocate slots and deploy the vertex when slots are returned.
protected void	cancelAllPendingSlotRequestsForVertex(ExecutionVertexID executionVertexId)
protected void	cancelAllPendingSlotRequestsInternal()
protected CompletableFuture<?>	cancelExecution(Execution execution)
protected FailureHandlingResultSnapshot	createFailureHandlingResultSnapshot(FailureHandlingResult failureHandlingResult)
protected long	getNumberOfRescales()
protected long	getNumberOfRestarts()
protected ClassLoader	getUserCodeLoader()
void	handleGlobalFailure(Throwable error)	Handles a global failure.
protected void	handleTaskFailure(Execution failedExecution, Throwable error)
protected void	maybeRestartTasks(FailureHandlingResult failureHandlingResult)
protected void	notifyCoordinatorsAboutTaskFailure(Execution execution, Throwable error)
protected void	onTaskFailed(Execution execution)
protected void	onTaskFinished(Execution execution, IOMetrics ioMetrics)
protected FailureHandlingResult	recordTaskFailure(Execution failedExecution, Throwable error)
protected void	startSchedulingInternal()

Methods inherited from class org.apache.flink.runtime.scheduler.SchedulerBase

acknowledgeCheckpoint, archiveFromFailureHandlingResult, archiveGlobalFailure, cancel, closeAsync, computeVertexParallelismStore, computeVertexParallelismStore, computeVertexParallelismStore, computeVertexParallelismStore, declineCheckpoint, deliverCoordinationRequestToCoordinator, deliverOperatorEventToCoordinator, failJob, getDefaultMaxParallelism, getDefaultMaxParallelism, getExceptionHistory, getExecutionGraph, getExecutionJobVertex, getExecutionVertex, getJobGraph, getJobId, getJobTerminationFuture, getMainThreadExecutor, getMarkPartitionFinishedStrategy, getResultPartitionAvailabilityChecker, getSchedulingTopology, notifyEndOfData, notifyKvStateRegistered, notifyKvStateUnregistered, registerJobMetrics, reportCheckpointMetrics, reportInitializationMetrics, requestCheckpointStats, requestJob, requestJobStatus, requestKvStateLocation, requestNextInputSplit, requestPartitionState, resetForNewExecution, resetForNewExecutions, restoreState, setGlobalFailureCause, startCheckpointScheduler, startScheduling, stopCheckpointScheduler, stopWithSavepoint, transitionExecutionGraphState, transitionToRunning, transitionToScheduled, triggerCheckpoint, triggerSavepoint, updateAccumulators, updateTaskExecutionState

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.util.AutoCloseableAsync

close

Methods inherited from interface org.apache.flink.runtime.scheduler.SchedulerNG

requestJobResourceRequirements, updateJobResourceRequirements, updateTaskExecutionState

Field Detail

log

protected final org.slf4j.Logger log

executionSlotAllocator

protected final ExecutionSlotAllocator executionSlotAllocator

schedulingStrategy

protected final SchedulingStrategy schedulingStrategy

shuffleMaster

protected final ShuffleMaster<?> shuffleMaster

executionDeployer

protected final ExecutionDeployer executionDeployer

failoverStrategy

protected final FailoverStrategy failoverStrategy

Constructor Detail

DefaultScheduler

protected DefaultScheduler(org.slf4j.Logger log,
                           JobGraph jobGraph,
                           Executor ioExecutor,
                           org.apache.flink.configuration.Configuration jobMasterConfiguration,
                           Consumer<org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor> startUpAction,
                           org.apache.flink.util.concurrent.ScheduledExecutor delayExecutor,
                           ClassLoader userCodeLoader,
                           CheckpointsCleaner checkpointsCleaner,
                           CheckpointRecoveryFactory checkpointRecoveryFactory,
                           JobManagerJobMetricGroup jobManagerJobMetricGroup,
                           SchedulingStrategyFactory schedulingStrategyFactory,
                           FailoverStrategy.Factory failoverStrategyFactory,
                           RestartBackoffTimeStrategy restartBackoffTimeStrategy,
                           ExecutionOperations executionOperations,
                           ExecutionVertexVersioner executionVertexVersioner,
                           ExecutionSlotAllocatorFactory executionSlotAllocatorFactory,
                           long initializationTimestamp,
                           org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor mainThreadExecutor,
                           JobStatusListener jobStatusListener,
                           Collection<org.apache.flink.core.failure.FailureEnricher> failureEnrichers,
                           ExecutionGraphFactory executionGraphFactory,
                           ShuffleMaster<?> shuffleMaster,
                           Duration rpcTimeout,
                           VertexParallelismStore vertexParallelismStore,
                           ExecutionDeployer.Factory executionDeployerFactory,
                           ExecutionPlanSchedulingContext executionPlanSchedulingContext)
                    throws Exception

Throws:

Exception

Method Detail

getNumberOfRestarts

protected long getNumberOfRestarts()

Specified by:

getNumberOfRestarts in class SchedulerBase

getNumberOfRescales

protected long getNumberOfRescales()

Specified by:

getNumberOfRescales in class SchedulerBase

cancelAllPendingSlotRequestsInternal

protected void cancelAllPendingSlotRequestsInternal()

Specified by:

cancelAllPendingSlotRequestsInternal in class SchedulerBase

startSchedulingInternal

protected void startSchedulingInternal()

Specified by:

startSchedulingInternal in class SchedulerBase

onTaskFinished

protected void onTaskFinished(Execution execution,
                              IOMetrics ioMetrics)

Specified by:

onTaskFinished in class SchedulerBase

getUserCodeLoader

protected ClassLoader getUserCodeLoader()

onTaskFailed

protected void onTaskFailed(Execution execution)

Specified by:

onTaskFailed in class SchedulerBase

handleTaskFailure

protected void handleTaskFailure(Execution failedExecution,
                                 @Nullable
                                 Throwable error)

recordTaskFailure

protected FailureHandlingResult recordTaskFailure(Execution failedExecution,
                                                  @Nullable
                                                  Throwable error)

notifyCoordinatorsAboutTaskFailure

protected void notifyCoordinatorsAboutTaskFailure(Execution execution,
                                                  @Nullable
                                                  Throwable error)

handleGlobalFailure

public void handleGlobalFailure(Throwable error)

Description copied from interface: GlobalFailureHandler

Handles a global failure.

Specified by:

handleGlobalFailure in interface GlobalFailureHandler

Parameters:

error - A cause that describes the global failure.

maybeRestartTasks

protected void maybeRestartTasks(FailureHandlingResult failureHandlingResult)

createFailureHandlingResultSnapshot

protected FailureHandlingResultSnapshot createFailureHandlingResultSnapshot(FailureHandlingResult failureHandlingResult)

cancelExecution

protected CompletableFuture<?> cancelExecution(Execution execution)

cancelAllPendingSlotRequestsForVertex

protected void cancelAllPendingSlotRequestsForVertex(ExecutionVertexID executionVertexId)

allocateSlotsAndDeploy

public void allocateSlotsAndDeploy(List<ExecutionVertexID> verticesToDeploy)

Description copied from interface: SchedulerOperations

Allocate slots and deploy the vertex when slots are returned. Vertices will be deployed only after all of them have been assigned slots. The given order will be respected, i.e. tasks with smaller indices will be deployed earlier. Only vertices in CREATED state will be accepted. Errors will happen if scheduling Non-CREATED vertices.

Specified by:

allocateSlotsAndDeploy in interface SchedulerOperations

Parameters:

verticesToDeploy - The execution vertices to deploy