org.apache.flink.runtime.jobmanager.scheduler

Class Scheduler

java.lang.Object

org.apache.flink.runtime.jobmanager.scheduler.Scheduler

All Implemented Interfaces:

InstanceListener, SlotProvider, SlotAvailabilityListener

public class Scheduler
extends Object
implements InstanceListener, SlotAvailabilityListener, SlotProvider

The scheduler is responsible for distributing the ready-to-run tasks among instances and slots.

The scheduler supports two scheduling modes:

• Immediate scheduling: A request for a task slot immediately returns a task slot, if one is available, or throws a NoResourceAvailableException.
• Queued Scheduling: A request for a task slot is queued and returns a future that will be fulfilled as soon as a slot becomes available.

Constructor Summary

Constructors

Constructor and Description
Scheduler(scala.concurrent.ExecutionContext executionContext) Creates a new scheduler.

Method Summary

Methods

Modifier and Type	Method and Description
Future<SimpleSlot>	allocateSlot(ScheduledUnit task, boolean allowQueued) Allocating slot with specific requirement.
protected SimpleSlot	getFreeSlotForTask(ExecutionVertex vertex, Iterable<TaskManagerLocation> requestedLocations, boolean localOnly) Gets a suitable instance to schedule the vertex execution to.
Map<String,List<Instance>>	getInstancesByHost()
protected SimpleSlot	getNewSlotForSharingGroup(ExecutionVertex vertex, Iterable<TaskManagerLocation> requestedLocations, SlotSharingGroupAssignment groupAssignment, CoLocationConstraint constraint, boolean localOnly) Tries to allocate a new slot for a vertex that is part of a slot sharing group.
int	getNumberOfAvailableInstances()
int	getNumberOfAvailableSlots() NOTE: In the presence of multi-threaded operations, this number may be inexact.
int	getNumberOfInstancesWithAvailableSlots()
int	getNumberOfLocalizedAssignments()
int	getNumberOfNonLocalizedAssignments()
int	getNumberOfUnconstrainedAssignments()
int	getTotalNumberOfSlots()
void	instanceDied(Instance instance) Called when an instance died.
void	newInstanceAvailable(Instance instance) Called when a new instance becomes available.
void	newSlotAvailable(Instance instance)
void	shutdown() Shuts the scheduler down.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Scheduler

public Scheduler(scala.concurrent.ExecutionContext executionContext)

Creates a new scheduler.

Method Detail

shutdown

public void shutdown()

Shuts the scheduler down. After shut down no more tasks can be added to the scheduler.

allocateSlot

public Future<SimpleSlot> allocateSlot(ScheduledUnit task,
                              boolean allowQueued)
                                throws NoResourceAvailableException

Description copied from interface: SlotProvider

Allocating slot with specific requirement.

Specified by:

allocateSlot in interface SlotProvider

Parameters:

task - The task to allocate the slot for

allowQueued - Whether allow the task be queued if we do not have enough resource

Returns:

The future of the allocation

Throws:

NoResourceAvailableException

getFreeSlotForTask

protected SimpleSlot getFreeSlotForTask(ExecutionVertex vertex,
                            Iterable<TaskManagerLocation> requestedLocations,
                            boolean localOnly)

Gets a suitable instance to schedule the vertex execution to.

NOTE: This method does is not thread-safe, it needs to be synchronized by the caller.

Parameters:

vertex - The task to run.

Returns:

The instance to run the vertex on, it null, if no instance is available.

getNewSlotForSharingGroup

protected SimpleSlot getNewSlotForSharingGroup(ExecutionVertex vertex,
                                   Iterable<TaskManagerLocation> requestedLocations,
                                   SlotSharingGroupAssignment groupAssignment,
                                   CoLocationConstraint constraint,
                                   boolean localOnly)

Tries to allocate a new slot for a vertex that is part of a slot sharing group. If one of the instances has a slot available, the method will allocate it as a shared slot, add that shared slot to the sharing group, and allocate a simple slot from that shared slot.

This method will try to allocate a slot from one of the local instances, and fall back to non-local instances, if permitted.

Parameters:

vertex - The vertex to allocate the slot for.

requestedLocations - The locations that are considered local. May be null or empty, if the vertex has no location preferences.

groupAssignment - The slot sharing group of the vertex. Mandatory parameter.

constraint - The co-location constraint of the vertex. May be null.

localOnly - Flag to indicate if non-local choices are acceptable.

Returns:

A sub-slot for the given vertex, or null, if no slot is available.

newSlotAvailable

public void newSlotAvailable(Instance instance)

Specified by:

newSlotAvailable in interface SlotAvailabilityListener

newInstanceAvailable

public void newInstanceAvailable(Instance instance)

Description copied from interface: InstanceListener

Called when a new instance becomes available.

Specified by:

newInstanceAvailable in interface InstanceListener

Parameters:

instance - The instance that became available.

instanceDied

public void instanceDied(Instance instance)

Description copied from interface: InstanceListener

Called when an instance died.

Specified by:

instanceDied in interface InstanceListener

Parameters:

instance - The instance that died.

getNumberOfAvailableSlots

public int getNumberOfAvailableSlots()

NOTE: In the presence of multi-threaded operations, this number may be inexact.

Returns:

The number of empty slots, for tasks.

getTotalNumberOfSlots

public int getTotalNumberOfSlots()

getNumberOfAvailableInstances

public int getNumberOfAvailableInstances()

getNumberOfInstancesWithAvailableSlots

public int getNumberOfInstancesWithAvailableSlots()

getInstancesByHost

public Map<String,List<Instance>> getInstancesByHost()

getNumberOfUnconstrainedAssignments

public int getNumberOfUnconstrainedAssignments()

getNumberOfLocalizedAssignments

public int getNumberOfLocalizedAssignments()

getNumberOfNonLocalizedAssignments

public int getNumberOfNonLocalizedAssignments()