Type Members

1.  abstract class AbstractPodsAllocator extends AnyRef

   :: DeveloperApi :: A abstract interface for allowing different types of pods allocation.

   :: DeveloperApi :: A abstract interface for allowing different types of pods allocation.

   The internal Spark implementations are StatefulSetPodsAllocator and ExecutorPodsAllocator. This may be useful for folks integrating with custom schedulers.

   This API may change or be removed at anytime.

   Annotations

   @DeveloperApi()

   Since

   3.3.0

2.  sealed trait ExecutorPodState extends AnyRef
3.  class ExecutorPodsAllocator extends AbstractPodsAllocator with Logging
4.  class ExecutorPodsPollingSnapshotSource extends Logging

   :: DeveloperApi ::

   :: DeveloperApi ::

   A class used for polling K8s executor pods by ExternalClusterManagers.

   Annotations

   @Stable() @DeveloperApi()

   Since

   3.1.3

5.  class ExecutorPodsWatchSnapshotSource extends Logging

   :: DeveloperApi ::

   :: DeveloperApi ::

   A class used for watching K8s executor pods by ExternalClusterManagers.

   Annotations

   @Stable() @DeveloperApi()

   Since

   3.1.3

6.  class ExecutorRollDriverPlugin extends DriverPlugin with Logging
7.  class ExecutorRollPlugin extends SparkPlugin

   Spark plugin to roll executor pods periodically.

   Spark plugin to roll executor pods periodically. This is independent from ExecutorPodsAllocator and aims to decommission executors one by one in both static and dynamic allocation.

   To use this plugin, we assume that a user has the required maximum number of executors + 1 in both static and dynamic allocation configurations.

8.  sealed trait FinalPodState extends ExecutorPodState
9.  case class GenerateExecID(podName: String) extends Serializable with Product
10.  case class PodDeleted(pod: Pod) extends FinalPodState with Product with Serializable
11.  case class PodFailed(pod: Pod) extends FinalPodState with Product with Serializable
12.  case class PodPending(pod: Pod) extends ExecutorPodState with Product with Serializable
13.  case class PodRunning(pod: Pod) extends ExecutorPodState with Product with Serializable
14.  case class PodSucceeded(pod: Pod) extends FinalPodState with Product with Serializable
15.  case class PodTerminating(pod: Pod) extends FinalPodState with Product with Serializable
16.  case class PodUnknown(pod: Pod) extends ExecutorPodState with Product with Serializable
17.  class StatefulSetPodsAllocator extends AbstractPodsAllocator with Logging