package executor
Executor components used with various cluster managers. See org.apache.spark.executor.Executor.
- Alphabetic
- By Inheritance
- executor
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Type Members
-
class
ExecutorMetrics extends Serializable
:: DeveloperApi :: Metrics tracked for executors and the driver.
:: DeveloperApi :: Metrics tracked for executors and the driver.
Executor-level metrics are sent from each executor to the driver as part of the Heartbeat.
- Annotations
- @DeveloperApi()
-
class
InputMetrics extends Serializable
:: DeveloperApi :: A collection of accumulators that represents metrics about reading data from external systems.
:: DeveloperApi :: A collection of accumulators that represents metrics about reading data from external systems.
- Annotations
- @DeveloperApi()
-
class
OutputMetrics extends Serializable
:: DeveloperApi :: A collection of accumulators that represents metrics about writing data to external systems.
:: DeveloperApi :: A collection of accumulators that represents metrics about writing data to external systems.
- Annotations
- @DeveloperApi()
-
class
ShuffleReadMetrics extends Serializable
:: DeveloperApi :: A collection of accumulators that represent metrics about reading shuffle data.
:: DeveloperApi :: A collection of accumulators that represent metrics about reading shuffle data. Operations are not thread-safe.
- Annotations
- @DeveloperApi()
-
class
ShuffleWriteMetrics extends ShuffleWriteMetricsReporter with Serializable
:: DeveloperApi :: A collection of accumulators that represent metrics about writing shuffle data.
:: DeveloperApi :: A collection of accumulators that represent metrics about writing shuffle data. Operations are not thread-safe.
- Annotations
- @DeveloperApi()
-
class
TaskMetrics extends Serializable
:: DeveloperApi :: Metrics tracked during the execution of a task.
:: DeveloperApi :: Metrics tracked during the execution of a task.
This class is wrapper around a collection of internal accumulators that represent metrics associated with a task. The local values of these accumulators are sent from the executor to the driver when the task completes. These values are then merged into the corresponding accumulator previously registered on the driver.
The accumulator updates are also sent to the driver periodically (on executor heartbeat) and when the task failed with an exception. The TaskMetrics object itself should never be sent to the driver.
- Annotations
- @DeveloperApi()
Value Members
-
object
DataReadMethod extends Enumeration with Serializable
:: DeveloperApi :: Method by which input data was read.
:: DeveloperApi :: Method by which input data was read. Network means that the data was read over the network from a remote block manager (which may have stored the data on-disk or in-memory). Operations are not thread-safe.
- Annotations
- @DeveloperApi()
-
object
DataWriteMethod extends Enumeration with Serializable
:: DeveloperApi :: Method by which output data was written.
:: DeveloperApi :: Method by which output data was written. Operations are not thread-safe.
- Annotations
- @DeveloperApi()