Packages

c

io.github.dutrevis

CPUMetrics

class CPUMetrics extends SparkPlugin

Collects CPU resource metrics from a unix-based operating system.

Use when Spark is running in clusters with standalone, Mesos or YARN resource managers.

CPU metrics are obtained from the numbers of the first line of the /proc/stat file, available at the proc pseudo-filesystem of unix-based operating systems. These numbers identify the amount of time the CPU has spent performing different kinds of work, arranged in columns at the following order: "cpu_user", "cpu_nice", "cpu_system", "cpu_idle", "cpu_iowait", "cpu_irq" and "cpu_softirq".

Note

All of the numbers retrieved are aggregates since the system first booted.

,

Time units are in USER_HZ or Jiffies (typically hundredths of a second)

,

Values for "cpu_steal", "cpu_guest" and "cpu_guest_nice", available at spectific Linux versions, are not parsed from the file.

Linear Supertypes
SparkPlugin, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. CPUMetrics
  2. SparkPlugin
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Instance Constructors

  1. new CPUMetrics()

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##: Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
  6. def collectIdleCPU(metricCollector: StatMetricCollector): Double

    Collects the aggregated CPU usage value for when no processes are running, as an average out of the total of the CPU usage time for all processes.

    Collects the aggregated CPU usage value for when no processes are running, as an average out of the total of the CPU usage time for all processes.

    metricCollector

    a StatMetricCollector instance

    Note

    Collected value is of type Double with precision of 2.

  7. def collectNiceCPU(metricCollector: StatMetricCollector): Double

    Collects the aggregated CPU usage value for niced processes (there is, run with the nice command) executing in user mode and calculates its average out of the total of the CPU usage time for all processes.

    Collects the aggregated CPU usage value for niced processes (there is, run with the nice command) executing in user mode and calculates its average out of the total of the CPU usage time for all processes.

    metricCollector

    a StatMetricCollector instance

    Note

    Collected value is of type Double with precision of 2.

  8. def collectSystemCPU(metricCollector: StatMetricCollector): Double

    Collects the aggregated CPU usage value processes executing in kernel mode as an average out of the total of the CPU usage time for all processes.

    Collects the aggregated CPU usage value processes executing in kernel mode as an average out of the total of the CPU usage time for all processes.

    metricCollector

    a StatMetricCollector instance

    Note

    Collected value is of type Double with precision of 2.

  9. def collectUserCPU(metricCollector: StatMetricCollector): Double

    Collects the aggregated CPU usage value for normal processes executing in user mode, as an average out of the total of CPU usage time for all processes.

    Collects the aggregated CPU usage value for normal processes executing in user mode, as an average out of the total of CPU usage time for all processes.

    metricCollector

    a StatMetricCollector instance

    Note

    Collected value is of type Double with precision of 2.

  10. def collectWaitCPU(metricCollector: StatMetricCollector): Double

    Collects the aggregated CPU usage value for when it's waiting for I/O to complete, as an average out of the total of the CPU usage time for all processes.

    Collects the aggregated CPU usage value for when it's waiting for I/O to complete, as an average out of the total of the CPU usage time for all processes.

    metricCollector

    a StatMetricCollector instance

    Note

    Collected value is of type Double with precision of 2.

  11. def createGaugeMetric(metricCollector: StatMetricCollector, collectorMethod: (StatMetricCollector) => Double): Gauge[Double]

    Creates a Dropwizard's Gauge metric type - an instantaneous reading of a particular value -, setting the provided collector method as the getValue method of the Gauge instance.

    Creates a Dropwizard's Gauge metric type - an instantaneous reading of a particular value -, setting the provided collector method as the getValue method of the Gauge instance.

    metricCollector

    a StatMetricCollector instance

    collectorMethod

    a method that receives the metricCollector and returns a metric value as a Double

  12. def driverPlugin(): DriverPlugin

    Returns the plugin's driver-side component.

    Returns the plugin's driver-side component. The returned DriverPlugin instance is called once, early in the initialization of the Spark driver. The operation it performs consists in the sequential registration of each mapped metric as a Gauge metric into an existing metric Registry. A test call is executed once on each collector method before its registration, assuring that the metric is available to be read and collected from the local OS, thus preventing future errors when the Gauge is first executed by the metrics system. If a NoSuchElementException is thrown in this attempt, the method is not registered, enabling the registration of the subsequent mapped metrics by the plugin. The plugin component ends its execution once all metrics are registered, leaving to the Dropwizard's Metrics system the job of collecting and exporting the registered metrics in a pre or user-defined frequency.

    returns

    An instance of the DriverPlugin

    Definition Classes
    CPUMetrics → SparkPlugin
    Note

    The driver's initialization is blocked during the operations inside init, so heavy performing operations must be avoided.

    ,

    The overriden init method must return a Map, that will be provided as extraConf to an ExecutorPlugin instance.

  13. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  14. def equals(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef → Any
  15. def executorPlugin(): ExecutorPlugin

    Returns the plugin's executor-side component.

    Returns the plugin's executor-side component. The returned ExecutorPlugin instance is called once, early in the initialization of the executor process. The operation it performs consists in the sequential registration of each mapped metric as a Gauge metric into an existing metric Registry. A test call is executed once on each collector method before its registration, assuring that the metric is available to be read and collected from the local OS, thus preventing future errors when the Gauge is first executed by the metrics system. If a NoSuchElementException is thrown in this attempt, the method is not registered, enabling the registration of the subsequent mapped metrics by the plugin. The plugin component ends its execution once all metrics are registered, leaving to the Dropwizard's Metrics system the job of collecting and exporting the registered metrics in a pre or user-defined frequency.

    returns

    An instance of the ExecutorPlugin Unit

    Definition Classes
    CPUMetrics → SparkPlugin
    Note

    The executor's initialization is blocked during the operations inside init, so heavy performing operations must be avoided.

  16. final def getClass(): Class[_ <: AnyRef]
    Definition Classes
    AnyRef → Any
    Annotations
    @IntrinsicCandidate() @native()
  17. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @IntrinsicCandidate() @native()
  18. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  19. val metricMapping: Map[String, (StatMetricCollector) => Double]

    Maps the collector methods to their respective metric names, that will be displayed in the Dropwizard's metric system.

  20. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  21. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @IntrinsicCandidate() @native()
  22. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @IntrinsicCandidate() @native()
  23. def registerMetric(metricRegistry: MetricRegistry, metricName: String, metricInstance: Metric): Unit

    Registers a provided Dropwizard's Metric instance into a metric registry under a metric name.

    Registers a provided Dropwizard's Metric instance into a metric registry under a metric name.

    metricRegistry

    a MetricRegistry instance from dropwizard.metrics

    metricName

    a metric name as a String

    metricInstance

    an instance of a dropwizard's Metric class to be registered

    Exceptions thrown

    IllegalArgumentException if the metric name is already registered

  24. final def synchronized[T0](arg0: => T0): T0
    Definition Classes
    AnyRef
  25. def toString(): String
    Definition Classes
    AnyRef → Any
  26. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  27. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException]) @native()
  28. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.Throwable]) @Deprecated
    Deprecated

    (Since version 9)

Inherited from SparkPlugin

Inherited from AnyRef

Inherited from Any

Ungrouped