Trait

com.groupon.sparklint.analyzer

SparklintAnalyzerLike

Related Doc: package analyzer

Permalink

trait SparklintAnalyzerLike extends AnyRef

An analyzer is responsible of providing useful stats after processing spark event logs

Since

6/13/16.

Linear Supertypes
AnyRef, Any
Known Subclasses
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SparklintAnalyzerLike
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def getCoreUtilizationPercentage: Option[Double]

    Permalink

    returns

    sum(idle time)/sum(allocated cpu time)

  2. abstract def getCumulativeCoreUsage: Option[Map[Int, Long]]

    Permalink

    returns

    cores -> milliseconds with this level of core usage

  3. abstract def getCurrentCores: Option[Int]

    Permalink

    returns

    how many cores are currently being used

  4. abstract def getCurrentTaskByExecutors: Option[Map[String, Iterable[SparklintTaskInfo]]]

    Permalink

    How tasks are allocated on each executor

    How tasks are allocated on each executor

    returns

    ExecutorId -> [TaskInfo]

  5. abstract def getExecutorInfo: Option[Map[String, SparklintExecutorInfo]]

    Permalink

    Get the cores for each executor allocated

    Get the cores for each executor allocated

    returns

    ExecutorId -> [ExecutorInfo]

  6. abstract def getFairSchedulerPools: Seq[String]

    Permalink

    If the app is using fair scheduler, return all the pools being used.

    If the app is using fair scheduler, return all the pools being used. Otherwise, empty

  7. abstract def getIdleTime: Option[Long]

    Permalink

    returns

    the number of milliseconds when the app is totally idle

  8. abstract def getIdleTimeSinceFirstTask: Option[Long]

    Permalink

    returns

    the number of milliseconds when the app is totally idle after the first task is submitted

  9. abstract def getLastUpdatedAt: Option[Long]

    Permalink

    returns

    Time when the last event was processed

  10. abstract def getLocalityStatsByStageIdentifier(stageIdentifier: SparklintStageIdentifier): Option[SparklintStageMetrics]

    Permalink

    stageIdentifier

    the identifier for a series of stages

    returns

    for each locality level, the cumulative task metrics for that locality level

  11. abstract def getMaxAllocatedCores: Option[Int]

    Permalink

    returns

    the highest allocated cores throughout the app history

  12. abstract def getMaxConcurrentTasks: Option[Int]

    Permalink

    returns

    the highest concurrent tasks throughout the app history

  13. abstract def getMaxCoreUsage: Option[Int]

    Permalink

    returns

    the highest core usage throughout the app history

  14. abstract def getRDDReferencedMoreThan(times: Int): Option[Seq[SparklintRDDInfo]]

    Permalink

    times

    filter rdds that has been referenced more than this many times

    returns

    rdd information

  15. abstract def getRunningTasks: Option[Int]

    Permalink

    Can be different from getCurrentCores since one task can sometimes use more than 1 cores

    Can be different from getCurrentCores since one task can sometimes use more than 1 cores

    returns

    how many tasks are currently being executed.

  16. abstract def getTimeSeriesCoreUsage: Option[Seq[CoreUsage]]

    Permalink

    returns

    [avg load], the time information is contained inside CoreUsage already

  17. abstract def getTimeUntilFirstTask: Option[Long]

    Permalink

    returns

    the number of milliseconds between sparklint listener was created and received first task info

Concrete Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  5. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  6. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  10. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  11. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  12. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  13. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  14. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  15. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  16. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  17. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  18. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  19. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from AnyRef

Inherited from Any

Ungrouped