Class/Object

org.apache.spark.deploy.history

HistoryServer

Related Docs: object HistoryServer | package history

Permalink

class HistoryServer extends WebUI with Logging with UIRoot with ApplicationCacheOperations

A web server that renders SparkUIs of completed applications.

For the standalone mode, MasterWebUI already achieves this functionality. Thus, the main use case of the HistoryServer is in other deploy modes (e.g. Yarn or Mesos).

The logging directory structure is as follows: Within the given base directory, each application's event logs are maintained in the application's own sub-directory. This is the same structure as maintained in the event log write code path in EventLoggingListener.

Linear Supertypes
ApplicationCacheOperations, UIRoot, WebUI, Logging, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. HistoryServer
  2. ApplicationCacheOperations
  3. UIRoot
  4. WebUI
  5. Logging
  6. AnyRef
  7. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new HistoryServer(conf: SparkConf, provider: ApplicationHistoryProvider, securityManager: SecurityManager, port: Int)

    Permalink

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. def addStaticHandler(resourceBase: String, path: String): Unit

    Permalink

    Add a handler for static content.

    Add a handler for static content.

    resourceBase

    Root of where to find resources to serve.

    path

    Path in UI where to mount the resources.

    Definition Classes
    WebUI
  5. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  6. def attachHandler(handler: ServletContextHandler): Unit

    Permalink

    Attach a handler to this UI.

    Attach a handler to this UI.

    Definition Classes
    WebUI
  7. def attachPage(page: WebUIPage): Unit

    Permalink

    Attach a page to this UI.

    Attach a page to this UI.

    Definition Classes
    WebUI
  8. def attachSparkUI(appId: String, attemptId: Option[String], ui: SparkUI, completed: Boolean): Unit

    Permalink

    Attach a reconstructed UI to this server.

    Attach a reconstructed UI to this server. Only valid after bind().

    appId

    application ID

    attemptId

    attempt ID

    ui

    UI

    completed

    flag to indicate that the UI has completed

    Definition Classes
    HistoryServer → ApplicationCacheOperations
  9. def attachTab(tab: WebUITab): Unit

    Permalink

    Attach a tab to this UI, along with all of its attached pages.

    Attach a tab to this UI, along with all of its attached pages.

    Definition Classes
    WebUI
  10. def bind(): Unit

    Permalink

    Bind to the HTTP server behind this web interface.

    Bind to the HTTP server behind this web interface.

    Definition Classes
    HistoryServer → WebUI
  11. def boundPort: Int

    Permalink

    Return the actual port to which this server is bound.

    Return the actual port to which this server is bound. Only valid after bind().

    Definition Classes
    WebUI
  12. val cacheMetrics: CacheMetrics

    Permalink
  13. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  14. def detachHandler(handler: ServletContextHandler): Unit

    Permalink

    Detach a handler from this UI.

    Detach a handler from this UI.

    Definition Classes
    WebUI
  15. def detachPage(page: WebUIPage): Unit

    Permalink
    Definition Classes
    WebUI
  16. def detachSparkUI(appId: String, attemptId: Option[String], ui: SparkUI): Unit

    Permalink

    Detach a reconstructed UI from this server.

    Detach a reconstructed UI from this server. Only valid after bind().

    ui

    Spark UI

    Definition Classes
    HistoryServer → ApplicationCacheOperations
  17. def detachTab(tab: WebUITab): Unit

    Permalink
    Definition Classes
    WebUI
  18. def emptyListingHtml(): Seq[Node]

    Permalink

    returns

    html text to display when the application list is empty

  19. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  20. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  21. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  22. def getAppUI(appId: String, attemptId: Option[String]): Option[LoadedAppUI]

    Permalink

    Get the application UI and whether or not it is completed

    Get the application UI and whether or not it is completed

    appId

    application ID

    attemptId

    attempt ID

    returns

    If found, the Spark UI and any history information to be used in the cache

    Definition Classes
    HistoryServer → ApplicationCacheOperations
  23. def getApplicationInfo(appId: String): Option[ApplicationInfo]

    Permalink
    Definition Classes
    HistoryServer → UIRoot
  24. def getApplicationInfoList: Iterator[ApplicationInfo]

    Permalink
    Definition Classes
    HistoryServer → UIRoot
  25. def getApplicationList(): Iterator[ApplicationHistoryInfo]

    Permalink

    Returns a list of available applications, in descending order according to their end time.

    Returns a list of available applications, in descending order according to their end time.

    returns

    List of all known applications.

  26. def getBasePath: String

    Permalink
    Definition Classes
    WebUI
  27. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  28. def getEventLogsUnderProcess(): Int

    Permalink
  29. def getHandlers: Seq[ServletContextHandler]

    Permalink
    Definition Classes
    WebUI
  30. def getLastUpdatedTime(): Long

    Permalink
  31. def getProviderConfig(): Map[String, String]

    Permalink

    Returns the provider configuration to show in the listing page.

    Returns the provider configuration to show in the listing page.

    returns

    A map with the provider's configuration.

  32. def getSecurityManager: SecurityManager

    Permalink
    Definition Classes
    WebUI
  33. def getSparkUI(appKey: String): Option[SparkUI]

    Permalink
    Definition Classes
    HistoryServer → UIRoot
  34. def getTabs: Seq[WebUITab]

    Permalink
    Definition Classes
    WebUI
  35. val handlers: ArrayBuffer[ServletContextHandler]

    Permalink
    Attributes
    protected
    Definition Classes
    WebUI
  36. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  37. def initialize(): Unit

    Permalink

    Initialize the history server.

    Initialize the history server.

    This starts a background thread that periodically synchronizes information displayed on this UI with the event logs in the provided base directory.

    Definition Classes
    HistoryServer → WebUI
  38. def initializeLogIfNecessary(isInterpreter: Boolean): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  39. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  40. def isTraceEnabled(): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  41. def log: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  42. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  43. def logDebug(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  44. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  45. def logError(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  46. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  47. def logInfo(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  48. def logName: String

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  49. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  50. def logTrace(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  51. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  52. def logWarning(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  53. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  54. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  55. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  56. val pageToHandlers: HashMap[WebUIPage, ArrayBuffer[ServletContextHandler]]

    Permalink
    Attributes
    protected
    Definition Classes
    WebUI
  57. val publicHostName: String

    Permalink
    Attributes
    protected
    Definition Classes
    WebUI
  58. def removeStaticHandler(path: String): Unit

    Permalink

    Remove a static content handler.

    Remove a static content handler.

    path

    Path in UI to unmount.

    Definition Classes
    WebUI
  59. val securityManager: SecurityManager

    Permalink
    Definition Classes
    WebUI
  60. var serverInfo: Option[ServerInfo]

    Permalink
    Attributes
    protected
    Definition Classes
    WebUI
  61. val sslOptions: SSLOptions

    Permalink
    Definition Classes
    WebUI
  62. def stop(): Unit

    Permalink

    Stop the server and close the file system.

    Stop the server and close the file system.

    Definition Classes
    HistoryServer → WebUI
  63. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  64. val tabs: ArrayBuffer[WebUITab]

    Permalink
    Attributes
    protected
    Definition Classes
    WebUI
  65. def toString(): String

    Permalink

    String value for diagnostics.

    String value for diagnostics.

    returns

    a multi-line description of the server state.

    Definition Classes
    HistoryServer → AnyRef → Any
  66. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  67. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  68. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  69. def webUrl: String

    Permalink

    Return the url of web interface.

    Return the url of web interface. Only valid after bind().

    Definition Classes
    WebUI
  70. def writeEventLogs(appId: String, attemptId: Option[String], zipStream: ZipOutputStream): Unit

    Permalink

    Write the event logs for the given app to the ZipOutputStream instance.

    Write the event logs for the given app to the ZipOutputStream instance. If attemptId is None, event logs for all attempts of this application will be written out.

    Definition Classes
    HistoryServer → UIRoot

Inherited from ApplicationCacheOperations

Inherited from UIRoot

Inherited from WebUI

Inherited from Logging

Inherited from AnyRef

Inherited from Any

Ungrouped