org.apache.spark.deploy

history

package history

Visibility
  1. Public
  2. All

Type Members

  1. class HistoryServer extends WebUI with Logging

    A web server that renders SparkUIs of completed applications.

    A web server that renders SparkUIs of completed applications.

    For the standalone mode, MasterWebUI already achieves this functionality. Thus, the main use case of the HistoryServer is in other deploy modes (e.g. Yarn or Mesos).

    The logging directory structure is as follows: Within the given base directory, each application's event logs are maintained in the application's own sub-directory. This is the same structure as maintained in the event log write code path in EventLoggingListener.

Value Members

  1. object HistoryServer extends Logging

    The recommended way of starting and stopping a HistoryServer is through the scripts start-history-server.sh and stop-history-server.sh.

    The recommended way of starting and stopping a HistoryServer is through the scripts start-history-server.sh and stop-history-server.sh. The path to a base log directory is must be specified, while the requested UI port is optional. For example:

    ./sbin/spark-history-server.sh /tmp/spark-events ./sbin/spark-history-server.sh hdfs://1.2.3.4:9000/spark-events

    This launches the HistoryServer as a Spark daemon.

Ungrouped