appmaster

Type Members

1. class AppMaster extends ApplicationMaster

   

   AppMaster is the head of a streaming application.

   AppMaster is the head of a streaming application.

   It contains: 1. ExecutorManager to manage all executors. 2. TaskManager to manage all tasks, 3. ClockService to track the global clock for this streaming application. 4. Scheduler to decide which a task should be scheduled to.

2. class ClockService extends Actor with Stash

   

   The clockService will maintain a global view of message timestamp in the application

3. class DagManager extends Actor with Stash

   

   Will handle dag modification and other stuff related with DAG

4. class JarScheduler extends AnyRef

   

   With JarScheduler, we allows a DAG to be partitioned into several parts, with each part use its own jar file.

5. case class ProcessorSummary(id: ProcessorId, taskClass: String, parallelism: Int, description: String, taskConf: UserConfig, life: LifeTime, executors: List[ExecutorId], taskCount: Map[ExecutorId, TaskCount]) extends Product with Serializable

   

6. case class StreamAppMasterSummary(appType: String = "streaming", appId: Int, appName: String = null, actorPath: String = null, clock: TimeStamp = 0L, status: AppMasterStatus = MasterToAppMaster.AppMasterActive, startTime: TimeStamp = 0L, uptime: TimeStamp = 0L, user: String = null, homeDirectory: String = "", logFile: String = "", dag: Graph[ProcessorId, String] = null, executors: List[ExecutorBrief] = null, processors: Map[ProcessorId, ProcessorSummary] = ..., processorLevels: Map[ProcessorId, Int] = Map.empty[ProcessorId, Int], historyMetricsConfig: HistoryMetricsConfig = null) extends AppMasterSummary with Product with Serializable

   

7. case class TaskCount(count: Int) extends Product with Serializable

   

8. class TaskLocator extends AnyRef

   

   TaskLocator is used to decide which machine one task should run on.

   TaskLocator is used to decide which machine one task should run on.

   User can specify config Constants.GEARPUMP_STREAMING_LOCALITIES to decide to control which machine the task is running on.

9. class TaskRegistry extends AnyRef

   

   TaskRegistry is used to track the registration of all tasks when one application is booting up.

10. trait TaskScheduler extends AnyRef

    

    This schedules tasks to run for new allocated resources.

11. class TaskSchedulerImpl extends TaskScheduler