op

Type Members

1. case class DataSinkOp[T](dataSink: DataSink, parallelism: Int, conf: UserConfig, description: String) extends ParameterizedOp[T] with Product with Serializable

   

2. case class DataSourceOp[T](dataSource: DataSource, parallelism: Int, conf: UserConfig, description: String) extends ParameterizedOp[T] with Product with Serializable

   

3. case class FlatMapOp[T, R](fun: (T) ⇒ TraversableOnce[R], description: String, conf: UserConfig = UserConfig.empty) extends SlaveOp[T] with Product with Serializable

   

4. case class GroupByOp[T, R](fun: (T) ⇒ R, parallelism: Int, description: String, conf: UserConfig = UserConfig.empty) extends ParameterizedOp[T] with Product with Serializable

   

5. trait MasterOp extends Op

   

6. case class MergeOp(description: String, conf: UserConfig = UserConfig.empty) extends MasterOp with Product with Serializable

   

7. sealed trait Op extends AnyRef

   

   Operators for the DSL

8. case class OpChain(ops: List[Op]) extends Op with Product with Serializable

   

   Contains operators which can be chained to single one.

   Contains operators which can be chained to single one.

   For example, flatmap().map().reduce() can be chained to single operator as no data shuffling is required.

   ops

   list of operations

9. trait OpEdge extends AnyRef

   

10. trait ParameterizedOp[T] extends MasterOp

    

11. case class ProcessorOp[T <: Task](processor: Class[T], parallelism: Int, conf: UserConfig, description: String) extends ParameterizedOp[T] with Product with Serializable

    

12. case class ReduceOp[T](fun: (T, T) ⇒ T, description: String, conf: UserConfig = UserConfig.empty) extends SlaveOp[T] with Product with Serializable

    

13. trait SlaveOp[T] extends Op

    

    When translated to running DAG, SlaveOP can be attach to MasterOP or other SlaveOP "Attach" means running in same Actor.