scalding

Type Members

1. type FactoryInput = (Interval[Timestamp], Mode)

   The Platform recursively passes this input around to describe a step forward: requested input time span, and scalding Mode

2. type FailureReason = String

   These are printed/logged only when we can't make any progress

3. type FlowInput = (FlowDef, Mode)

   When it is time to run build the final flow, this is what scalding needs.

   When it is time to run build the final flow, this is what scalding needs. It is modified in the Reader[FlowInput, T]

4. case class FlowPlanException(errs: List[FailureReason]) extends Exception with Product with Serializable

   Puts a type on the final exception thrown by the scalding Planner

5. type FlowProducer[+T] = Reader[(FlowDef, Mode), T]

   This is a function that modifies a flow to return T generally T will be some kind of TypedPipe

6. type FlowToPipe[+T] = Reader[(FlowDef, Mode), TimedPipe[T]]

   We so commonly talk about producing TimedPipe we define this

7. type KeyValuePipe[+K, +V] = TypedPipe[(Timestamp, (K, V))]

8. class Memo[T] extends Serializable

9. type PipeFactory[+T] = StateWithError[(Interval[Timestamp], Mode), List[FailureReason], FlowToPipe[T]]

   We are usually producing Pipes in the Planner

10. class PipeFactoryOps[+T] extends AnyRef

11. type PlannerOutput[+T] = StateWithError[(Interval[Timestamp], Mode), List[FailureReason], T]

    The recursive planner produces these objects which are Monads

12. class Scalding extends Platform[Scalding]

13. trait ScaldingExecutionConfig extends ChillExecutionConfig[Scalding]

14. trait Service[K, +V] extends Serializable

15. trait Sink[T] extends AnyRef

16. trait Store[K, V] extends Serializable

17. type TimedPipe[+T] = TypedPipe[(Timestamp, T)]

    How we represent the streams in scalding

18. type Try[+T] = Either[List[FailureReason], T]

19. case class WriteDot(filename: String) extends Product with Serializable

    Use this option to write the logical graph that cascading produces before Map/Reduce planning.

20. case class WriteStepsDot(filename: String) extends Product with Serializable

    Use this option to write map/reduce graph that cascading produces Use the job name as the key