com.twitter.scalding

Execution

Related Docs: trait Execution | package scalding

object Execution extends Serializable

Execution has many methods for creating Execution[T] instances, which are the preferred way to compose computations in scalding libraries.

Linear Supertypes
Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. Execution
  2. Serializable
  3. Serializable
  4. AnyRef
  5. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Type Members

  1. trait EvalCache extends AnyRef

Value Members

  1. final def !=(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  4. implicit object ExecutionMonad extends Monad[Execution]

    This is an instance of Monad for execution so it can be used in functions that apply to all Monads

  5. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  6. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  7. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  8. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  9. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  10. def from[T](t: ⇒ T): Execution[T]

    This makes a constant execution that runs no job.

    This makes a constant execution that runs no job. Note this is a lazy parameter that is evaluated every time run is called.

  11. def fromFn[T](fn: (Config, Mode) ⇒ (FlowDef, (JobStats) ⇒ Future[T])): Execution[T]

    This converts a function into an Execution monad.

    This converts a function into an Execution monad. The flowDef returned is never mutated. The returned callback funcion is called after the flow is run and succeeds.

  12. def fromFuture[T](fn: (scala.concurrent.ExecutionContext) ⇒ Future[T]): Execution[T]

    The call to fn will happen when the run method on the result is called.

    The call to fn will happen when the run method on the result is called. The ConcurrentExecutionContext will be the same one used on run. This is intended for cases where you need to make asynchronous calls in the middle or end of execution. Presumably this is used with flatMap either before or after

  13. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  14. def getConfig: Execution[Config]

    Use this to read the configuration, which may contain Args or options which describe input on which to run

  15. def getConfigMode: Execution[(Config, Mode)]

    Use this to get the config and mode.

  16. def getMode: Execution[Mode]

    Use this to get the mode, which may contain the job conf

  17. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  18. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  19. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  20. final def notify(): Unit

    Definition Classes
    AnyRef
  21. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  22. def run[C](flow: Flow[C]): Future[JobStats]

  23. def sequence[T](exs: Seq[Execution[T]]): Execution[Seq[T]]

  24. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  25. def toString(): String

    Definition Classes
    AnyRef → Any
  26. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  27. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  28. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  29. def waitFor[C](flow: Flow[C]): Try[JobStats]

  30. def withId[T](fn: (UniqueID) ⇒ Execution[T]): Execution[T]

    Use this to use counters/stats with Execution.

    Use this to use counters/stats with Execution. You do this: Execution.withId { implicit uid => val myStat = Stat("myStat") // uid is implicitly pulled in pipe.map { t => if(someCase(t)) myStat.inc fn(t) } .writeExecution(mySink) }

  31. def zip[A, B, C, D, E](ax: Execution[A], bx: Execution[B], cx: Execution[C], dx: Execution[D], ex: Execution[E]): Execution[(A, B, C, D, E)]

    combine several executions and run them in parallel when .run is called

  32. def zip[A, B, C, D](ax: Execution[A], bx: Execution[B], cx: Execution[C], dx: Execution[D]): Execution[(A, B, C, D)]

    combine several executions and run them in parallel when .run is called

  33. def zip[A, B, C](ax: Execution[A], bx: Execution[B], cx: Execution[C]): Execution[(A, B, C)]

    combine several executions and run them in parallel when .run is called

  34. def zip[A, B](ax: Execution[A], bx: Execution[B]): Execution[(A, B)]

    combine several executions and run them in parallel when .run is called

Deprecated Value Members

  1. def buildFlow[T](conf: Config, mode: Mode)(op: Reader[ExecutionContext, T]): (T, Try[Flow[_]])

    This creates a new ExecutionContext, passes to the reader, builds the flow and cleans up the state of the FlowDef

    This creates a new ExecutionContext, passes to the reader, builds the flow and cleans up the state of the FlowDef

    Annotations
    @deprecated
    Deprecated

    (Since version 2014-07-29) Use Execution[T]

  2. def run[T](conf: Config, mode: Mode)(op: Reader[ExecutionContext, T]): (T, Future[JobStats])

    Annotations
    @deprecated
    Deprecated

    (Since version 2014-07-29) Use Execution[T].run

  3. def validateSources: Reader[ExecutionContext, Try[Unit]]

    Annotations
    @deprecated
    Deprecated

    (Since version 2014-07-29) Use Execution[T].run

  4. def waitFor[T](conf: Config, mode: Mode)(op: Reader[ExecutionContext, T]): (T, Try[JobStats])

    Annotations
    @deprecated
    Deprecated

    (Since version 2014-07-29) Use Execution[T].run

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped