Scalding

Value Members

1. final def !=(arg0: AnyRef): Boolean

   Definition Classes

   AnyRef

2. final def !=(arg0: Any): Boolean

   Definition Classes

   Any

3. final def ##(): Int

   Definition Classes

   AnyRef → Any

4. final def ==(arg0: AnyRef): Boolean

   Definition Classes

   AnyRef

5. final def ==(arg0: Any): Boolean

   Definition Classes

   Any

6. final def asInstanceOf[T0]: T0

   Definition Classes

   Any

7. def clone(): AnyRef

   Attributes

   protected[java.lang]

   Definition Classes

   AnyRef

   Annotations

   @throws()

8. implicit val dateRangeInjection: Injection[DateRange, Interval[Time]]

9. def emptyFlowProducer[T]: FlowProducer[TypedPipe[T]]

10. final def eq(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

11. def equals(arg0: Any): Boolean

    Definition Classes

    AnyRef → Any

12. def finalize(): Unit

    Attributes

    protected[java.lang]

    Definition Classes

    AnyRef

    Annotations

    @throws()

13. def forcePipe[U](pipe: TimedPipe[U]): TimedPipe[U]

    The typed API (currently, as of 0.

    The typed API (currently, as of 0.9.0) hides all flatMaps from cascading, so if you fork a pipe, the whole input is recomputed. This function forces a cascading node so that cascading can see that any forks of the output share a common prefix of operations

14. final def getClass(): Class[_]

    Definition Classes

    AnyRef → Any

15. def hashCode(): Int

    Definition Classes

    AnyRef → Any

16. def intersect(dr1: DateRange, dr2: DateRange): Option[DateRange]

17. final def isInstanceOf[T0]: Boolean

    Definition Classes

    Any

18. def limitTimes[T](range: Interval[Time], in: FlowToPipe[T]): FlowToPipe[T]

19. def memoize[T](pf: PipeFactory[T]): PipeFactory[T]

    Memoize the inner reader This is not a performance optimization, but a correctness one applicable to some cases (namely any function that mutates the FlowDef or does IO).

    Memoize the inner reader This is not a performance optimization, but a correctness one applicable to some cases (namely any function that mutates the FlowDef or does IO). Though we are working in a referentially transparent manner, the application of the function inside the PipeFactory (the Reader) mutates the FlowDef. For a fixed PipeFactory, we only want to mutate a given FlowDef once. If we memoize with this function, it guarantees that the PipeFactory is idempotent.

20. def merge[T](left: FlowToPipe[T], right: FlowToPipe[T]): FlowToPipe[T]

21. def minify(mode: Mode, desired: DateRange)(factory: (DateRange) ⇒ scalding.Source): Either[List[FailureReason], DateRange]

    Given a constructor function, computes the maximum available range of time or gives an error.

    Given a constructor function, computes the maximum available range of time or gives an error.

    Works by calling validateTaps on the Mappable, so if that does not work correctly this will be incorrect.

22. final def ne(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

23. final def notify(): Unit

    Definition Classes

    AnyRef

24. final def notifyAll(): Unit

    Definition Classes

    AnyRef

25. def pipeFactory[T](factory: (DateRange) ⇒ Mappable[T])(implicit timeOf: TimeExtractor[T]): PipeFactory[T]

    This uses minify to find the smallest subset we can run.

    This uses minify to find the smallest subset we can run. If you don't want this behavior, then use pipeFactoryExact which either produces all the DateRange or the whole job fails.

26. def pipeFactoryExact[T](factory: (DateRange) ⇒ Mappable[T])(implicit timeOf: TimeExtractor[T]): PipeFactory[T]

27. def plan[T](options: Map[String, Options], prod: Producer[Scalding, T]): PipeFactory[T]

28. def sourceFromMappable[T](factory: (DateRange) ⇒ Mappable[T])(implicit arg0: TimeExtractor[T], arg1: Manifest[T]): Producer[Scalding, T]

29. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes

    AnyRef

30. def toDateRange(timeSpan: Interval[Time]): Try[DateRange]

31. def toPipe[T](timeSpan: Interval[Time], flowDef: FlowDef, mode: Mode, pf: PipeFactory[T]): Try[(Interval[Time], TimedPipe[T])]

32. def toPipe[T](dr: DateRange, prod: Producer[Scalding, T], opts: Map[String, Options])(implicit fd: FlowDef, mode: Mode): Try[(DateRange, TypedPipe[(Long, T)])]

    Use this method to interop with existing scalding code Note this may return a smaller DateRange than you ask for If you need an exact DateRange see toPipeExact.

33. def toPipeExact[T](timeSpan: Interval[Time], flowDef: FlowDef, mode: Mode, pf: PipeFactory[T]): Try[TimedPipe[T]]

34. def toPipeExact[T](dr: DateRange, prod: Producer[Scalding, T], opts: Map[String, Options])(implicit fd: FlowDef, mode: Mode): Try[TypedPipe[(Long, T)]]

    Use this method to interop with existing scalding code that expects to schedule an exact DateRange or fail.

35. def toString(): String

    Definition Classes

    AnyRef → Any

36. final def wait(): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws()

37. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws()

38. final def wait(arg0: Long): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws()