typed

Type Members

1. trait CoGroupable[K, +R] extends HasReducers with Serializable

   Represents something than can be CoGrouped with another CoGroupable

2. trait CoGrouped[K, +R] extends KeyedListLike[K, R, CoGrouped] with CoGroupable[K, R] with WithReducers[CoGrouped[K, R]]

3. abstract class CoGroupedJoiner[K] extends Joiner

4. case class ComputedValue[T](toTypedPipe: TypedPipe[T]) extends ValuePipe[T] with Product with Serializable

5. case class Converter[R](conv: TupleConverter[R]) extends FlatMapFn[R] with Product with Serializable

6. class DistinctCoGroupJoiner[K] extends CoGroupedJoiner[K]

7. final case class EmptyTypedPipe(fd: FlowDef, mode: Mode) extends TypedPipe[Nothing] with Product with Serializable

8. case class EmptyValue()(implicit flowDef: FlowDef, mode: Mode) extends ValuePipe[Nothing] with Product with Serializable

9. case class FilteredFn[R](fmap: FlatMapFn[R], fn: (R) ⇒ Boolean) extends FlatMapFn[R] with Product with Serializable

10. sealed trait FlatMapFn[+R] extends (TupleEntry) ⇒ TraversableOnce[R] with Serializable

    Closures are difficult for serialization.

11. case class FlatMappedFn[T, R](fmap: FlatMapFn[T], fn: (T) ⇒ TraversableOnce[R]) extends FlatMapFn[R] with Product with Serializable

12. trait Grouped[K, +V] extends KeyedListLike[K, V, UnsortedGrouped] with HashJoinable[K, V] with Sortable[V, [+x]SortedGrouped[K, x] with Reversable[SortedGrouped[K, x]]] with WithReducers[Grouped[K, V]]

    This encodes the rules that 1) sorting is only possible before doing any reduce, 2) reversing is only possible after sorting.

13. trait HasReducers extends AnyRef

    used for types that may know how many reducers they need e.

14. trait HashJoinable[K, +V] extends CoGroupable[K, V] with KeyedPipe[K]

    If we can HashJoin, then we can CoGroup, but not vice-versa i.

15. class HashJoiner[K, V, W, R] extends Joiner

    Only intended to be use to implement the hashCogroup on TypedPipe/Grouped

16. case class IdentityReduce[K, V1](keyOrdering: Ordering[K], mapped: TypedPipe[(K, V1)], reducers: Option[Int]) extends ReduceStep[K, V1] with Grouped[K, V1] with Product with Serializable

17. case class IdentityValueSortedReduce[K, V1](keyOrdering: Ordering[K], mapped: TypedPipe[(K, V1)], valueSort: Ordering[_ >: V1], reducers: Option[Int]) extends ReduceStep[K, V1] with SortedGrouped[K, V1] with Reversable[IdentityValueSortedReduce[K, V1]] with Product with Serializable

18. final case class IterablePipe[T](iterable: Iterable[T], fd: FlowDef, mode: Mode) extends TypedPipe[T] with Product with Serializable

    You should use a view here If you avoid toPipe, this class is more efficient than IterableSource.

19. case class IteratorMappedReduce[K, V1, V2](keyOrdering: Ordering[K], mapped: TypedPipe[(K, V1)], reduceFn: (K, Iterator[V1]) ⇒ Iterator[V2], reducers: Option[Int]) extends ReduceStep[K, V1] with UnsortedGrouped[K, V2] with Product with Serializable

20. trait KeyedList[K, +T] extends KeyedListLike[K, T, KeyedList]

    This is for the case where you don't want to expose any structure but the ability to operate on an iterator of the values

21. trait KeyedListLike[K, +T, +This[K, +T] <: KeyedListLike[K, T, This]] extends Serializable

    Represents sharded lists of items of type T There are exactly two the fundamental operations: toTypedPipe: marks the end of the grouped-on-key operations.

22. trait KeyedPipe[K] extends AnyRef

    Represents anything that starts as a TypedPipe of Key Value, where the value type has been erased.

23. case class LiteralValue[T](value: T)(implicit flowDef: FlowDef, mode: Mode) extends ValuePipe[T] with Product with Serializable

24. case class MapFn[T, R](fmap: FlatMapFn[T], fn: (T) ⇒ R) extends FlatMapFn[R] with Product with Serializable

25. class MappablePipeJoinEnrichment[T] extends AnyRef

26. final case class MergedTypedPipe[T](left: TypedPipe[T], right: TypedPipe[T]) extends TypedPipe[T] with Product with Serializable

27. class PipeTExtensions extends Serializable

28. sealed trait ReduceStep[K, V1] extends KeyedPipe[K]

    This is a class that models the logical portion of the reduce step.

29. trait Reversable[+R] extends AnyRef

30. case class SketchJoined[K, V, V2, R](left: Sketched[K, V], right: TypedPipe[(K, V2)], numReducers: Int)(joiner: (K, V, Iterable[V2]) ⇒ Iterator[R])(implicit evidence$1: Ordering[K]) extends HasReducers with Product with Serializable

31. case class Sketched[K, V](pipe: TypedPipe[(K, V)], numReducers: Int, delta: Double, eps: Double, seed: Int)(implicit serialization: (K) ⇒ Array[Byte], ordering: Ordering[K]) extends HasReducers with Product with Serializable

32. trait Sortable[+T, +Sorted[+_]] extends AnyRef

33. trait SortedGrouped[K, +V] extends KeyedListLike[K, V, SortedGrouped] with WithReducers[SortedGrouped[K, V]]

    After sorting, we are no longer CoGroupable, and we can only call reverse in the initial SortedGrouped created from the Sortable: .

34. class TuplePipeJoinEnrichment[K, V] extends AnyRef

35. trait TypedPipe[+T] extends Serializable

    Think of a TypedPipe as a distributed unordered list that may or may not yet have been materialized in memory or disk.

36. final case class TypedPipeInst[T](inpipe: Pipe, fields: Fields, flatMapFn: FlatMapFn[T]) extends TypedPipe[T] with Product with Serializable

    This is an instance of a TypedPipe that wraps a cascading Pipe

37. trait TypedSink[-T] extends Serializable

    Opposite of TypedSource, used for writing into

38. trait TypedSource[+T] extends Serializable

39. trait UnsortedGrouped[K, +V] extends KeyedListLike[K, V, UnsortedGrouped] with HashJoinable[K, V] with WithReducers[UnsortedGrouped[K, V]]

    This is the state after we have done some reducing.

40. sealed trait ValuePipe[+T] extends Serializable

    ValuePipe is special case of a TypedPipe of just a optional single element.

41. case class ValueSortedReduce[K, V1, V2](keyOrdering: Ordering[K], mapped: TypedPipe[(K, V1)], valueSort: Ordering[_ >: V1], reduceFn: (K, Iterator[V1]) ⇒ Iterator[V2], reducers: Option[Int]) extends ReduceStep[K, V1] with SortedGrouped[K, V2] with Product with Serializable

42. trait WithReducers[+This <: WithReducers[This]] extends HasReducers

    used for objects that may _set_ how many reducers they need e.