com.twitter.scalding.typed

Grouped

class Grouped[K, +T] extends KeyedList[K, T] with Serializable

Represents a grouping which is the transition from map to reduce phase in hadoop. Grouping is on a key of type K by ordering Ordering[K].

Linear Supertypes
Serializable, KeyedList[K, T], AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. Grouped
  2. Serializable
  3. KeyedList
  4. AnyRef
  5. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  6. def aggregate[B, C](agg: Aggregator[T, B, C]): TypedPipe[(K, C)]

    Use Algebird Aggregator to do the reduction

    Use Algebird Aggregator to do the reduction

    Definition Classes
    KeyedList
  7. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  8. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  9. def cogroup[W, R](smaller: Grouped[K, W])(joiner: (K, Iterator[T], Iterable[W]) ⇒ Iterator[R]): KeyedList[K, R]

  10. def count(fn: (T) ⇒ Boolean): TypedPipe[(K, Long)]

    Definition Classes
    KeyedList
  11. def drop(n: Int): KeyedList[K, T]

    Selects all elements except first n ones.

    Selects all elements except first n ones.

    Definition Classes
    KeyedList
  12. def dropWhile(p: (T) ⇒ Boolean): KeyedList[K, T]

    Drops longest prefix of elements that satisfy the given predicate.

    Drops longest prefix of elements that satisfy the given predicate.

    Definition Classes
    KeyedList
  13. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  14. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  15. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  16. def foldLeft[B](z: B)(fn: (B, T) ⇒ B): TypedPipe[(K, B)]

    Definition Classes
    KeyedList
  17. def forall(fn: (T) ⇒ Boolean): TypedPipe[(K, Boolean)]

    Definition Classes
    KeyedList
  18. def forceToReducers: Grouped[K, T]

  19. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  20. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  21. def hashCogroup[W, R](smaller: Grouped[K, W])(joiner: (K, T, Iterable[W]) ⇒ Iterator[R]): TypedPipe[(K, R)]

    WARNING This behaves semantically very differently than cogroup.

    WARNING This behaves semantically very differently than cogroup. this is because we handle (K,T) tuples on the left as we see them. the iterator on the right is over all elements with a matching key K, and it may be empty if there are no values for this key K. (because you haven't actually cogrouped, but only read the right hand side into a hashtable)

  22. def hashJoin[W](smaller: Grouped[K, W]): TypedPipe[(K, (T, W))]

  23. def hashLeftJoin[W](smaller: Grouped[K, W]): TypedPipe[(K, (T, Option[W]))]

  24. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  25. def join[W](smaller: Grouped[K, W]): KeyedList[K, (T, W)]

  26. def keys: TypedPipe[K]

    Definition Classes
    KeyedList
  27. def leftJoin[W](smaller: Grouped[K, W]): KeyedList[K, (T, Option[W])]

  28. def mapValueStream[V](nmf: (Iterator[T]) ⇒ Iterator[V]): Grouped[K, V]

    Operate on a Stream[T] of all the values for each key at one time.

    Operate on a Stream[T] of all the values for each key at one time. Avoid accumulating the whole list in memory if you can. Prefer reduce.

    Definition Classes
    GroupedKeyedList
  29. def mapValues[V](fn: (T) ⇒ V): Grouped[K, V]

    This is a special case of mapValueStream, but can be optimized because it doesn't need all the values for a given key at once.

    This is a special case of mapValueStream, but can be optimized because it doesn't need all the values for a given key at once. An unoptimized implementation is: mapValueStream { _.map { fn } } but for Grouped we can avoid resorting to mapValueStream

    Definition Classes
    GroupedKeyedList
  30. def max[B >: T](implicit cmp: Ordering[B]): TypedPipe[(K, T)]

    Definition Classes
    KeyedList
  31. def maxBy[B](fn: (T) ⇒ B)(implicit cmp: Ordering[B]): TypedPipe[(K, T)]

    Definition Classes
    KeyedList
  32. def min[B >: T](implicit cmp: Ordering[B]): TypedPipe[(K, T)]

    Definition Classes
    KeyedList
  33. def minBy[B](fn: (T) ⇒ B)(implicit cmp: Ordering[B]): TypedPipe[(K, T)]

    Definition Classes
    KeyedList
  34. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  35. final def notify(): Unit

    Definition Classes
    AnyRef
  36. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  37. def operate[T1](fn: (GroupBuilder) ⇒ GroupBuilder): TypedPipe[(K, T1)]

    Attributes
    protected
  38. val ordering: Ordering[K]

  39. def outerJoin[W](smaller: Grouped[K, W]): KeyedList[K, (Option[T], Option[W])]

  40. val pipe: Pipe

  41. def product[U >: T](implicit ring: Ring[U]): TypedPipe[(K, U)]

    Definition Classes
    KeyedList
  42. def reduce[U >: T](fn: (U, U) ⇒ U): TypedPipe[(K, U)]

    reduce with fn which must be associative and commutative.

    reduce with fn which must be associative and commutative. Like the above this can be optimized in some Grouped cases. If you don't have a commutative operator, use reduceLeft

    Definition Classes
    KeyedList
  43. def reduceLeft[U >: T](fn: (U, U) ⇒ U): TypedPipe[(K, U)]

    Definition Classes
    KeyedList
  44. val reducers: Int

  45. def reverse: Grouped[K, T]

  46. def rightJoin[W](smaller: Grouped[K, W]): KeyedList[K, (Option[T], W)]

  47. def scanLeft[B](z: B)(fn: (B, T) ⇒ B): KeyedList[K, B]

    Definition Classes
    KeyedList
  48. def size: TypedPipe[(K, Long)]

    Definition Classes
    KeyedList
  49. def sortBy[B](fn: (T) ⇒ B)(implicit ord: Ordering[B]): Grouped[K, T]

  50. def sortIfNeeded(gb: GroupBuilder): GroupBuilder

    Attributes
    protected
  51. def sortWith(lt: (T, T) ⇒ Boolean): Grouped[K, T]

  52. def sorted[B >: T](implicit ord: Ordering[B]): Grouped[K, T]

  53. def sum[U >: T](implicit sg: Semigroup[U]): TypedPipe[(K, U)]

    If there is no ordering, we default to assuming the Semigroup is commutative.

    If there is no ordering, we default to assuming the Semigroup is commutative. If you don't want that, define an ordering on the Values, or .forceToReducers.

    Semigroups MAY have a faster implementation of sum for iterators, so prefer using sum/sumLeft to reduce

    Definition Classes
    GroupedKeyedList
  54. def sumLeft[U >: T](implicit sg: Semigroup[U]): TypedPipe[(K, U)]

    Semigroups MAY have a faster implementation of sum for iterators, so prefer using sum/sumLeft to reduce

    Semigroups MAY have a faster implementation of sum for iterators, so prefer using sum/sumLeft to reduce

    Definition Classes
    KeyedList
  55. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  56. def take(n: Int): KeyedList[K, T]

    Selects first n elements.

    Selects first n elements.

    Definition Classes
    KeyedList
  57. def takeWhile(p: (T) ⇒ Boolean): KeyedList[K, T]

    Takes longest prefix of elements that satisfy the given predicate.

    Takes longest prefix of elements that satisfy the given predicate.

    Definition Classes
    KeyedList
  58. def toList: TypedPipe[(K, List[T])]

    Definition Classes
    KeyedList
  59. val toReducers: Boolean

  60. def toSet[U >: T]: TypedPipe[(K, Set[U])]

    Definition Classes
    KeyedList
  61. def toString(): String

    Definition Classes
    AnyRef → Any
  62. lazy val toTypedPipe: TypedPipe[(K, T)]

    Definition Classes
    GroupedKeyedList
  63. def values: TypedPipe[T]

    Definition Classes
    KeyedList
  64. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  65. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  66. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  67. def withReducers(red: Int): Grouped[K, T]

  68. def withSortOrdering[U >: T](so: Ordering[U]): Grouped[K, T]

Inherited from Serializable

Inherited from KeyedList[K, T]

Inherited from AnyRef

Inherited from Any

Ungrouped