Represents something than can be CoGrouped with another CoGroupable
Closures are difficult for serialization.
This encodes the rules that 1) sorting is only possible before doing any reduce, 2) reversing is only possible after sorting.
Used for objects that may have a description set to be used in .
used for types that may know how many reducers they need e.
If we can HashJoin, then we can CoGroup, but not vice-versa i.
Only intended to be use to implement the hashCogroup on TypedPipe/Grouped
Creates a TypedPipe from an Iterable[T].
This is for the case where you don't want to expose any structure but the ability to operate on an iterator of the values
Represents sharded lists of items of type T There are exactly two fundamental operations: toTypedPipe: marks the end of the grouped-on-key operations.
Represents anything that starts as a TypedPipe of Key Value, where the value type has been erased.
This class is for the syntax enrichment enabling .
used for types that must know how many reducers they need e.
This type is used to implement .
Trait to assist with creating partitioned sources.
Trait to assist with creating objects such as PartitionedTsv to read from separated files.
Scalding source to read or write partitioned delimited text.
Scalding source to read or write partitioned text.
This is a class that models the logical portion of the reduce step.
This class is generally only created by users with the TypedPipe.
All sorting methods defined here trigger Hadoop secondary sort on key + value.
After sorting, we are no longer CoGroupable, and we can only call reverse in the initial SortedGrouped created from the Sortable: .
Creates a partition using the given template string.
Think of a TypedPipe as a distributed unordered list that may or may not yet have been materialized in memory or disk.
This is a TypedPipe that delays having access to the FlowDef and Mode until toPipe is called
This is an instance of a TypedPipe that wraps a cascading Pipe
Opposite of TypedSource, used for writing into
This is the state after we have done some reducing.
ValuePipe is special case of a TypedPipe of just a optional single element.
Used for objects that may _set_ a description to be used in .
used for objects that may _set_ how many reducers they need e.
Extension for TypedPipe to add a cumulativeSum method.
This object is the EmptyTypedPipe.
Autogenerated methods for flattening the nested value tuples that result after joining many pipes together.
lookupJoin simulates the behavior of a realtime system attempting to leftJoin (K, V) pairs against some other value type (JoinedV) by performing realtime lookups on a key-value Store.
This is an autogenerated object which gives you easy access to doing N-way joins so the types are cleaner.
Utility functions to assist with creating partitioned sourced.
Partitioned typed commma separated source.
Partitioned typed \1
separated source (commonly used by Pig).
Partitioned typed pipe separated source.
Partitioned typed tab separated source.
These are named syntax extensions that users can optionally import.
implicits for the type-safe DSL import TDsl.
factory methods for TypedPipe, which is the typed representation of distributed lists in scalding.
Some methods for comparing two typed pipes and finding out the difference between them.
This is an implementation detail (and should be marked private)