com.twitter.summingbird.scalding
The batcher for this store
The batcher for this store
instances of this trait MAY NOT change the logic here.
instances of this trait MAY NOT change the logic here. This always follows the rule that we look for existing data (avoiding reading deltas in that case), then we fall back to the last checkpointed output by calling readLast. In that case, we compute the results by rolling forward
Get the most recent last batch and the ID (strictly less than the input ID) The "Last" is the stream with only the newest value for each key, within the batch combining the last from batchID and the deltas from batchID.
Get the most recent last batch and the ID (strictly less than the input ID) The "Last" is the stream with only the newest value for each key, within the batch combining the last from batchID and the deltas from batchID.next you get the stream for batchID.next
Override select if you don't want to materialize every batch.
Override select if you don't want to materialize every batch. Note that select MUST return a list containing the final batch in the supplied list; otherwise data would be lost.
For (firstNonZero - 1) we read empty.
For (firstNonZero - 1) we read empty. For all before we error on read. For all later, we proxy On write, we throw if batchID is less than firstNonZero
Record a computed batch of code
Record a computed batch of code
For (firstNonZero - 1) we read empty. For all before we error on read. For all later, we proxy On write, we throw if batchID is less than firstNonZero