com.twitter.summingbird.scalding

BatchedScaldingStore

trait BatchedScaldingStore[K, V] extends ScaldingStore[K, V]

Self Type
BatchedScaldingStore[K, V]
Linear Supertypes
ScaldingStore[K, V], Serializable, AnyRef, Any
Known Subclasses
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. BatchedScaldingStore
  2. ScaldingStore
  3. Serializable
  4. AnyRef
  5. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def batcher: Batcher

    The batcher for this store

  2. implicit abstract def ordering: Ordering[K]

  3. abstract def readLast(exclusiveUB: BatchID, mode: Mode): Try[(BatchID, FlowProducer[TypedPipe[(K, V)]])]

    Get the most recent last batch and the ID (strictly less than the input ID) The "Last" is the stream with only the newest value for each key, within the batch combining the last from batchID and the deltas from batchID.

    Get the most recent last batch and the ID (strictly less than the input ID) The "Last" is the stream with only the newest value for each key, within the batch combining the last from batchID and the deltas from batchID.next you get the stream for batchID.next

  4. abstract def writeLast(batchID: BatchID, lastVals: TypedPipe[(K, V)])(implicit flowDef: FlowDef, mode: Mode): Unit

    Record a computed batch of code

Concrete Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  6. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  7. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws()
  8. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  9. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  10. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws()
  11. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  12. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  13. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  14. final def merge(delta: PipeFactory[(K, V)], sg: Semigroup[V], commutativity: Commutativity, reducers: Int): PipeFactory[(K, V)]

    instances of this trait MAY NOT change the logic here.

    instances of this trait MAY NOT change the logic here. This always follows the rule that we look for existing data (avoiding reading deltas in that case), then we fall back to the last checkpointed output by calling readLast. In that case, we compute the results by rolling forward

    Definition Classes
    BatchedScaldingStoreScaldingStore
  15. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  16. final def notify(): Unit

    Definition Classes
    AnyRef
  17. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  18. def select(b: List[BatchID]): List[BatchID]

    Override select if you don't want to materialize every batch.

    Override select if you don't want to materialize every batch. Note that select MUST return a list containing the final batch in the supplied list; otherwise data would be lost.

  19. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  20. def toString(): String

    Definition Classes
    AnyRef → Any
  21. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws()
  22. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws()
  23. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws()
  24. def withInitialBatch(firstNonZero: BatchID): BatchedScaldingStore[K, V]

    For (firstNonZero - 1) we read empty.

    For (firstNonZero - 1) we read empty. For all before we error on read. For all later, we proxy On write, we throw if batchID is less than firstNonZero

Inherited from ScaldingStore[K, V]

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped