summingbird

Type Members

1. case class AlsoProducer[P <: Platform[P], +T, +R](ensure: TailProducer[P, T], result: Producer[P, R]) extends Producer[P, R] with Product with Serializable

   

   This is a special node that ensures that the first argument is planned, but produces values equivalent to the result.

2. class AlsoTailProducer[P <: Platform[P], +T, +R] extends AlsoProducer[P, T, R] with TailProducer[P, R]

   

3. case class Counter(group: Group, name: Name)(implicit jobID: JobId) extends Product with Serializable

   

4. trait CounterIncrementor extends AnyRef

   

5. case class Dependants[P <: Platform[P]](tail: Producer[P, Any]) extends DependantGraph[Producer[P, Any]] with Product with Serializable

   

   Producers are Directed Acyclic Graphs by the fact that they are immutable.

6. trait ExecutionConfig[P <: Platform[P]] extends AnyRef

   

7. case class FlatMappedProducer[P <: Platform[P], T, +U](producer: Producer[P, T], fn: (T) ⇒ TraversableOnce[U]) extends Producer[P, U] with Product with Serializable

   

8. final case class Group(getString: String) extends AnyVal with Product with Serializable

   

9. case class IdentityKeyedProducer[P <: Platform[P], K, V](producer: Producer[P, (K, V)]) extends KeyedProducer[P, K, V] with Product with Serializable

   

10. case class KeyFlatMappedProducer[P <: Platform[P], K, V, K2](producer: Producer[P, (K, V)], fn: (K) ⇒ TraversableOnce[K2]) extends KeyedProducer[P, K2, V] with Product with Serializable

    

11. sealed trait KeyedProducer[P <: Platform[P], K, V] extends Producer[P, (K, V)]

    

    This has the methods on Key-Value streams.

    This has the methods on Key-Value streams. The rule is: if you can easily express your logic on the keys and values independently, do it! This is how you communicate structure to Summingbird and it uses these hints to attempt the most efficient run of your code.

12. case class LeftJoinedProducer[P <: Platform[P], K, V, JoinedV](left: Producer[P, (K, V)], joined: P.Service[K, JoinedV]) extends KeyedProducer[P, K, (V, Option[JoinedV])] with Product with Serializable

    

13. case class MergedProducer[P <: Platform[P], +T](left: Producer[P, T], right: Producer[P, T]) extends Producer[P, T] with Product with Serializable

    

14. trait MutableStringConfig extends AnyRef

    

15. final case class Name(getString: String) extends AnyVal with Product with Serializable

    

16. case class NamedProducer[P <: Platform[P], +T](producer: Producer[P, T], id: String) extends Producer[P, T] with Product with Serializable

    

17. case class OptionMappedProducer[P <: Platform[P], T, +U](producer: Producer[P, T], fn: (T) ⇒ Option[U]) extends Producer[P, U] with Product with Serializable

    

    Represents filters and maps which may be optimized differently Note that "option-mapping" is closed under composition and hence useful to call out

18. class OptionalPlatform2[P1 <: Platform[P1], P2 <: Platform[P2]] extends Platform[OptionalPlatform2[P1, P2]]

    

    Platform capable of planning and executing at most 2 underlying platforms in parallel.

19. case class OptionalUnzip2[P1 <: Platform[P1], P2 <: Platform[P2]]() extends Product with Serializable

    

20. class Options extends AnyRef

    

21. trait Platform[P <: Platform[P]] extends AnyRef

    

    Base trait for summingbird compilers.

22. class Platform2[P1 <: Platform[P1], P2 <: Platform[P2]] extends Platform[Platform2[P1, P2]]

    

    Platform capable of planning and executing two underlying platforms in parallel.

23. trait PlatformStatProvider extends AnyRef

    

24. sealed trait Producer[P <: Platform[P], +T] extends AnyRef

    

    A Producer is a node in our tree, able to generate new items and have operations applied to it.

    A Producer is a node in our tree, able to generate new items and have operations applied to it. In Storm, this might be an in-progress TopologyBuilder.

25. trait ReadableMap extends AnyRef

    

26. case class Source[P <: Platform[P], T](source: P.Source[T]) extends Producer[P, T] with Product with Serializable

    

    Wraps the sources of the given Platform

27. case class Summer[P <: Platform[P], K, V](producer: Producer[P, (K, V)], store: P.Store[K, V], semigroup: Semigroup[V]) extends KeyedProducer[P, K, (Option[V], V)] with TailProducer[P, (K, (Option[V], V))] with Product with Serializable

    

28. trait SummingbirdConfig extends AnyRef

    

29. class TPNamedProducer[P <: Platform[P], +T] extends NamedProducer[P, T] with TailProducer[P, T]

    

30. sealed trait TailProducer[P <: Platform[P], +T] extends Producer[P, T]

    

    Only TailProducers can be planned.

    Only TailProducers can be planned. There is nothing after a tail

31. trait TimeExtractor[T] extends Serializable

    

    This cannot be a subclass of function and use the pattern of implicit dependencies, since then you get an implicit function.

    This cannot be a subclass of function and use the pattern of implicit dependencies, since then you get an implicit function. Not good

32. case class Unzip2[P1 <: Platform[P1], P2 <: Platform[P2]]() extends Product with Serializable

    

    I tried to make Unzip2 an object with an apply method that took all three type parameters, but ran into issues:

    I tried to make Unzip2 an object with an apply method that took all three type parameters, but ran into issues:

    [error] /Users/sritchie/code/twitter/summingbird/summingbird-core/src/main/scala/com/twitter/summingbird/PairedPlatform.scala:38: constructor of type com.twitter.summingbird.IdentityKeyedProducer[P,K,V] cannot be uniquely instantiated to expected type com.twitter.summingbird.Producer[com.twitter.summingbird.Platform2[P1,P2],T] [error] --- because --- [error] undetermined type

33. case class ValueFlatMappedProducer[P <: Platform[P], K, V, V2](producer: Producer[P, (K, V)], fn: (V) ⇒ TraversableOnce[V2]) extends KeyedProducer[P, K, V2] with Product with Serializable

    

34. case class WrappingConfig(backingConfig: ReadableMap, updates: Map[String, AnyRef], removes: Set[String]) extends SummingbirdConfig with Product with Serializable

    

35. case class WrittenProducer[P <: Platform[P], T, U >: T](producer: Producer[P, T], sink: P.Sink[U]) extends TailProducer[P, T] with Product with Serializable