The SinkBolt takes two related options: CacheSize and MaxWaitingFutures.
CacheSize sets the number of key-value pairs that the SinkBolt will accept
(and sum into an internal map) before committing out to the online store.
To perform this commit, the SinkBolt iterates through the map of aggregated
kv pairs and performs a "+" on the store for each pair, sequencing these
"+" calls together using the Future monad. If the store has high latency,
these calls might take a bit of time to complete.
MaxWaitingFutures(count) handles this problem by realizing a future
representing the "+" of kv-pair n only when kvpair n + 100 shows up in the bolt,
effectively pushing back against latency bumps in the host.
The allowed latency before a future is forced is equal to
(MaxWaitingFutures * execute latency).
Linear Supertypes
BaseBolt, IRichBolt, IComponent, IBolt, Serializable, AnyRef, Any
The SinkBolt takes two related options: CacheSize and MaxWaitingFutures. CacheSize sets the number of key-value pairs that the SinkBolt will accept (and sum into an internal map) before committing out to the online store.
To perform this commit, the SinkBolt iterates through the map of aggregated kv pairs and performs a "+" on the store for each pair, sequencing these "+" calls together using the Future monad. If the store has high latency, these calls might take a bit of time to complete.
MaxWaitingFutures(count) handles this problem by realizing a future representing the "+" of kv-pair n only when kvpair n + 100 shows up in the bolt, effectively pushing back against latency bumps in the host.
The allowed latency before a future is forced is equal to (MaxWaitingFutures * execute latency).