contrib
Accumulates elements of type Element while a property extracted with propertyExtractor remains unchanged, emits an accumulated sequence when the property changes, maxElements is reached or maxDuration has passed.
Flow stage for universal delay management, allows to manage delay through DelayStrategy.
Flow stage for universal delay management, allows to manage delay through DelayStrategy.
It determines delay for each ongoing element invoking DelayStrategy.nextDelay(elem: T): FiniteDuration.
Implementing DelayStrategy with your own gives you flexible ability to manage delay value depending on coming elements.
It is important notice that DelayStrategy can be stateful.
There are also predefined strategies, see DelayStrategy companion object's methods.
DelayStrategy
Sends elements from buffer if upstream does not emit for a configured amount of time.
Sends elements from buffer if upstream does not emit for a configured amount of time. In other words, this stage attempts to maintains a base rate of emitted elements towards the downstream using elements from upstream.
If upstream emits new elements until the accumulated elements in the buffer exceed the specified minimum size used as the keep alive elements, then the base rate is no longer maintained until we reach another period without elements form upstream.
The keep alive period is the keep alive failover size times the interval.
Emits when upstream emits an element or if the upstream was idle for the configured period
Backpressures when downstream backpressures
Completes when upstream completes
Cancels when downstream cancels
akka.stream.scaladsl.FlowOps#expand
akka.stream.scaladsl.FlowOps#keepAlive
This stage materializes to the last element pushed before upstream completion, if any, thereby recovering from any failure.
This stage materializes to the last element pushed before upstream completion, if any, thereby recovering from any failure. Pushed elements are just passed along.
input and output type
This stage partitions input to 2 different outlets, applying different transformations on the elements, according to the received partition function.
This stage partitions input to 2 different outlets, applying different transformations on the elements, according to the received partition function.
input type
left output type
right output type
Signals demand only once every interval (pulse) and then back-pressures.
Signals demand only once every interval (pulse) and then back-pressures. Requested element is emitted downstream if there is demand.
It can be used to implement simple time-window processing where data is aggregated for predefined amount of time and the computed aggregate is emitted once per this time. See TimeWindow
type of element
supports sampling on stream
supports sampling on stream
a lambda returns next sample position
Provides operations needed to implement the timedIntervalBetween DSL
Provides operations needed to implement the timed DSL
Graph stage for token-based throttling.
Graph stage for token-based throttling. Use it directly for constructing more complex graphs.
element type
Materializes into a Future of ValveSwitch which provides a the method flip that stops or restarts the flow of elements passing through the stage.
Materializes into a Future of ValveSwitch which provides a the method flip that stops or restarts the flow of elements passing through the stage. As long as the valve is closed it will backpressure.
Note that closing the valve could result in one element being buffered inside the stage, and if the stream completes or fails while being closed, that element may be lost.
Pause/ Resume a Flow
A stage that works as a Source of data chunks extracted from zip files.
A stage that works as a Source of data chunks extracted from zip files. In addition to regular files, the zip file might contain directories and other zip files. Every chunk is a tuple of ZipEntryData and ByteString, where the former carries basic info about the file from which the bytes come and the latter carries those bytes. This stage materializes to the total amount of read bytes.
This stage emits folded values like scan, but the first element emitted is not the zero value but the result of
applying the given function to the given zero value and the first pushed element.
This stage emits folded values like scan, but the first element emitted is not the zero value but the result of
applying the given function to the given zero value and the first pushed element.
input type
output type
(Since version 0.5) Use scan and drop(1) instead
This companion defines a factory for Accumulate instances, see Accumulate.apply.
Additional akka.stream.scaladsl.Flow and akka.stream.scaladsl.Flow operators.
This companion defines a factory for LastElement instances, see LastElement.apply.
Defines a factory for PagedSource.
This companion defines a factory for PartitionWith instances, see PartitionWith.apply.
Given Flow[In, Out] and f: (In, Out) => Result,
transforms original flow into a Flow[In, Result], i.e.
Given Flow[In, Out] and f: (In, Out) => Result,
transforms original flow into a Flow[In, Result], i.e.
for every element e signalled from upstream, emits f(e, flow(e)).
Has an overloaded factory that fixes f to be a tuple constructor, i.e.
for every element e signalled from upstream, emits a tuple (e, flow(e)).
IMPORTANT! This flow combinator is guaranteed to work correctly on flows that have behavior of classic total functions, meaning that they should be a one-t-one functions that don't reorder, drop, inject etc new elements. In the future these restrictions may be lifted, for now please refer to the following resources for more: - https://github.com/akka/akka-stream-contrib/pull/142#discussion_r228875614 - https://github.com/akka/akka/issues/15957 and linked issues - https://discuss.lightbend.com/t/passing-stream-elements-into-and-over-flow/2536 and linked resources
Applies no internal buffering / flow control etc.
Emits when upstream emits an element
Backpressures when downstream backpressures
Completes when upstream completes
Cancels when downstream cancels
Examples:
1. Consuming from Kafka: Allows for busines logic to stay unaware of commits:
val logic: Flow[CommittableMessage[String, Array[Byte]], ProcessingResult] = Flow[CommittableMessage[String, Array[Byte]]] .map(m => process(m.record)) // Used like this: Consumer .committableSource(settings, Subscriptions.topics("my-topic")) .via(PassThroughFlow(logic)) .map { case (committableMessage, processingResult) => // decide to commit or not based on `processingResult` } // or: Consumer .committableSource(settings, Subscriptions.topics("my-topic")) .via(PassThroughFlow(logic, Keep.left)) // process messages but return original elements .mapAsync(1)(_.commitScalaDsl())
2. Logging HTTP request-response based on some rule
// assuming Akka HTTP entities: val route: Route = ??? // Route has an implicit conversion to Flow[HttpRequest, HttpResponse] Flow[HttpRequest] .map { r => // don't log web crawlers' requests if (userAgent(r) != "google-bot") { logRequest(r) } r } .via(PassThroughFlow(route)) // req => (req, resp) .map { case (req, resp) => // don't log responses to web crawlers if (userAgent(req) != "google-bot") { logResponse(resp) } resp }
This object defines methods for retry operations.
Source factory methods are placed here
Create a Source that will output elements of type A
given a "producer" function
Create a Source that will output elements of type A
given a "producer" function
Examples:
stream of current times:
SourceRepeatEval(() => System.currentTimeMillis)stream of random numbers:
SourceRepeatEval(() => Random.nextInt)Behavior is the same as in
Source.repeat(()).map(_ => x)Supports cancellation via materialized Cancellable.
Throttles a flow based on tokens provided consumed from a provided token source.
Throttles a flow based on tokens provided consumed from a provided token source. The flow emits elements only if the needed amount of tokens for the next element is available. The amount of tokens needed is specified by a cost function.
Tokens are consumed as-needed, and pre-fetched once the internal token buffer is empty. Also, an initial fetch is performed to initially fill the internal token bucket.
Emits when an input element e is available and costCalculation(e) tokens are available
Backpressures when downstream backpressures, or when not enough tokens are available for the next element. For the token input: when no tokens are needed (tokens are pulled as needed)
Completes when upstream completes, and also when token source completes and all internally stored tokens were consumed (best effort; also completes when the next element cost is higher than the available tokens)
Cancels when downstream cancels
This companion defines a factory for ZipInputStreamSource instances, see ZipInputStreamSource.apply.
Accumulates elements of type Element while a property extracted with propertyExtractor remains unchanged, emits an accumulated sequence when the property changes, maxElements is reached or maxDuration has passed.
type of accumulated elements
type of the observed property