The BulkIndexerActor provides a non-bulk (message-by-message) abstraction on top of the ElasticSearch Bulk API.
ScanAndScrollSource wraps Elasticsearch's Scroll API as a akka-streams source.
ScanAndScrollSource wraps Elasticsearch's Scroll API as a akka-streams source. By creating and subscribing to this source, you will get a stream of every message in Elasticsearch matching your query. Internally, the messages are batched. The batch size is configurable by setting a size parameter on the query. Externally, results are streamed message-by-message.
The Source will only continue getting more results from Elasticsearch will downstream is consuming. In the future, we may need two enhancements: - Keep alive messages to keep the source alive >1m - Buffering of more messages
The BulkIndexerActor provides a non-bulk (message-by-message) abstraction on top of the ElasticSearch Bulk API.
Every message sent will either return
IndexingComplete
orIndexingFailure
Note: The BulkIndexerActor does not implement it's own backpressure. The client must account for the number of messages in flight to control memory usage.