Module org.elasticsearch.server

Package org.elasticsearch.action.bulk

Class BulkProcessor2

java.lang.Object

org.elasticsearch.action.bulk.BulkProcessor2

All Implemented Interfaces:

Closeable, AutoCloseable

public class BulkProcessor2
extends Object
implements Closeable

A bulk processor is a thread safe bulk processing class, allowing to easily set when to "flush" a new bulk request (either based on number of actions, based on the size, or time), and to easily control the number of concurrent bulk requests allowed to be executed in parallel.

In order to create a new bulk processor, use the BulkProcessor2.Builder.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	BulkProcessor2.Builder	A builder used to create a build an instance of a bulk processor.
static interface	BulkProcessor2.Listener	A listener for the execution.

Method Summary

Modifier and Type	Method	Description
BulkProcessor2	add(DeleteRequest request)	Adds an DeleteRequest to the list of actions to execute.
BulkProcessor2	add(IndexRequest request)	Adds an IndexRequest to the list of actions to execute.
BulkProcessor2	addWithBackpressure(IndexRequest request, Supplier<Boolean> shouldAbort)	This method is similar to add(), except that if adding the approximate size in bytes of the request to totalBytesInFlight would exceed maxBytesInFlight then this method will block until the request can be added without exceeding maxBytesInFlight (or until shouldAbort returns false).
boolean	awaitClose(long timeout, TimeUnit unit)	Closes the processor.
static BulkProcessor2.Builder	builder(BiConsumer<BulkRequest,ActionListener<BulkResponse>> consumer, BulkProcessor2.Listener listener, ThreadPool threadPool)
void	close()	This method flushes any requests, prevents any future retries, closes this processor, and returns once any pending requests complete.
protected void	ensureOpen()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

builder

public static BulkProcessor2.Builder builder(BiConsumer<BulkRequest,ActionListener<BulkResponse>> consumer,
 BulkProcessor2.Listener listener,
 ThreadPool threadPool)

Parameters:

consumer - The consumer that is called to fulfil bulk operations. This consumer _must_ operate either very fast or asynchronously.

listener - The BulkProcessor2 listener that gets called on bulk events

threadPool - The threadpool used to schedule the flush task for this bulk processor, if flushInterval is not null.

Returns:

the builder for BulkProcessor2

awaitClose

public boolean awaitClose(long timeout,
 TimeUnit unit)
                   throws InterruptedException

Closes the processor. Any remaining bulk actions are flushed if they can be flushed in the given time.

Waits for up to the specified timeout for all bulk requests to complete then returns

Parameters:

timeout - The maximum time to wait for the bulk requests to complete

unit - The time unit of the timeout argument

Returns:

True if the bulk processor was able to be closed in the given time, false otherwise

Throws:

InterruptedException - If the current thread is interrupted

close

public void close()

This method flushes any requests, prevents any future retries, closes this processor, and returns once any pending requests complete.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

add

public BulkProcessor2 add(IndexRequest request)
                   throws EsRejectedExecutionException

Adds an IndexRequest to the list of actions to execute. Follows the same behavior of IndexRequest (for example, if no id is provided, one will be generated, or usage of the create flag).

Throws:

EsRejectedExecutionException - if adding the approximate size in bytes of the request to totalBytesInFlight would exceed maxBytesInFlight

addWithBackpressure

public BulkProcessor2 addWithBackpressure(IndexRequest request,
 Supplier<Boolean> shouldAbort)
                                   throws EsRejectedExecutionException

This method is similar to add(), except that if adding the approximate size in bytes of the request to totalBytesInFlight would exceed maxBytesInFlight then this method will block until the request can be added without exceeding maxBytesInFlight (or until shouldAbort returns false). This method should be used carefully, ideally from a single thread. This is because calling it with multiple threads would either hang up all the threads in the pool (in the case of a bounded thread pool) or effectively create an unbounded queue (in the case of an unbounded thread pool).

Parameters:

request - The request to add to a batch to be consumed

shouldAbort - If this returns true then this method bails out with an EsRejectedExecutionException

Returns:

this BulkProcessor2

Throws:

EsRejectedExecutionException - if shouldAbort returns true before the request has been added to a batch

add

public BulkProcessor2 add(DeleteRequest request)
                   throws EsRejectedExecutionException

Adds an DeleteRequest to the list of actions to execute.

Throws:

EsRejectedExecutionException - if adding the approximate size in bytes of the request to totalBytesInFlight would exceed maxBytesInFlight

ensureOpen

protected void ensureOpen()