org.apache.camel.processor.aggregate

Class AggregateProcessor

java.lang.Object

org.apache.camel.support.ServiceSupport

org.apache.camel.processor.aggregate.AggregateProcessor

All Implemented Interfaces:

AsyncProcessor, Navigate<Processor>, Processor, Service, ShutdownableService, HasId, IdAware, ShutdownAware, ShutdownPrepared, StatefulService, SuspendableService, Traceable

public class AggregateProcessor
extends ServiceSupport
implements AsyncProcessor, Navigate<Processor>, Traceable, ShutdownPrepared, ShutdownAware, IdAware

An implementation of the Aggregator pattern where a batch of messages are processed (up to a maximum amount or until some timeout is reached) and messages for the same correlation key are combined together using some kind of AggregationStrategy (by default the latest message is used) to compress many message exchanges into a smaller number of exchanges.

A good example of this is stock market data; you may be receiving 30,000 messages/second and you may want to throttle it right down so that multiple messages for the same stock are combined (or just the latest message is used and older prices are discarded). Another idea is to combine line item messages together into a single invoice message.

Field Summary

Fields

Modifier and Type	Field and Description
static String	AGGREGATE_TIMEOUT_CHECKER

Fields inherited from class org.apache.camel.support.ServiceSupport

shutdown, shuttingdown, started, starting, stopped, stopping, suspended, suspending

Constructor Summary

Constructors

Constructor and Description
AggregateProcessor(CamelContext camelContext, Processor processor, Expression correlationExpression, AggregationStrategy aggregationStrategy, ExecutorService executorService, boolean shutdownExecutorService)

Method Summary

Modifier and Type	Method and Description
void	clearClosedCorrelationKeysCache() Clear all the closed correlation keys stored in the cache
boolean	deferShutdown(ShutdownRunningTask shutdownRunningTask) To defer shutdown during first phase of shutdown.
protected void	doAggregationComplete(String complete, List<Exchange> list, String key, Exchange originalExchange, Exchange answer)
protected void	doAggregationRepositoryAdd(CamelContext camelContext, String key, Exchange oldExchange, Exchange newExchange)
protected void	doProcess(Exchange exchange)
protected void	doShutdown() Implementations override this method to perform customized shutdown.
protected void	doStart() Implementations override this method to support customized start/stop.
protected void	doStop() Implementations override this method to support customized start/stop.
int	forceCompletionOfAllGroups()
int	forceCompletionOfGroup(String key)
AggregateController	getAggregateController()
AggregationRepository	getAggregationRepository()
AggregationStrategy	getAggregationStrategy()
Integer	getCloseCorrelationKeyOnCompletion()
int	getClosedCorrelationKeysCacheSize() Current number of closed correlation keys in the memory cache
long	getCompletionInterval()
Predicate	getCompletionPredicate()
int	getCompletionSize()
Expression	getCompletionSizeExpression()
long	getCompletionTimeout()
long	getCompletionTimeoutCheckerInterval()
Expression	getCompletionTimeoutExpression()
Expression	getCorrelationExpression()
ExceptionHandler	getExceptionHandler()
String	getId() Returns the id
int	getInProgressCompleteExchanges()
OptimisticLockRetryPolicy	getOptimisticLockRetryPolicy()
int	getPendingExchangesSize() Gets the number of pending exchanges.
AggregateProcessorStatistics	getStatistics()
ScheduledExecutorService	getTimeoutCheckerExecutorService()
String	getTraceLabel() Gets the trace label used for logging when tracing is enabled.
boolean	hasNext() Are there more outputs? Important only invoke this once, as this method do not carry state, and is not intended to be used in a while loop, but used by a if statement instead.
boolean	isCompleteAllOnStop()
protected String	isCompleted(String key, Exchange exchange) Tests whether the given exchange is complete or not
boolean	isCompletionFromBatchConsumer()
boolean	isCompletionOnNewCorrelationGroup()
boolean	isDiscardOnCompletionTimeout()
boolean	isEagerCheckCompletion()
boolean	isIgnoreInvalidCorrelationKeys()
boolean	isOptimisticLocking()
boolean	isParallelProcessing()
protected String	isPreCompleted(String key, Exchange oldExchange, Exchange newExchange) Tests whether the given exchanges is pre-complete or not
boolean	isShutdownTimeoutCheckerExecutorService()
List<Processor>	next() Next group of outputs Important only invoke this once, as this method do not carry state, and is not intended to be used in a while loop, but used by a if statement instead.
protected Exchange	onAggregation(Exchange oldExchange, Exchange newExchange)
protected Exchange	onCompletion(String key, Exchange original, Exchange aggregated, boolean fromTimeout)
protected void	onOptimisticLockingFailure(Exchange oldExchange, Exchange newExchange)
protected boolean	onPreCompletionAggregation(Exchange oldExchange, Exchange newExchange)
void	prepareShutdown(boolean suspendOnly, boolean forced) Prepares for stop/shutdown.
void	process(Exchange exchange) Processes the message exchange
boolean	process(Exchange exchange, AsyncCallback callback) Processes the message exchange.
protected void	restoreTimeoutMapFromAggregationRepository() Restores the timeout map with timeout values from the aggregation repository.
void	setAggregateController(AggregateController aggregateController)
void	setAggregationRepository(AggregationRepository aggregationRepository)
void	setAggregationStrategy(AggregationStrategy aggregationStrategy)
void	setCloseCorrelationKeyOnCompletion(Integer closeCorrelationKeyOnCompletion)
void	setCompleteAllOnStop(boolean completeAllOnStop)
void	setCompletionFromBatchConsumer(boolean completionFromBatchConsumer)
void	setCompletionInterval(long completionInterval)
void	setCompletionOnNewCorrelationGroup(boolean completionOnNewCorrelationGroup)
void	setCompletionPredicate(Predicate completionPredicate)
void	setCompletionSize(int completionSize)
void	setCompletionSizeExpression(Expression completionSizeExpression)
void	setCompletionTimeout(long completionTimeout)
void	setCompletionTimeoutCheckerInterval(long completionTimeoutCheckerInterval)
void	setCompletionTimeoutExpression(Expression completionTimeoutExpression)
void	setCorrelationExpression(Expression correlationExpression)
void	setDiscardOnCompletionTimeout(boolean discardOnCompletionTimeout)
void	setEagerCheckCompletion(boolean eagerCheckCompletion)
void	setExceptionHandler(ExceptionHandler exceptionHandler)
void	setForceCompletionOnStop(boolean forceCompletionOnStop)
void	setId(String id) Sets the id
void	setIgnoreInvalidCorrelationKeys(boolean ignoreInvalidCorrelationKeys)
void	setOptimisticLocking(boolean optimisticLocking)
void	setOptimisticLockRetryPolicy(OptimisticLockRetryPolicy optimisticLockRetryPolicy)
void	setParallelProcessing(boolean parallelProcessing)
void	setShutdownTimeoutCheckerExecutorService(boolean shutdownTimeoutCheckerExecutorService)
void	setTimeoutCheckerExecutorService(ScheduledExecutorService timeoutCheckerExecutorService)
String	toString()
protected void	trackTimeout(String key, Exchange exchange)

Methods inherited from class org.apache.camel.support.ServiceSupport

doResume, doSuspend, getStatus, getVersion, isRunAllowed, isStarted, isStarting, isStopped, isStopping, isStoppingOrStopped, isSuspended, isSuspending, isSuspendingOrSuspended, resume, shutdown, start, stop, suspend

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

AGGREGATE_TIMEOUT_CHECKER

public static final String AGGREGATE_TIMEOUT_CHECKER

See Also:

Constant Field Values

Constructor Detail

AggregateProcessor

public AggregateProcessor(CamelContext camelContext,
                          Processor processor,
                          Expression correlationExpression,
                          AggregationStrategy aggregationStrategy,
                          ExecutorService executorService,
                          boolean shutdownExecutorService)

Method Detail

toString

public String toString()

Overrides:

toString in class Object

getTraceLabel

public String getTraceLabel()

Description copied from interface: Traceable

Gets the trace label used for logging when tracing is enabled.

The label should be short and precise.

Specified by:

getTraceLabel in interface Traceable

Returns:

the label

next

public List<Processor> next()

Description copied from interface: Navigate

Next group of outputs

Important only invoke this once, as this method do not carry state, and is not intended to be used in a while loop, but used by a if statement instead.

Specified by:

next in interface Navigate<Processor>

Returns:

next group or null if no more outputs

hasNext

public boolean hasNext()

Description copied from interface: Navigate

Are there more outputs?

Important only invoke this once, as this method do not carry state, and is not intended to be used in a while loop, but used by a if statement instead.

Specified by:

hasNext in interface Navigate<Processor>

Returns:

true if more outputs

getId

public String getId()

Description copied from interface: HasId

Returns the id

Specified by:

getId in interface HasId

Returns:

the id

setId

public void setId(String id)

Description copied from interface: IdAware

Sets the id

Specified by:

setId in interface IdAware

Parameters:

id - the id

process

public void process(Exchange exchange)
             throws Exception

Description copied from interface: Processor

Processes the message exchange

Specified by:

process in interface Processor

Parameters:

exchange - the message exchange

Throws:

Exception - if an internal processing error has occurred.

process

public boolean process(Exchange exchange,
                       AsyncCallback callback)

Description copied from interface: AsyncProcessor

Processes the message exchange. Similar to Processor.process(org.apache.camel.Exchange), but the caller supports having the exchange asynchronously processed.

If there was a failure processing then the caused Exception would be set on the Exchange.

Specified by:

process in interface AsyncProcessor

Parameters:

exchange - the message exchange

callback - the AsyncCallback will be invoked when the processing of the exchange is completed. If the exchange is completed synchronously, then the callback is also invoked synchronously. The callback should therefore be careful of starting recursive loop.

Returns:

(doneSync) true to continue execute synchronously, false to continue being executed asynchronously

doProcess

protected void doProcess(Exchange exchange)
                  throws Exception

Throws:

Exception

doAggregationComplete

protected void doAggregationComplete(String complete,
                                     List<Exchange> list,
                                     String key,
                                     Exchange originalExchange,
                                     Exchange answer)

doAggregationRepositoryAdd

protected void doAggregationRepositoryAdd(CamelContext camelContext,
                                          String key,
                                          Exchange oldExchange,
                                          Exchange newExchange)

onOptimisticLockingFailure

protected void onOptimisticLockingFailure(Exchange oldExchange,
                                          Exchange newExchange)

isPreCompleted

protected String isPreCompleted(String key,
                                Exchange oldExchange,
                                Exchange newExchange)

Tests whether the given exchanges is pre-complete or not

Parameters:

key - the correlation key

oldExchange - the existing exchange

newExchange - the incoming exchange

Returns:

null if not pre-completed, otherwise a String with the type that triggered the pre-completion

isCompleted

protected String isCompleted(String key,
                             Exchange exchange)

Tests whether the given exchange is complete or not

Parameters:

key - the correlation key

exchange - the incoming exchange

Returns:

null if not completed, otherwise a String with the type that triggered the completion

trackTimeout

protected void trackTimeout(String key,
                            Exchange exchange)

onAggregation

protected Exchange onAggregation(Exchange oldExchange,
                                 Exchange newExchange)

onPreCompletionAggregation

protected boolean onPreCompletionAggregation(Exchange oldExchange,
                                             Exchange newExchange)

onCompletion

protected Exchange onCompletion(String key,
                                Exchange original,
                                Exchange aggregated,
                                boolean fromTimeout)

restoreTimeoutMapFromAggregationRepository

protected void restoreTimeoutMapFromAggregationRepository()
                                                   throws Exception

Restores the timeout map with timeout values from the aggregation repository.

This is needed in case the aggregator has been stopped and started again (for example a server restart). Then the existing exchanges from the AggregationRepository must have their timeout conditions restored.

Throws:

Exception

getClosedCorrelationKeysCacheSize

public int getClosedCorrelationKeysCacheSize()

Current number of closed correlation keys in the memory cache

clearClosedCorrelationKeysCache

public void clearClosedCorrelationKeysCache()

Clear all the closed correlation keys stored in the cache

getStatistics

public AggregateProcessorStatistics getStatistics()

getInProgressCompleteExchanges

public int getInProgressCompleteExchanges()

getCompletionPredicate

public Predicate getCompletionPredicate()

setCompletionPredicate

public void setCompletionPredicate(Predicate completionPredicate)

isEagerCheckCompletion

public boolean isEagerCheckCompletion()

setEagerCheckCompletion

public void setEagerCheckCompletion(boolean eagerCheckCompletion)

getCompletionTimeout

public long getCompletionTimeout()

setCompletionTimeout

public void setCompletionTimeout(long completionTimeout)

getCompletionTimeoutExpression

public Expression getCompletionTimeoutExpression()

setCompletionTimeoutExpression

public void setCompletionTimeoutExpression(Expression completionTimeoutExpression)

getCompletionInterval

public long getCompletionInterval()

setCompletionInterval

public void setCompletionInterval(long completionInterval)

getCompletionSize

public int getCompletionSize()

setCompletionSize

public void setCompletionSize(int completionSize)

getCompletionSizeExpression

public Expression getCompletionSizeExpression()

setCompletionSizeExpression

public void setCompletionSizeExpression(Expression completionSizeExpression)

isIgnoreInvalidCorrelationKeys

public boolean isIgnoreInvalidCorrelationKeys()

setIgnoreInvalidCorrelationKeys

public void setIgnoreInvalidCorrelationKeys(boolean ignoreInvalidCorrelationKeys)

getCloseCorrelationKeyOnCompletion

public Integer getCloseCorrelationKeyOnCompletion()

setCloseCorrelationKeyOnCompletion

public void setCloseCorrelationKeyOnCompletion(Integer closeCorrelationKeyOnCompletion)

isCompletionFromBatchConsumer

public boolean isCompletionFromBatchConsumer()

setCompletionFromBatchConsumer

public void setCompletionFromBatchConsumer(boolean completionFromBatchConsumer)

isCompletionOnNewCorrelationGroup

public boolean isCompletionOnNewCorrelationGroup()

setCompletionOnNewCorrelationGroup

public void setCompletionOnNewCorrelationGroup(boolean completionOnNewCorrelationGroup)

isCompleteAllOnStop

public boolean isCompleteAllOnStop()

getCompletionTimeoutCheckerInterval

public long getCompletionTimeoutCheckerInterval()

setCompletionTimeoutCheckerInterval

public void setCompletionTimeoutCheckerInterval(long completionTimeoutCheckerInterval)

getExceptionHandler

public ExceptionHandler getExceptionHandler()

setExceptionHandler

public void setExceptionHandler(ExceptionHandler exceptionHandler)

isParallelProcessing

public boolean isParallelProcessing()

setParallelProcessing

public void setParallelProcessing(boolean parallelProcessing)

isOptimisticLocking

public boolean isOptimisticLocking()

setOptimisticLocking

public void setOptimisticLocking(boolean optimisticLocking)

getAggregationRepository

public AggregationRepository getAggregationRepository()

setAggregationRepository

public void setAggregationRepository(AggregationRepository aggregationRepository)

isDiscardOnCompletionTimeout

public boolean isDiscardOnCompletionTimeout()

setDiscardOnCompletionTimeout

public void setDiscardOnCompletionTimeout(boolean discardOnCompletionTimeout)

setForceCompletionOnStop

public void setForceCompletionOnStop(boolean forceCompletionOnStop)

setCompleteAllOnStop

public void setCompleteAllOnStop(boolean completeAllOnStop)

setTimeoutCheckerExecutorService

public void setTimeoutCheckerExecutorService(ScheduledExecutorService timeoutCheckerExecutorService)

getTimeoutCheckerExecutorService

public ScheduledExecutorService getTimeoutCheckerExecutorService()

isShutdownTimeoutCheckerExecutorService

public boolean isShutdownTimeoutCheckerExecutorService()

setShutdownTimeoutCheckerExecutorService

public void setShutdownTimeoutCheckerExecutorService(boolean shutdownTimeoutCheckerExecutorService)

setOptimisticLockRetryPolicy

public void setOptimisticLockRetryPolicy(OptimisticLockRetryPolicy optimisticLockRetryPolicy)

getOptimisticLockRetryPolicy

public OptimisticLockRetryPolicy getOptimisticLockRetryPolicy()

getAggregationStrategy

public AggregationStrategy getAggregationStrategy()

setAggregationStrategy

public void setAggregationStrategy(AggregationStrategy aggregationStrategy)

getCorrelationExpression

public Expression getCorrelationExpression()

setCorrelationExpression

public void setCorrelationExpression(Expression correlationExpression)

getAggregateController

public AggregateController getAggregateController()

setAggregateController

public void setAggregateController(AggregateController aggregateController)

doStart

protected void doStart()
                throws Exception

Description copied from class: ServiceSupport

Implementations override this method to support customized start/stop.

Important: See ServiceSupport.doStop() for more details.

Specified by:

doStart in class ServiceSupport

Throws:

Exception

See Also:

ServiceSupport.doStop()

doStop

protected void doStop()
               throws Exception

Description copied from class: ServiceSupport

Implementations override this method to support customized start/stop.

Important: Camel will invoke this ServiceSupport.doStop() method when the service is being stopped. This method will also be invoked if the service is still in uninitialized state (eg has not been started). The method is always called to allow the service to do custom logic when the service is being stopped, such as when CamelContext is shutting down.

Specified by:

doStop in class ServiceSupport

Throws:

Exception

See Also:

ServiceSupport.doStart()

prepareShutdown

public void prepareShutdown(boolean suspendOnly,
                            boolean forced)

Description copied from interface: ShutdownPrepared

Prepares for stop/shutdown.

The ShutdownStrategy supports preparing for shutdown using two steps. First a regular preparation, where the given forced parameter will be false. And if the shutdown times out, then the ShutdownStrategy performs a more aggressive shutdown, calling this method a second time with true for the given forced parameter. For example by graceful stopping any threads or the likes.

In addition a service can also be suspended (not stopped), and when this happens the parameter suspendOnly has the value true. This can be used to prepare the service for suspension, such as marking a worker thread to skip action.

For forced shutdown, then the service is expected to aggressively shutdown any child services, such as thread pools etc. This is the last chance it has to perform such duties.

Specified by:

prepareShutdown in interface ShutdownPrepared

Parameters:

suspendOnly - true if the intention is to only suspend the service, and not stop/shutdown the service.

forced - true is forcing a more aggressive shutdown, false is for preparing to shutdown.

deferShutdown

public boolean deferShutdown(ShutdownRunningTask shutdownRunningTask)

Description copied from interface: ShutdownAware

To defer shutdown during first phase of shutdown. This allows any pending exchanges to be completed and therefore ensure a graceful shutdown without loosing messages. At the very end when there are no more inflight and pending messages the consumer could then safely be shutdown.

This is needed by SedaConsumer.

Specified by:

deferShutdown in interface ShutdownAware

Parameters:

shutdownRunningTask - the configured option for how to act when shutting down running tasks.

Returns:

true to defer shutdown to very last.

getPendingExchangesSize

public int getPendingExchangesSize()

Description copied from interface: ShutdownAware

Gets the number of pending exchanges.

Some consumers has internal queues with Exchange which are pending. For example the SedaConsumer.

Return zero to indicate no pending exchanges and therefore ready to shutdown.

Specified by:

getPendingExchangesSize in interface ShutdownAware

Returns:

number of pending exchanges

doShutdown

protected void doShutdown()
                   throws Exception

Description copied from class: ServiceSupport

Implementations override this method to perform customized shutdown.

Overrides:

doShutdown in class ServiceSupport

Throws:

Exception

forceCompletionOfGroup

public int forceCompletionOfGroup(String key)

forceCompletionOfAllGroups

public int forceCompletionOfAllGroups()