org.apache.camel.component.file
Class GenericFileConsumer<T>

java.lang.Object
  org.apache.camel.support.ServiceSupport
      org.apache.camel.impl.DefaultConsumer
          org.apache.camel.impl.ScheduledPollConsumer
              org.apache.camel.impl.ScheduledBatchPollingConsumer
                  org.apache.camel.component.file.GenericFileConsumer<T>

All Implemented Interfaces:

Runnable, BatchConsumer, Consumer, PollingConsumerPollingStrategy, Service, ShutdownableService, ShutdownAware, ShutdownPrepared, StatefulService, SuspendableService

Direct Known Subclasses:

FileConsumer

public abstract class GenericFileConsumer<T>
extends ScheduledBatchPollingConsumer

Base class for file consumers.

Field Summary

protected Processor	customProcessor
protected boolean	eagerLimitMaxMessagesPerPoll
protected GenericFileEndpoint<T>	endpoint
protected String	fileExpressionResult
protected org.slf4j.Logger	log
protected boolean	loggedIn
protected GenericFileOperations<T>	operations
protected int	pendingExchanges
protected ShutdownRunningTask	shutdownRunningTask

Fields inherited from class org.apache.camel.impl.ScheduledBatchPollingConsumer

maxMessagesPerPoll

Fields inherited from class org.apache.camel.support.ServiceSupport

shutdown, shuttingdown, started, starting, stopped, stopping, suspended, suspending

Constructor Summary

GenericFileConsumer(GenericFileEndpoint<T> endpoint, Processor processor, GenericFileOperations<T> operations)

Method Summary

boolean	canPollMoreFiles(List<?> fileList) Whether or not we can continue polling for more files
protected void	customProcessExchange(Exchange exchange, Processor processor) Processes the exchange using a custom processor.
protected void	doStart() Implementations override this method to support customized start/stop.
Processor	getCustomProcessor()
boolean	isEagerLimitMaxMessagesPerPoll()
protected boolean	isInProgress(GenericFile<T> file) Is the given file already in progress.
protected boolean	isMatched(GenericFile<T> file, boolean isDirectory, List<T> files) Strategy to perform file matching based on endpoint configuration.
protected abstract boolean	isMatched(GenericFile<T> file, String doneFileName, List<T> files) Strategy to perform file matching based on endpoint configuration in terms of done file name.
protected boolean	isValidFile(GenericFile<T> file, boolean isDirectory, List<T> files) Strategy for validating if the given remote file should be included or not
protected int	poll() Poll for files
protected abstract boolean	pollDirectory(String fileName, List<GenericFile<T>> fileList, int depth) Polls the given directory for files to process
protected void	postPollCheck() Override if required.
protected boolean	prePollCheck() Override if required.
int	processBatch(Queue<Object> exchanges) Processes the list of Exchange objects in a batch.
protected void	processExchange(Exchange exchange) Processes the exchange
protected void	removeExcessiveInProgressFiles(Deque<Exchange> exchanges, int limit)
void	setCustomProcessor(Processor processor) Use a custom processor to process the exchange.
void	setEagerLimitMaxMessagesPerPoll(boolean eagerLimitMaxMessagesPerPoll)
void	setOperations(GenericFileOperations<T> operations) Sets the operations to be used.

Methods inherited from class org.apache.camel.impl.ScheduledBatchPollingConsumer

deferShutdown, getMaxMessagesPerPoll, getPendingExchangesSize, isBatchAllowed, prepareShutdown, processEmptyMessage, setMaxMessagesPerPoll

Methods inherited from class org.apache.camel.impl.ScheduledPollConsumer

afterPoll, beforePoll, doShutdown, doStop, doSuspend, getDelay, getInitialDelay, getPollStrategy, getRunLoggingLevel, getScheduledExecutorService, getTimeUnit, isGreedy, isPollAllowed, isPolling, isSchedulerStarted, isSendEmptyMessageWhenIdle, isStartScheduler, isUseFixedDelay, onInit, run, setDelay, setGreedy, setInitialDelay, setPollStrategy, setRunLoggingLevel, setScheduledExecutorService, setSendEmptyMessageWhenIdle, setStartScheduler, setTimeUnit, setUseFixedDelay, startScheduler

Methods inherited from class org.apache.camel.impl.DefaultConsumer

getAsyncProcessor, getEndpoint, getExceptionHandler, getProcessor, handleException, handleException, setExceptionHandler, toString

Methods inherited from class org.apache.camel.support.ServiceSupport

doResume, getStatus, getVersion, isRunAllowed, isStarted, isStarting, isStopped, isStopping, isSuspended, isSuspending, resume, shutdown, start, stop, suspend

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.camel.Consumer

getEndpoint

Methods inherited from interface org.apache.camel.Service

start, stop

Methods inherited from interface org.apache.camel.SuspendableService

isSuspended, resume, suspend

Field Detail

log

protected final transient org.slf4j.Logger log

endpoint

protected GenericFileEndpoint<T> endpoint

operations

protected GenericFileOperations<T> operations

loggedIn

protected boolean loggedIn

fileExpressionResult

protected String fileExpressionResult

shutdownRunningTask

protected volatile ShutdownRunningTask shutdownRunningTask

pendingExchanges

protected volatile int pendingExchanges

customProcessor

protected Processor customProcessor

eagerLimitMaxMessagesPerPoll

protected boolean eagerLimitMaxMessagesPerPoll

Constructor Detail

GenericFileConsumer

public GenericFileConsumer(GenericFileEndpoint<T> endpoint,
                           Processor processor,
                           GenericFileOperations<T> operations)

Method Detail

getCustomProcessor

public Processor getCustomProcessor()

setCustomProcessor

public void setCustomProcessor(Processor processor)

Use a custom processor to process the exchange.

Only set this if you need to do custom processing, instead of the regular processing.

This is for example used to browse file endpoints by leveraging the file consumer to poll the directory to gather the list of exchanges. But to avoid processing the files regularly we can use a custom processor.

Parameters:

processor - a custom processor

isEagerLimitMaxMessagesPerPoll

public boolean isEagerLimitMaxMessagesPerPoll()

setEagerLimitMaxMessagesPerPoll

public void setEagerLimitMaxMessagesPerPoll(boolean eagerLimitMaxMessagesPerPoll)

poll

protected int poll()
            throws Exception

Poll for files

Specified by:

poll in class ScheduledPollConsumer

Returns:

number of messages polled, will be 0 if no message was polled at all.

Throws:

Exception - can be thrown if an exception occurred during polling

processBatch

public int processBatch(Queue<Object> exchanges)

Description copied from interface: BatchConsumer

Processes the list of Exchange objects in a batch.

Each message exchange will be processed individually but the batch consumer will add properties with the current index and total in the batch. The items in the Queue may actually be Holder objects that store other data alongside the Exchange.

Parameters:

exchanges - list of items in this batch

Returns:

number of messages actually processed

removeExcessiveInProgressFiles

protected void removeExcessiveInProgressFiles(Deque<Exchange> exchanges,
                                              int limit)

canPollMoreFiles

public boolean canPollMoreFiles(List<?> fileList)

Whether or not we can continue polling for more files

Parameters:

fileList - the current list of gathered files

Returns:

true to continue, false to stop due hitting maxMessagesPerPoll limit

prePollCheck

protected boolean prePollCheck()
                        throws Exception

Override if required. Perform some checks (and perhaps actions) before we poll.

Returns:

true to poll, false to skip this poll.

Throws:

Exception

postPollCheck

protected void postPollCheck()

Override if required. Perform some checks (and perhaps actions) after we have polled.

pollDirectory

protected abstract boolean pollDirectory(String fileName,
                                         List<GenericFile<T>> fileList,
                                         int depth)

Polls the given directory for files to process

Parameters:

fileName - current directory or file

fileList - current list of files gathered

depth - the current depth of the directory (will start from 0)

Returns:

whether or not to continue polling, false means the maxMessagesPerPoll limit has been hit

setOperations

public void setOperations(GenericFileOperations<T> operations)

Sets the operations to be used.

Can be used to set a fresh operations in case of recovery attempts

Parameters:

operations - the operations

processExchange

protected void processExchange(Exchange exchange)

Processes the exchange

Parameters:

exchange - the exchange

customProcessExchange

protected void customProcessExchange(Exchange exchange,
                                     Processor processor)

Processes the exchange using a custom processor.

Parameters:

exchange - the exchange

processor - the custom processor

isValidFile

protected boolean isValidFile(GenericFile<T> file,
                              boolean isDirectory,
                              List<T> files)

Strategy for validating if the given remote file should be included or not

Parameters:

file - the file

isDirectory - whether the file is a directory or a file

files - files in the directory

Returns:

true to include the file, false to skip it

isMatched

protected boolean isMatched(GenericFile<T> file,
                            boolean isDirectory,
                            List<T> files)

Strategy to perform file matching based on endpoint configuration.

Will always return false for certain files/folders:

• Starting with a dot
• lock files

And then true for directories.

Parameters:

file - the file

isDirectory - whether the file is a directory or a file

files - files in the directory

Returns:

true if the file is matched, false if not

isMatched

protected abstract boolean isMatched(GenericFile<T> file,
                                     String doneFileName,
                                     List<T> files)

Strategy to perform file matching based on endpoint configuration in terms of done file name.

Parameters:

file - the file

doneFileName - the done file name (without any paths)

files - files in the directory

Returns:

true if the file is matched, false if not

isInProgress

protected boolean isInProgress(GenericFile<T> file)

Is the given file already in progress.

Parameters:

file - the file

Returns:

true if the file is already in progress

doStart

protected void doStart()
                throws Exception

Description copied from class: ServiceSupport

Implementations override this method to support customized start/stop.

Important: See ServiceSupport.doStop() for more details.

Overrides:

doStart in class ScheduledPollConsumer

Throws:

Exception

See Also:

ServiceSupport.doStop()