Package org.apache.camel.processor.resume.kafka

Class SingleNodeKafkaResumeStrategy<K,V>

java.lang.Object

org.apache.camel.processor.resume.kafka.SingleNodeKafkaResumeStrategy<K,V>

Type Parameters:

K - the type of key

V - the type of the value

All Implemented Interfaces:

AutoCloseable, KafkaResumeStrategy<K,V>, org.apache.camel.resume.ResumeStrategy, org.apache.camel.resume.UpdatableConsumerResumeStrategy<K,V,org.apache.camel.resume.Resumable<K,V>>, org.apache.camel.Service

Direct Known Subclasses:

MultiNodeKafkaResumeStrategy

public class SingleNodeKafkaResumeStrategy<K,V>
extends Object
implements KafkaResumeStrategy<K,V>

A resume strategy that publishes offsets to a Kafka topic. This resume strategy is suitable for single node integrations. For multi-node integrations (i.e: using clusters with the master component check MultiNodeKafkaResumeStrategy.

Constructor Summary

Constructors

Constructor	Description
SingleNodeKafkaResumeStrategy(String bootstrapServers, String topic, org.apache.camel.resume.cache.ResumeCache<K,V> resumeCache, org.apache.camel.resume.ResumeAdapter resumeAdapter)	Builds an instance of this class
SingleNodeKafkaResumeStrategy(String topic, org.apache.camel.resume.cache.ResumeCache<K,V> resumeCache, org.apache.camel.resume.ResumeAdapter resumeAdapter, Properties producerConfig, Properties consumerConfig)	Builds an instance of this class

Method Summary

Modifier and Type	Method	Description
void	build()
protected void	checkAndSubscribe(String topic)	Subscribe to the topic if not subscribed yet
void	checkAndSubscribe(String topic, long remaining)	Subscribe to the topic if not subscribed yet
void	close()
protected org.apache.kafka.clients.consumer.ConsumerRecords<K,V>	consume()	Consumes message from the topic previously setup
protected org.apache.kafka.clients.consumer.ConsumerRecords<K,V>	consume(int retries)	Consumes message from the topic previously setup
static Properties	createConsumer(String bootstrapServers)	Creates a basic string-based consumer
static Properties	createProducer(String bootstrapServers)	Creates a basic string-based producer
org.apache.camel.resume.ResumeAdapter	getAdapter()
protected org.apache.kafka.clients.consumer.Consumer<K,V>	getConsumer()
protected Properties	getConsumerConfig()
protected org.apache.kafka.clients.consumer.ConsumerRebalanceListener	getConsumerRebalanceListener(long remaining)	Creates a new consumer rebalance listener.
Duration	getPollDuration()
protected org.apache.kafka.clients.producer.Producer<K,V>	getProducer()
protected Properties	getProducerConfig()
protected Collection<RecordError>	getProducerErrors()	Gets the set record of sent items
protected org.apache.camel.resume.cache.ResumeCache<K,V>	getResumeCache()
protected String	getTopic()
void	init()
protected void	loadCache()	Loads the existing data into the cache
protected void	produce(K key, V message)	Sends data to a topic.
void	resetProducerErrors()	Clear the producer errors
void	setPollDuration(Duration pollDuration)
void	start()
void	stop()
void	subscribe()
protected void	unsubscribe()	Unsubscribe from the topic
void	updateLastOffset(org.apache.camel.resume.Resumable<K,V> offset)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.camel.resume.ResumeStrategy

getAdapter

Constructor Detail

SingleNodeKafkaResumeStrategy

public SingleNodeKafkaResumeStrategy(String bootstrapServers,
                                     String topic,
                                     org.apache.camel.resume.cache.ResumeCache<K,V> resumeCache,
                                     org.apache.camel.resume.ResumeAdapter resumeAdapter)

Builds an instance of this class

Parameters:

bootstrapServers - the address of the Kafka broker

topic - the topic where to publish the offsets

resumeCache - a cache instance where to store the offsets locally for faster access

resumeAdapter - the component-specific resume adapter

SingleNodeKafkaResumeStrategy

public SingleNodeKafkaResumeStrategy(String topic,
                                     org.apache.camel.resume.cache.ResumeCache<K,V> resumeCache,
                                     org.apache.camel.resume.ResumeAdapter resumeAdapter,
                                     Properties producerConfig,
                                     Properties consumerConfig)

Builds an instance of this class

Parameters:

topic - the topic where to publish the offsets

resumeCache - a cache instance where to store the offsets locally for faster access

resumeAdapter - the component-specific resume adapter

producerConfig - the set of properties to be used by the Kafka producer within this class

consumerConfig - the set of properties to be used by the Kafka consumer within this class

Method Detail

createProducer

public static Properties createProducer(String bootstrapServers)

Creates a basic string-based producer

Parameters:

bootstrapServers - the Kafka host

Returns:

A set of default properties for producing string-based key/pair records from Kafka

createConsumer

public static Properties createConsumer(String bootstrapServers)

Creates a basic string-based consumer

Parameters:

bootstrapServers - the Kafka host

Returns:

A set of default properties for consuming string-based key/pair records from Kafka

produce

protected void produce(K key,
                       V message)
                throws ExecutionException,
                       InterruptedException

Sends data to a topic. The records will always be sent asynchronously. If there's an error, a producer error counter will be increased.

Parameters:

message - the message to send

Throws:

ExecutionException

InterruptedException

See Also:

getProducerErrors()

updateLastOffset

public void updateLastOffset(org.apache.camel.resume.Resumable<K,V> offset)
                      throws Exception

Specified by:

updateLastOffset in interface org.apache.camel.resume.UpdatableConsumerResumeStrategy<K,V,org.apache.camel.resume.Resumable<K,V>>

Throws:

Exception

loadCache

protected void loadCache()
                  throws Exception

Loads the existing data into the cache

Throws:

Exception

checkAndSubscribe

protected void checkAndSubscribe(String topic)

Subscribe to the topic if not subscribed yet

Parameters:

topic - the topic to consume the messages from

checkAndSubscribe

public void checkAndSubscribe(String topic,
                              long remaining)

Subscribe to the topic if not subscribed yet

Parameters:

topic - the topic to consume the messages from

remaining - the number of messages to rewind from the last offset position (used to fill the cache)

getConsumerRebalanceListener

protected org.apache.kafka.clients.consumer.ConsumerRebalanceListener getConsumerRebalanceListener(long remaining)

Creates a new consumer rebalance listener. This can be useful for setting the exact Kafka offset when necessary to read a limited amount of messages or customize the resume strategy behavior when a rebalance occurs.

Parameters:

remaining -

Returns:

unsubscribe

protected void unsubscribe()

Unsubscribe from the topic

consume

protected org.apache.kafka.clients.consumer.ConsumerRecords<K,V> consume()

Consumes message from the topic previously setup

Returns:

An instance of the consumer records

consume

protected org.apache.kafka.clients.consumer.ConsumerRecords<K,V> consume(int retries)

Consumes message from the topic previously setup

Parameters:

retries - how many times to retry consuming data from the topic

Returns:

An instance of the consumer records

subscribe

public void subscribe()
               throws Exception

Throws:

Exception

getAdapter

public org.apache.camel.resume.ResumeAdapter getAdapter()

Specified by:

getAdapter in interface org.apache.camel.resume.ResumeStrategy

getProducerErrors

protected Collection<RecordError> getProducerErrors()

Gets the set record of sent items

Returns:

build

public void build()

Specified by:

build in interface org.apache.camel.Service

init

public void init()

Specified by:

init in interface org.apache.camel.Service

stop

public void stop()

Specified by:

stop in interface org.apache.camel.Service

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface org.apache.camel.Service

Throws:

IOException

start

public void start()

Specified by:

start in interface org.apache.camel.Service

getPollDuration

public Duration getPollDuration()

setPollDuration

public void setPollDuration(Duration pollDuration)

getConsumer

protected org.apache.kafka.clients.consumer.Consumer<K,V> getConsumer()

getProducer

protected org.apache.kafka.clients.producer.Producer<K,V> getProducer()

getProducerConfig

protected Properties getProducerConfig()

getConsumerConfig

protected Properties getConsumerConfig()

getTopic

protected String getTopic()

getResumeCache

protected org.apache.camel.resume.cache.ResumeCache<K,V> getResumeCache()

resetProducerErrors

public void resetProducerErrors()

Clear the producer errors