The time to wait for the Java KafkaConsumer
to shutdown.
The default value is 20 seconds.
The CommitRecovery strategy for recovering from offset
commit exceptions.
The default is CommitRecovery#Default.
The time to wait for offset commits to complete.
The time to wait for offset commits to complete. If an offset commit
doesn't complete within this time, a CommitTimeoutException will
be raised instead.
The default value is 15 seconds.
The ConsumerFactory for creating the Java Consumer
.
The ConsumerFactory for creating the Java Consumer
.
The default is ConsumerFactory#Default. Note that you
normally don't need to have a custom ConsumerFactory,
and you should instead prefer to create a custom trait or
class similar to KafkaConsumer for testing purposes.
The ExecutionContext
on which to run blocking Kafka operations.
The ExecutionContext
on which to run blocking Kafka operations.
If not explicitly provided, a default ExecutionContext
will be
instantiated when creating a KafkaConsumer
instance.
The Deserializer
to use for deserializing record keys.
The maximum number of record batches to prefetch per topic-partition.
The maximum number of record batches to prefetch per topic-partition.
This means that, while records are being processed, there can be up
to maxPrefetchBatches * max.poll.records
records per topic-partition
that have already been fetched, and are waiting to be processed. You can
use withMaxPollRecords to control the max.poll.records
setting.
This setting effectively controls backpressure, i.e. the maximum number
of batches to prefetch per topic-parititon before starting to slow down
(not fetching more records) until processing has caught-up.
Note that prefetching cannot be disabled and is generally preferred since
it yields improved performance. The minimum value for this setting is 2
.
How often we should attempt to call poll
on the Java KafkaConsumer
.
The default value is 50 milliseconds.
How long we should allow the poll
call to block for in the
Java KafkaConsumer
.
The default value is 50 milliseconds.
Properties which can be provided when creating a Java KafkaConsumer
instance.
Properties which can be provided when creating a Java KafkaConsumer
instance. Numerous functions in ConsumerSettings add properties
here if the settings are used by the Java KafkaConsumer
.
The function used to specify metadata for records.
The function used to specify metadata for records. This
metadata will be included in OffsetAndMetadata
in the
CommittableOffsets, and can then be committed with
the offsets.
By default, there will be no metadata, as determined by
OffsetFetchResponse.NO_METADATA
.
The Deserializer
to use for deserializing record values.
Returns a new ConsumerSettings instance with the specified auto commit interval.
Returns a new ConsumerSettings instance with the specified
auto commit interval. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration
instead of a String
.
ConsumerConfig.AUTO_COMMIT_INTERVAL_MS_CONFIG
Returns a new ConsumerSettings instance with the specified auto offset reset.
Returns a new ConsumerSettings instance with the specified
auto offset reset. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with AutoOffsetReset instead of a String
.
ConsumerConfig.AUTO_OFFSET_RESET_CONFIG
Returns a new ConsumerSettings instance with the specified bootstrap servers.
Returns a new ConsumerSettings instance with the specified bootstrap servers. This is equivalent to setting the following property using the withProperty function.
ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG
Returns a new ConsumerSettings instance with the specified client id.
Returns a new ConsumerSettings instance with the specified client id. This is equivalent to setting the following property using the withProperty function.
ConsumerConfig.CLIENT_ID_CONFIG
Creates a new ConsumerSettings with the specified closeTimeout.
Creates a new ConsumerSettings with the specified CommitRecovery as the commitRecovery to use.
Creates a new ConsumerSettings with the specified commitTimeout.
Creates a new ConsumerSettings with the specified ConsumerFactory as the consumerFactory to use.
Creates a new ConsumerSettings with the specified ConsumerFactory as the consumerFactory to use. Note that under normal usage you don't need to have a custom ConsumerFactory instance. For testing, you should prefer to use a custom trait or class similar to KafkaConsumer.
Returns a new ConsumerSettings instance with the specified default api timeout.
Returns a new ConsumerSettings instance with the specified
default api timeout. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration
instead of a String
.
ConsumerConfig.DEFAULT_API_TIMEOUT_MS_CONFIG
Returns a new ConsumerSettings instance with the specified auto commit setting.
Returns a new ConsumerSettings instance with the specified
auto commit setting. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a Boolean
instead of a String
.
ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG
Note that by default, this setting is set to false
.
Returns a new ConsumerSettings instance with the specified group id.
Returns a new ConsumerSettings instance with the specified group id. This is equivalent to setting the following property using the withProperty function.
ConsumerConfig.GROUP_ID_CONFIG
Returns a new ConsumerSettings instance with the specified heartbeat interval.
Returns a new ConsumerSettings instance with the specified
heartbeat interval. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration
instead of a String
.
ConsumerConfig.HEARTBEAT_INTERVAL_MS_CONFIG
Returns a new ConsumerSettings instance with the specified max poll interval.
Returns a new ConsumerSettings instance with the specified
max poll interval. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration
instead of a String
.
ConsumerConfig.MAX_POLL_INTERVAL_MS_CONFIG
Returns a new ConsumerSettings instance with the specified max poll records.
Returns a new ConsumerSettings instance with the specified
max poll records. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with an Int
instead of a String
.
ConsumerConfig.MAX_POLL_RECORDS_CONFIG
Creates a new ConsumerSettings with the specified value for maxPrefetchBatches.
Creates a new ConsumerSettings with the specified value
for maxPrefetchBatches. Note that if a value lower than
the minimum 2
is specified, maxPrefetchBatches will
instead be set to 2
and not the specified value.
Creates a new ConsumerSettings with the specified pollInterval.
Creates a new ConsumerSettings with the specified pollTimeout.
Includes the specified keys and values as properties.
Includes the specified keys and values as properties. The
keys should be part of the ConsumerConfig
keys, and
the values should be valid choices for the keys.
Includes the specified keys and values as properties.
Includes the specified keys and values as properties. The
keys should be part of the ConsumerConfig
keys, and
the values should be valid choices for the keys.
Includes a property with the specified key
and value
.
Includes a property with the specified key
and value
.
The key should be one of the keys in ConsumerConfig
,
and the value should be a valid choice for the key.
Creates a new ConsumerSettings with the specified recordMetadata.
Returns a new ConsumerSettings instance with the specified request timeout.
Returns a new ConsumerSettings instance with the specified
request timeout. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration
instead of a String
.
ConsumerConfig.REQUEST_TIMEOUT_MS_CONFIG
Returns a new ConsumerSettings instance with the specified session timeout.
Returns a new ConsumerSettings instance with the specified
session timeout. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration
instead of a String
.
ConsumerConfig.SESSION_TIMEOUT_MS_CONFIG
ConsumerSettings contain settings necessary to create a KafkaConsumer. At the very least, this includes key and value deserializers.
Several convenience functions are provided so that you don't have to work with
String
values andConsumerConfig
for configuration. It's still possible to specifyConsumerConfig
values with functions like withProperty.ConsumerSettings instances are immutable and all modification functions return a new ConsumerSettings instance.
Use
ConsumerSettings#apply
to create a new instance.