ConsumerSettings contain settings necessary to create a
KafkaConsumer. At the very least, this includes key and
value deserializers.
The following consumer configuration defaults are used.
auto.offset.resetis set tononeto avoid the surprise of the otherwise defaultlatestsetting.enable.auto.commitis set tofalsesince offset commits are managed manually.
Several convenience functions are provided so that you don't have to work with `String` values and `ConsumerConfig` for configuration. It's still possible to specify `ConsumerConfig` values with functions like [[withProperty]].
[[ConsumerSettings]] instances are immutable and all modification functions return a new [[ConsumerSettings]] instance.
Use `ConsumerSettings#apply` to create a new instance.
- Companion:
- object
Value members
Abstract methods
The time to wait for the Java KafkaConsumer to shutdown.
The default value is 20 seconds.
The time to wait for the Java KafkaConsumer to shutdown.
The default value is 20 seconds.
The CommitRecovery strategy for recovering from offset
commit exceptions.
The default is CommitRecovery#Default.
The CommitRecovery strategy for recovering from offset
commit exceptions.
The default is CommitRecovery#Default.
The time to wait for offset commits to complete. If an offset commit
doesn't complete within this time, a CommitTimeoutException will
be raised instead.
The default value is 15 seconds.
The time to wait for offset commits to complete. If an offset commit
doesn't complete within this time, a CommitTimeoutException will
be raised instead.
The default value is 15 seconds.
A custom ExecutionContext to use for blocking Kafka operations. If not
provided, a default single-threaded ExecutionContext will be created
when creating a KafkaConsumer instance.
A custom ExecutionContext to use for blocking Kafka operations. If not
provided, a default single-threaded ExecutionContext will be created
when creating a KafkaConsumer instance.
The Deserializer to use for deserializing record keys.
The Deserializer to use for deserializing record keys.
The maximum number of record batches to prefetch per topic-partition.
This means that, while records are being processed, there can be up
to maxPrefetchBatches * max.poll.records records per topic-partition
that have already been fetched, and are waiting to be processed. You can
use withMaxPollRecords to control the max.poll.records setting.
This setting effectively controls backpressure, i.e. the maximum number
of batches to prefetch per topic-parititon before starting to slow down
(not fetching more records) until processing has caught-up.
Note that prefetching cannot be disabled and is generally preferred since
it yields improved performance. The minimum value for this setting is 2.
The maximum number of record batches to prefetch per topic-partition.
This means that, while records are being processed, there can be up
to maxPrefetchBatches * max.poll.records records per topic-partition
that have already been fetched, and are waiting to be processed. You can
use withMaxPollRecords to control the max.poll.records setting.
This setting effectively controls backpressure, i.e. the maximum number
of batches to prefetch per topic-parititon before starting to slow down
(not fetching more records) until processing has caught-up.
Note that prefetching cannot be disabled and is generally preferred since
it yields improved performance. The minimum value for this setting is 2.
How often we should attempt to call poll on the Java KafkaConsumer.
The default value is 50 milliseconds.
How often we should attempt to call poll on the Java KafkaConsumer.
The default value is 50 milliseconds.
How long we should allow the poll call to block for in the
Java KafkaConsumer.
The default value is 50 milliseconds.
How long we should allow the poll call to block for in the
Java KafkaConsumer.
The default value is 50 milliseconds.
Properties which can be provided when creating a Java KafkaConsumer
instance. Numerous functions in ConsumerSettings add properties
here if the settings are used by the Java KafkaConsumer.
Properties which can be provided when creating a Java KafkaConsumer
instance. Numerous functions in ConsumerSettings add properties
here if the settings are used by the Java KafkaConsumer.
The function used to specify metadata for records. This
metadata will be included in OffsetAndMetadata in the
CommittableOffsets, and can then be committed with
the offsets.
By default, there will be no metadata, as determined by
OffsetFetchResponse.NO_METADATA.
The function used to specify metadata for records. This
metadata will be included in OffsetAndMetadata in the
CommittableOffsets, and can then be committed with
the offsets.
By default, there will be no metadata, as determined by
OffsetFetchResponse.NO_METADATA.
The Deserializer to use for deserializing record values.
The Deserializer to use for deserializing record values.
Returns a new ConsumerSettings instance with the specified
allow auto create topics. This is equivalent to setting the
following property using the withProperty function, except
you can specify it with a Boolean instead of a String.
Returns a new ConsumerSettings instance with the specified
allow auto create topics. This is equivalent to setting the
following property using the withProperty function, except
you can specify it with a Boolean instead of a String.
ConsumerConfig.ALLOW_AUTO_CREATE_TOPICS_CONFIG
Returns a new ConsumerSettings instance with the specified
auto commit interval. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration instead of a String.
Returns a new ConsumerSettings instance with the specified
auto commit interval. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration instead of a String.
ConsumerConfig.AUTO_COMMIT_INTERVAL_MS_CONFIG
Returns a new ConsumerSettings instance with the specified
auto offset reset. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with AutoOffsetReset instead of a String.
Returns a new ConsumerSettings instance with the specified
auto offset reset. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with AutoOffsetReset instead of a String.
ConsumerConfig.AUTO_OFFSET_RESET_CONFIG
Returns a new ConsumerSettings instance with the specified bootstrap servers. This is equivalent to setting the following property using the withProperty function.
Returns a new ConsumerSettings instance with the specified bootstrap servers. This is equivalent to setting the following property using the withProperty function.
ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG
Returns a new ConsumerSettings instance with the specified client id. This is equivalent to setting the following property using the withProperty function.
Returns a new ConsumerSettings instance with the specified client id. This is equivalent to setting the following property using the withProperty function.
ConsumerConfig.CLIENT_ID_CONFIG
Returns a new ConsumerSettings instance with the specified client rack. This is equivalent to setting the following property using the withProperty function.
Returns a new ConsumerSettings instance with the specified client rack. This is equivalent to setting the following property using the withProperty function.
ConsumerConfig.CLIENT_RACK_CONFIG
Creates a new ConsumerSettings with the specified closeTimeout.
Creates a new ConsumerSettings with the specified closeTimeout.
Creates a new ConsumerSettings with the specified CommitRecovery as the commitRecovery to use.
Creates a new ConsumerSettings with the specified CommitRecovery as the commitRecovery to use.
Creates a new ConsumerSettings with the specified commitTimeout.
Creates a new ConsumerSettings with the specified commitTimeout.
Includes the credentials properties from the provided KafkaCredentialStore
Includes the credentials properties from the provided KafkaCredentialStore
Returns a new ConsumerSettings instance with the specified ExecutionContext to use for blocking operations.
Returns a new ConsumerSettings instance with the specified ExecutionContext to use for blocking operations.
Because the underlying Java consumer is not thread-safe, the ExecutionContext must be single-threaded. If in doubt, leave this unset so that a default single-threaded blocker will be provided.
Returns a new ConsumerSettings instance with the specified
default api timeout. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration instead of a String.
Returns a new ConsumerSettings instance with the specified
default api timeout. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration instead of a String.
ConsumerConfig.DEFAULT_API_TIMEOUT_MS_CONFIG
Creates a new ConsumerSettings instance that replaces the serializers with those provided.
Note that this will remove any custom recordMetadata configuration.
Creates a new ConsumerSettings instance that replaces the serializers with those provided.
Note that this will remove any custom recordMetadata configuration.
Returns a new ConsumerSettings instance with the specified
auto commit setting. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a Boolean instead of a String.
Returns a new ConsumerSettings instance with the specified
auto commit setting. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a Boolean instead of a String.
ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG
Note that by default, this setting is set to false.
Returns a new ConsumerSettings instance with the specified group id. This is equivalent to setting the following property using the withProperty function.
Returns a new ConsumerSettings instance with the specified group id. This is equivalent to setting the following property using the withProperty function.
ConsumerConfig.GROUP_ID_CONFIG
Returns a new ConsumerSettings instance with the specified group instance id. This is equivalent to setting the following property using the withProperty function.
Returns a new ConsumerSettings instance with the specified group instance id. This is equivalent to setting the following property using the withProperty function.
ConsumerConfig.GROUP_INSTANCE_ID_CONFIG
Returns a new ConsumerSettings instance with the specified
heartbeat interval. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration instead of a String.
Returns a new ConsumerSettings instance with the specified
heartbeat interval. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration instead of a String.
ConsumerConfig.HEARTBEAT_INTERVAL_MS_CONFIG
Returns a new ConsumerSettings instance with the specified
isolation level. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with an IsolationLevel instead of a String.
Returns a new ConsumerSettings instance with the specified
isolation level. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with an IsolationLevel instead of a String.
ConsumerConfig.ISOLATION_LEVEL_CONFIG
Returns a new ConsumerSettings instance with the specified
max poll interval. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration instead of a String.
Returns a new ConsumerSettings instance with the specified
max poll interval. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration instead of a String.
ConsumerConfig.MAX_POLL_INTERVAL_MS_CONFIG
Returns a new ConsumerSettings instance with the specified
max poll records. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with an Int instead of a String.
Returns a new ConsumerSettings instance with the specified
max poll records. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with an Int instead of a String.
ConsumerConfig.MAX_POLL_RECORDS_CONFIG
Creates a new ConsumerSettings with the specified value
for maxPrefetchBatches. Note that if a value lower than
the minimum 2 is specified, maxPrefetchBatches will
instead be set to 2 and not the specified value.
Creates a new ConsumerSettings with the specified value
for maxPrefetchBatches. Note that if a value lower than
the minimum 2 is specified, maxPrefetchBatches will
instead be set to 2 and not the specified value.
Creates a new ConsumerSettings with the specified pollInterval.
Creates a new ConsumerSettings with the specified pollInterval.
Creates a new ConsumerSettings with the specified pollTimeout.
Creates a new ConsumerSettings with the specified pollTimeout.
Includes the specified keys and values as properties. The
keys should be part of the ConsumerConfig keys, and
the values should be valid choices for the keys.
Includes the specified keys and values as properties. The
keys should be part of the ConsumerConfig keys, and
the values should be valid choices for the keys.
Includes the specified keys and values as properties. The
keys should be part of the ConsumerConfig keys, and
the values should be valid choices for the keys.
Includes the specified keys and values as properties. The
keys should be part of the ConsumerConfig keys, and
the values should be valid choices for the keys.
Includes a property with the specified key and value.
The key should be one of the keys in ConsumerConfig,
and the value should be a valid choice for the key.
Includes a property with the specified key and value.
The key should be one of the keys in ConsumerConfig,
and the value should be a valid choice for the key.
Creates a new ConsumerSettings with the specified recordMetadata.
Note that replacing the serializers via withSerializers will reset
this to the default.
Creates a new ConsumerSettings with the specified recordMetadata.
Note that replacing the serializers via withSerializers will reset
this to the default.
Returns a new ConsumerSettings instance with the specified
request timeout. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration instead of a String.
Returns a new ConsumerSettings instance with the specified
request timeout. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration instead of a String.
ConsumerConfig.REQUEST_TIMEOUT_MS_CONFIG
Returns a new ConsumerSettings instance with the specified
session timeout. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration instead of a String.
Returns a new ConsumerSettings instance with the specified
session timeout. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration instead of a String.
ConsumerConfig.SESSION_TIMEOUT_MS_CONFIG