embedded-kafka-streams
API
Generated with
embedded-kafka-streams/io.github.embeddedkafka.streams/EmbeddedKafkaStreams

EmbeddedKafkaStreams

io.github.embeddedkafka.streams.EmbeddedKafkaStreams
See theEmbeddedKafkaStreams companion object
trait EmbeddedKafkaStreams extends EmbeddedKafka

Helper trait for running Kafka Streams. Use .runStreams to execute your streams.

Attributes

Companion
object
Graph
Supertypes
trait EmbeddedKafka
trait KafkaOps
trait ZooKeeperOps
trait ProducerOps[EmbeddedKafkaConfig]
trait ConsumerOps[EmbeddedKafkaConfig]
trait AdminOps[EmbeddedKafkaConfig]
class Object
trait Matchable
class Any
Show all
Known subtypes

Members list

Value members

Inherited methods

def consumeFirstKeyedMessageFrom[K, V](topic: String, autoCommit: Boolean, timeout: Duration)(implicit config: EmbeddedKafkaConfig, keyDeserializer: Deserializer[K], valueDeserializer: Deserializer[V]): (K, V)

Consumes the first message available in a given topic, deserializing it as type (K, V).

Consumes the first message available in a given topic, deserializing it as type (K, V).

Only the message that is returned is committed if autoCommit is false. If autoCommit is true then all messages that were polled will be committed.

Value parameters

autoCommit

if false, only the offset for the consumed message will be committed. if true, the offset for the last polled message will be committed instead.

config

an implicit EmbeddedKafkaConfig

keyDeserializer

an implicit Deserializer for the type K

timeout

the interval to wait for messages before throwing TimeoutException

topic

the topic to consume a message from

valueDeserializer

an implicit Deserializer for the type V

Attributes

Returns

the first message consumed from the given topic, with a type (K, V)

Inherited from:
ConsumerOps
def consumeFirstMessageFrom[V](topic: String, autoCommit: Boolean, timeout: Duration)(implicit config: EmbeddedKafkaConfig, valueDeserializer: Deserializer[V]): V

Consumes the first message available in a given topic, deserializing it as type V.

Consumes the first message available in a given topic, deserializing it as type V.

Only the message that is returned is committed if autoCommit is false. If autoCommit is true then all messages that were polled will be committed.

Value parameters

autoCommit

if false, only the offset for the consumed message will be committed. if true, the offset for the last polled message will be committed instead.

config

an implicit EmbeddedKafkaConfig

timeout

the interval to wait for messages before throwing TimeoutException

topic

the topic to consume a message from

valueDeserializer

an implicit Deserializer for the type V

Attributes

Returns

the first message consumed from the given topic, with a type V

Inherited from:
ConsumerOps
def consumeFirstStringMessageFrom(topic: String, autoCommit: Boolean, timeout: Duration)(implicit config: EmbeddedKafkaConfig): String

Attributes

Inherited from:
ConsumerOps
def consumeNumberKeyedMessagesFrom[K, V](topic: String, number: Int, autoCommit: Boolean, timeout: Duration)(implicit config: EmbeddedKafkaConfig, keyDeserializer: Deserializer[K], valueDeserializer: Deserializer[V]): List[(K, V)]

Attributes

Inherited from:
ConsumerOps
def consumeNumberKeyedMessagesFromTopics[K, V](topics: Set[String], number: Int, autoCommit: Boolean, timeout: Duration, resetTimeoutOnEachMessage: Boolean)(implicit config: EmbeddedKafkaConfig, keyDeserializer: Deserializer[K], valueDeserializer: Deserializer[V]): Map[String, List[(K, V)]]

Consumes the first n messages available in given topics, deserializes them as type (K, V), and returns the n messages in a Map from topic name to List[(K, V)].

Consumes the first n messages available in given topics, deserializes them as type (K, V), and returns the n messages in a Map from topic name to List[(K, V)].

Only the messages that are returned are committed if autoCommit is false. If autoCommit is true then all messages that were polled will be committed.

Value parameters

autoCommit

if false, only the offset for the consumed messages will be committed. if true, the offset for the last polled message will be committed instead.

config

an implicit EmbeddedKafkaConfig

keyDeserializer

an implicit Deserializer for the type K

number

the number of messages to consume in a batch

resetTimeoutOnEachMessage

when true, throw TimeoutException if we have a silent period (no incoming messages) for the timeout interval; when false, throw TimeoutException after the timeout interval if we haven't received all of the expected messages

timeout

the interval to wait for messages before throwing TimeoutException

topics

the topics to consume messages from

valueDeserializer

an implicit Deserializer for the type V

Attributes

Returns

the List of messages consumed from the given topics, each with a type (K, V)

Inherited from:
ConsumerOps
def consumeNumberMessagesFrom[V](topic: String, number: Int, autoCommit: Boolean, timeout: Duration)(implicit config: EmbeddedKafkaConfig, valueDeserializer: Deserializer[V]): List[V]

Attributes

Inherited from:
ConsumerOps
def consumeNumberMessagesFromTopics[V](topics: Set[String], number: Int, autoCommit: Boolean, timeout: Duration, resetTimeoutOnEachMessage: Boolean)(implicit config: EmbeddedKafkaConfig, valueDeserializer: Deserializer[V]): Map[String, List[V]]

Consumes the first n messages available in given topics, deserializes them as type V, and returns the n messages in a Map from topic name to List[V].

Consumes the first n messages available in given topics, deserializes them as type V, and returns the n messages in a Map from topic name to List[V].

Only the messages that are returned are committed if autoCommit is false. If autoCommit is true then all messages that were polled will be committed.

Value parameters

autoCommit

if false, only the offset for the consumed messages will be committed. if true, the offset for the last polled message will be committed instead.

config

an implicit EmbeddedKafkaConfig

number

the number of messages to consume in a batch

resetTimeoutOnEachMessage

when true, throw TimeoutException if we have a silent period (no incoming messages) for the timeout interval; when false, throw TimeoutException after the timeout interval if we haven't received all of the expected messages

timeout

the interval to wait for messages before throwing TimeoutException

topics

the topics to consume messages from

valueDeserializer

an implicit Deserializer for the type V

Attributes

Returns

the List of messages consumed from the given topics, each with a type V

Inherited from:
ConsumerOps
def consumeNumberStringMessagesFrom(topic: String, number: Int, autoCommit: Boolean, timeout: Duration)(implicit config: EmbeddedKafkaConfig): List[String]

Attributes

Inherited from:
ConsumerOps
def createCustomTopic(topic: String, topicConfig: Map[String, String], partitions: Int, replicationFactor: Int)(implicit config: EmbeddedKafkaConfig): Try[Unit]

Creates a topic with a custom configuration.

Creates a topic with a custom configuration.

Value parameters

config

an implicit EmbeddedKafkaConfig

partitions

number of partitions

replicationFactor

replication factor

topic

the topic name

topicConfig

per topic configuration Map

Attributes

Inherited from:
AdminOps
def deleteTopics(topics: List[String])(implicit config: EmbeddedKafkaConfig): Try[Unit]

Either deletes or marks for deletion a list of topics.

Either deletes or marks for deletion a list of topics.

Value parameters

config

an implicit EmbeddedKafkaConfig

topics

the topic names

Attributes

Inherited from:
AdminOps
def publishStringMessageToKafka(topic: String, message: String)(implicit config: EmbeddedKafkaConfig): Unit

Publishes synchronously a message of type String to the running Kafka broker.

Publishes synchronously a message of type String to the running Kafka broker.

Value parameters

config

an implicit EmbeddedKafkaConfig

message

the message to publish

topic

the topic to which publish the message (it will be auto-created)

Attributes

Throws
KafkaUnavailableException

if unable to connect to Kafka

Inherited from:
ProducerOps
def publishToKafka[K, T](topic: String, messages: Seq[(K, T)])(implicit config: EmbeddedKafkaConfig, keySerializer: Serializer[K], serializer: Serializer[T]): Unit

Publishes synchronously a batch of message to the running Kafka broker.

Publishes synchronously a batch of message to the running Kafka broker.

Value parameters

config

an implicit EmbeddedKafkaConfig

keySerializer

an implicit Serializer for the type K

messages

the keys and messages of type (K, T) to publish

serializer

an implicit Serializer for the type T

topic

the topic to which publish the message (it will be auto-created)

Attributes

Throws
KafkaUnavailableException

if unable to connect to Kafka

Inherited from:
ProducerOps
def publishToKafka[K, T](topic: String, key: K, message: T)(implicit config: EmbeddedKafkaConfig, keySerializer: Serializer[K], serializer: Serializer[T]): Unit

Publishes synchronously a message to the running Kafka broker.

Publishes synchronously a message to the running Kafka broker.

Value parameters

config

an implicit EmbeddedKafkaConfig

key

the key of type K to publish

message

the message of type T to publish

serializer

an implicit Serializer for the type T

topic

the topic to which publish the message (it will be auto-created)

Attributes

Throws
KafkaUnavailableException

if unable to connect to Kafka

Inherited from:
ProducerOps
def publishToKafka[T](producerRecord: ProducerRecord[String, T])(implicit config: EmbeddedKafkaConfig, serializer: Serializer[T]): Unit

Publishes synchronously a message to the running Kafka broker.

Publishes synchronously a message to the running Kafka broker.

Value parameters

config

an implicit EmbeddedKafkaConfig

producerRecord

the producerRecord of type T to publish

serializer

an implicit Serializer for the type T

Attributes

Throws
KafkaUnavailableException

if unable to connect to Kafka

Inherited from:
ProducerOps
def publishToKafka[T](topic: String, message: T)(implicit config: EmbeddedKafkaConfig, serializer: Serializer[T]): Unit

Publishes synchronously a message to the running Kafka broker.

Publishes synchronously a message to the running Kafka broker.

Value parameters

config

an implicit EmbeddedKafkaConfig

message

the message of type T to publish

serializer

an implicit Serializer for the type T

topic

the topic to which publish the message (it will be auto-created)

Attributes

Throws
KafkaUnavailableException

if unable to connect to Kafka

Inherited from:
ProducerOps
def runStreams[T](topicsToCreate: Seq[String], topology: Topology, extraConfig: Map[String, AnyRef])(block: => T)(implicit config: EmbeddedKafkaConfig): T

Execute Kafka streams and pass a block of code that can operate while the streams are active. The code block can be used for publishing and consuming messages in Kafka.

Execute Kafka streams and pass a block of code that can operate while the streams are active. The code block can be used for publishing and consuming messages in Kafka.

Value parameters

block

the code block that will executed while the streams are active. Once the block has been executed the streams will be closed.

extraConfig

additional Kafka Streams configuration (overwrite existing keys in default config)

topicsToCreate

the topics that should be created in Kafka before launching the streams.

topology

the streams topology that will be used to instantiate the streams with a default configuration (all state directories are different and in temp folders)

Attributes

Inherited from:
EmbeddedKafkaStreamsSupport (hidden)
def runStreamsOnFoundPort[T](config: EmbeddedKafkaConfig)(topicsToCreate: Seq[String], topology: Topology, extraConfig: Map[String, AnyRef])(block: EmbeddedKafkaConfig => T): T

Execute Kafka streams and pass a block of code that can operate while the streams are active. The code block can be used for publishing and consuming messages in Kafka. The actual ports of the servers will be detected and inserted into a copied version of the EmbeddedKafkaConfig that gets passed to body. This is useful if you set any port to 0, which will listen on an arbitrary available port.

Execute Kafka streams and pass a block of code that can operate while the streams are active. The code block can be used for publishing and consuming messages in Kafka. The actual ports of the servers will be detected and inserted into a copied version of the EmbeddedKafkaConfig that gets passed to body. This is useful if you set any port to 0, which will listen on an arbitrary available port.

Value parameters

block

the code block that will executed while the streams are active, given an EmbeddedKafkaConfig with the actual ports the servers are running on. Once the block has been executed the streams will be closed.

config

the user-defined EmbeddedKafkaConfig

extraConfig

additional Kafka Streams configuration (overwrite existing keys in default config)

topicsToCreate

the topics that should be created in Kafka before launching the streams.

topology

the streams topology that will be used to instantiate the streams with a default configuration (all state directories are different and in temp folders)

Attributes

Inherited from:
EmbeddedKafkaStreamsSupport (hidden)
protected def withAdminClient[T](body: AdminClient => T)(implicit config: EmbeddedKafkaConfig): Try[T]

Creates an AdminClient, then executes the body passed as a parameter.

Creates an AdminClient, then executes the body passed as a parameter.

Value parameters

body

the function to execute

config

an implicit EmbeddedKafkaConfig

Attributes

Inherited from:
AdminOps
def withConsumer[K, V, T](body: (KafkaConsumer[K, V]) => T)(implicit config: EmbeddedKafkaConfig, keyDeserializer: Deserializer[K], valueDeserializer: Deserializer[V]): T

Loaner pattern that allows running a code block with a newly created producer. The producer's lifecycle will be automatically handled and closed at the end of the given code block.

Loaner pattern that allows running a code block with a newly created producer. The producer's lifecycle will be automatically handled and closed at the end of the given code block.

Value parameters

body

the function to execute that returns T

config

an implicit EmbeddedKafkaConfig

keyDeserializer

an implicit Deserializer for the type K

valueDeserializer

an implicit Deserializer for the type V

Attributes

Inherited from:
ConsumerOps
def withProducer[K, V, T](body: (KafkaProducer[K, V]) => T)(implicit config: EmbeddedKafkaConfig, keySerializer: Serializer[K], valueSerializer: Serializer[V]): T

Loaner pattern that allows running a code block with a newly created consumer. The consumer's lifecycle will be automatically handled and closed at the end of the given code block.

Loaner pattern that allows running a code block with a newly created consumer. The consumer's lifecycle will be automatically handled and closed at the end of the given code block.

Value parameters

body

the function to execute that returns T

config

an implicit EmbeddedKafkaConfig

keySerializer

an implicit Serializer for the type K

valueSerializer

an implicit Serializer for the type V

Attributes

Inherited from:
ProducerOps
def withRunningKafka[T](body: => T)(implicit config: EmbeddedKafkaConfig): T

Starts a ZooKeeper instance and a Kafka broker (and performs additional logic, if any), then executes the body passed as a parameter.

Starts a ZooKeeper instance and a Kafka broker (and performs additional logic, if any), then executes the body passed as a parameter.

Value parameters

body

the function to execute

config

an implicit EmbeddedKafkaConfig

Attributes

Inherited from:
EmbeddedKafkaSupport (hidden)
def withRunningKafkaOnFoundPort[T](config: EmbeddedKafkaConfig)(body: EmbeddedKafkaConfig => T): T

Starts a ZooKeeper instance and a Kafka broker (and performs additional logic, if any), then executes the body passed as a parameter. The actual ports of the servers will be detected and inserted into a copied version of the EmbeddedKafkaConfig that gets passed to body. This is useful if you set any port to 0, which will listen on an arbitrary available port.

Starts a ZooKeeper instance and a Kafka broker (and performs additional logic, if any), then executes the body passed as a parameter. The actual ports of the servers will be detected and inserted into a copied version of the EmbeddedKafkaConfig that gets passed to body. This is useful if you set any port to 0, which will listen on an arbitrary available port.

Value parameters

body

the function to execute, given an EmbeddedKafkaConfig with the actual ports the servers are running on

config

the user-defined EmbeddedKafkaConfig

Attributes

Inherited from:
EmbeddedKafkaSupport (hidden)

Inherited fields

protected val adminClientCloseTimeout: FiniteDuration

Attributes

Inherited from:
AdminOps
protected val autoCreateTopics: Boolean

Attributes

Inherited from:
KafkaOps
protected val brokerId: Short

Attributes

Inherited from:
KafkaOps
protected val consumerPollingTimeout: FiniteDuration

Attributes

Inherited from:
ConsumerOps
protected val logCleanerDedupeBufferSize: Int

Attributes

Inherited from:
KafkaOps
protected val producerPublishTimeout: FiniteDuration

Attributes

Inherited from:
ProducerOps
protected val topicCreationTimeout: FiniteDuration

Attributes

Inherited from:
AdminOps
protected val topicDeletionTimeout: FiniteDuration

Attributes

Inherited from:
AdminOps
val zkConnectionTimeoutMs: Int

Attributes

Inherited from:
AdminOps
val zkSessionTimeoutMs: Int

Attributes

Inherited from:
AdminOps
Generated with