trait EmbeddedKafka extends EmbeddedKafkaSupport[EmbeddedKafkaConfig] with EmbeddedKafkaOps[EmbeddedKafkaConfig, EmbeddedK]
- Alphabetic
- By Inheritance
- EmbeddedKafka
- EmbeddedKafkaOps
- KafkaOps
- ZooKeeperOps
- ProducerOps
- ConsumerOps
- AdminOps
- EmbeddedKafkaSupport
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##(): Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- val adminClientCloseTimeout: FiniteDuration
- Attributes
- protected
- Definition Classes
- AdminOps
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- val autoCreateTopics: Boolean
- Attributes
- protected
- Definition Classes
- KafkaOps
- val brokerId: Short
- Attributes
- protected
- Definition Classes
- KafkaOps
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- def consumeFirstKeyedMessageFrom[K, V](topic: String, autoCommit: Boolean = false)(implicit config: EmbeddedKafkaConfig, keyDeserializer: Deserializer[K], valueDeserializer: Deserializer[V]): (K, V)
Consumes the first message available in a given topic, deserializing it as type V).
Consumes the first message available in a given topic, deserializing it as type V).
Only the message that is returned is committed if autoCommit is false. If autoCommit is true then all messages that were polled will be committed.
- topic
the topic to consume a message from
- autoCommit
if false, only the offset for the consumed message will be committed. if true, the offset for the last polled message will be committed instead. Defaulted to false.
- config
an implicit EmbeddedKafkaConfig
- keyDeserializer
an implicit org.apache.kafka.common.serialization.Deserializer for the type K
- valueDeserializer
an implicit org.apache.kafka.common.serialization.Deserializer for the type V
- returns
the first message consumed from the given topic, with a type V)
- Definition Classes
- ConsumerOps
- Annotations
- @throws(classOf[TimeoutException]) @throws(classOf[KafkaUnavailableException])
- Exceptions thrown
KafkaUnavailableException
if unable to connect to KafkaTimeoutException
if unable to consume a message within 5 seconds
- def consumeFirstMessageFrom[V](topic: String, autoCommit: Boolean = false)(implicit config: EmbeddedKafkaConfig, valueDeserializer: Deserializer[V]): V
Consumes the first message available in a given topic, deserializing it as type V.
Consumes the first message available in a given topic, deserializing it as type V.
Only the message that is returned is committed if autoCommit is false. If autoCommit is true then all messages that were polled will be committed.
- topic
the topic to consume a message from
- autoCommit
if false, only the offset for the consumed message will be committed. if true, the offset for the last polled message will be committed instead. Defaulted to false.
- config
an implicit EmbeddedKafkaConfig
- valueDeserializer
an implicit org.apache.kafka.common.serialization.Deserializer for the type V
- returns
the first message consumed from the given topic, with a type V
- Definition Classes
- ConsumerOps
- Annotations
- @throws(classOf[TimeoutException]) @throws(classOf[KafkaUnavailableException])
- Exceptions thrown
KafkaUnavailableException
if unable to connect to KafkaTimeoutException
if unable to consume a message within 5 seconds
- def consumeFirstStringMessageFrom(topic: String, autoCommit: Boolean = false)(implicit config: EmbeddedKafkaConfig): String
- Definition Classes
- ConsumerOps
- def consumeNumberKeyedMessagesFrom[K, V](topic: String, number: Int, autoCommit: Boolean = false)(implicit config: EmbeddedKafkaConfig, keyDeserializer: Deserializer[K], valueDeserializer: Deserializer[V]): List[(K, V)]
- Definition Classes
- ConsumerOps
- def consumeNumberKeyedMessagesFromTopics[K, V](topics: Set[String], number: Int, autoCommit: Boolean = false, timeout: Duration = 5.seconds, resetTimeoutOnEachMessage: Boolean = true)(implicit config: EmbeddedKafkaConfig, keyDeserializer: Deserializer[K], valueDeserializer: Deserializer[V]): Map[String, List[(K, V)]]
Consumes the first n messages available in given topics, deserializes them as type V), and returns the n messages in a Map from topic name to List[(K, V)].
Consumes the first n messages available in given topics, deserializes them as type V), and returns the n messages in a Map from topic name to List[(K, V)].
Only the messages that are returned are committed if autoCommit is false. If autoCommit is true then all messages that were polled will be committed.
- topics
the topics to consume messages from
- number
the number of messages to consume in a batch
- autoCommit
if false, only the offset for the consumed messages will be committed. if true, the offset for the last polled message will be committed instead. Defaulted to false.
- timeout
the interval to wait for messages before throwing TimeoutException
- resetTimeoutOnEachMessage
when true, throw TimeoutException if we have a silent period (no incoming messages) for the timeout interval; when false, throw TimeoutException after the timeout interval if we haven't received all of the expected messages
- config
an implicit EmbeddedKafkaConfig
- keyDeserializer
an implicit org.apache.kafka.common.serialization.Deserializer for the type K
- valueDeserializer
an implicit org.apache.kafka.common.serialization.Deserializer for the type V
- returns
the List of messages consumed from the given topics, each with a type V)
- Definition Classes
- ConsumerOps
- Exceptions thrown
KafkaUnavailableException
if unable to connect to KafkaTimeoutException
if unable to consume messages within specified timeout
- def consumeNumberMessagesFrom[V](topic: String, number: Int, autoCommit: Boolean = false)(implicit config: EmbeddedKafkaConfig, valueDeserializer: Deserializer[V]): List[V]
- Definition Classes
- ConsumerOps
- def consumeNumberMessagesFromTopics[V](topics: Set[String], number: Int, autoCommit: Boolean = false, timeout: Duration = 5.seconds, resetTimeoutOnEachMessage: Boolean = true)(implicit config: EmbeddedKafkaConfig, valueDeserializer: Deserializer[V]): Map[String, List[V]]
Consumes the first n messages available in given topics, deserializes them as type V, and returns the n messages in a Map from topic name to List[V].
Consumes the first n messages available in given topics, deserializes them as type V, and returns the n messages in a Map from topic name to List[V].
Only the messages that are returned are committed if autoCommit is false. If autoCommit is true then all messages that were polled will be committed.
- topics
the topics to consume messages from
- number
the number of messages to consume in a batch
- autoCommit
if false, only the offset for the consumed messages will be committed. if true, the offset for the last polled message will be committed instead. Defaulted to false.
- timeout
the interval to wait for messages before throwing TimeoutException
- resetTimeoutOnEachMessage
when true, throw TimeoutException if we have a silent period (no incoming messages) for the timeout interval; when false, throw TimeoutException after the timeout interval if we haven't received all of the expected messages
- config
an implicit EmbeddedKafkaConfig
- valueDeserializer
an implicit org.apache.kafka.common.serialization.Deserializer for the type V
- returns
the List of messages consumed from the given topics, each with a type V
- Definition Classes
- ConsumerOps
- Exceptions thrown
KafkaUnavailableException
if unable to connect to KafkaTimeoutException
if unable to consume messages within specified timeout
- def consumeNumberStringMessagesFrom(topic: String, number: Int, autoCommit: Boolean = false)(implicit config: EmbeddedKafkaConfig): List[String]
- Definition Classes
- ConsumerOps
- val consumerPollingTimeout: FiniteDuration
- Attributes
- protected
- Definition Classes
- ConsumerOps
- def createCustomTopic(topic: String, topicConfig: Map[String, String] = Map.empty, partitions: Int = 1, replicationFactor: Int = 1)(implicit config: EmbeddedKafkaConfig): Unit
Creates a topic with a custom configuration.
Creates a topic with a custom configuration.
- topic
the topic name
- topicConfig
per topic configuration Map
- partitions
number of partitions Int
- replicationFactor
replication factor Int
- config
an implicit EmbeddedKafkaConfig
- Definition Classes
- AdminOps
- def deleteTopics(topics: List[String])(implicit config: EmbeddedKafkaConfig): Try[Unit]
Either deletes or marks for deletion a list of topics.
Either deletes or marks for deletion a list of topics.
- topics
the topic names
- config
an implicit EmbeddedKafkaConfig
- Definition Classes
- AdminOps
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- val logCleanerDedupeBufferSize: Int
- Attributes
- protected
- Definition Classes
- KafkaOps
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- val producerPublishTimeout: FiniteDuration
- Attributes
- protected
- Definition Classes
- ProducerOps
- def publishStringMessageToKafka(topic: String, message: String)(implicit config: EmbeddedKafkaConfig): Unit
Publishes synchronously a message of type String to the running Kafka broker.
Publishes synchronously a message of type String to the running Kafka broker.
- topic
the topic to which publish the message (it will be auto-created)
- message
the String message to publish
- config
an implicit EmbeddedKafkaConfig
- Definition Classes
- ProducerOps
- Exceptions thrown
KafkaUnavailableException
if unable to connect to Kafka- See also
- def publishToKafka[K, T](topic: String, messages: Seq[(K, T)])(implicit config: EmbeddedKafkaConfig, keySerializer: Serializer[K], serializer: Serializer[T]): Unit
Publishes synchronously a batch of message to the running Kafka broker.
Publishes synchronously a batch of message to the running Kafka broker.
- topic
the topic to which publish the message (it will be auto-created)
- messages
the keys and messages of type T) to publish
- config
an implicit EmbeddedKafkaConfig
- keySerializer
an implicit Serializer for the type K
- serializer
an implicit Serializer for the type T
- Definition Classes
- ProducerOps
- Annotations
- @throws(classOf[KafkaUnavailableException])
- Exceptions thrown
KafkaUnavailableException
if unable to connect to Kafka
- def publishToKafka[K, T](topic: String, key: K, message: T)(implicit config: EmbeddedKafkaConfig, keySerializer: Serializer[K], serializer: Serializer[T]): Unit
Publishes synchronously a message to the running Kafka broker.
Publishes synchronously a message to the running Kafka broker.
- topic
the topic to which publish the message (it will be auto-created)
- key
the key of type K to publish
- message
the message of type T to publish
- config
an implicit EmbeddedKafkaConfig
- serializer
an implicit Serializer for the type T
- Definition Classes
- ProducerOps
- Annotations
- @throws(classOf[KafkaUnavailableException])
- Exceptions thrown
KafkaUnavailableException
if unable to connect to Kafka
- def publishToKafka[T](producerRecord: ProducerRecord[String, T])(implicit config: EmbeddedKafkaConfig, serializer: Serializer[T]): Unit
Publishes synchronously a message to the running Kafka broker.
Publishes synchronously a message to the running Kafka broker.
- producerRecord
the producerRecord of type T to publish
- config
an implicit EmbeddedKafkaConfig
- serializer
an implicit Serializer for the type T
- Definition Classes
- ProducerOps
- Annotations
- @throws(classOf[KafkaUnavailableException])
- Exceptions thrown
KafkaUnavailableException
if unable to connect to Kafka
- def publishToKafka[T](topic: String, message: T)(implicit config: EmbeddedKafkaConfig, serializer: Serializer[T]): Unit
Publishes synchronously a message to the running Kafka broker.
Publishes synchronously a message to the running Kafka broker.
- topic
the topic to which publish the message (it will be auto-created)
- message
the message of type T to publish
- config
an implicit EmbeddedKafkaConfig
- serializer
an implicit Serializer for the type T
- Definition Classes
- ProducerOps
- Annotations
- @throws(classOf[KafkaUnavailableException])
- Exceptions thrown
KafkaUnavailableException
if unable to connect to Kafka
- def startKafka(config: EmbeddedKafkaConfig, kafkaLogDir: Path): KafkaServer
- Definition Classes
- KafkaOps
- def startZooKeeper(zooKeeperPort: Int, zkLogsDir: Path): ServerCnxnFactory
- Definition Classes
- ZooKeeperOps
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- val topicCreationTimeout: FiniteDuration
- Attributes
- protected
- Definition Classes
- AdminOps
- val topicDeletionTimeout: FiniteDuration
- Attributes
- protected
- Definition Classes
- AdminOps
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- def withAdminClient[T](body: (AdminClient) => T)(implicit config: EmbeddedKafkaConfig): Try[T]
Creates an AdminClient, then executes the body passed as a parameter.
Creates an AdminClient, then executes the body passed as a parameter.
- body
the function to execute
- config
an implicit EmbeddedKafkaConfig
- Attributes
- protected
- Definition Classes
- AdminOps
- def withConsumer[K, V, T](body: (KafkaConsumer[K, V]) => T)(implicit config: EmbeddedKafkaConfig, keyDeserializer: Deserializer[K], valueDeserializer: Deserializer[V]): T
Loaner pattern that allows running a code block with a newly created producer.
Loaner pattern that allows running a code block with a newly created producer. The producer's lifecycle will be automatically handled and closed at the end of the given code block.
- body
the function to execute that returns T
- config
an implicit EmbeddedKafkaConfig
- keyDeserializer
an implicit Deserializer for the type K
- valueDeserializer
an implicit Deserializer for the type V
- Definition Classes
- ConsumerOps
- def withProducer[K, V, T](body: (KafkaProducer[K, V]) => T)(implicit config: EmbeddedKafkaConfig, keySerializer: Serializer[K], valueSerializer: Serializer[V]): T
Loaner pattern that allows running a code block with a newly created consumer.
Loaner pattern that allows running a code block with a newly created consumer. The consumer's lifecycle will be automatically handled and closed at the end of the given code block.
- body
the function to execute that returns T
- config
an implicit EmbeddedKafkaConfig
- keySerializer
an implicit Serializer for the type K
- valueSerializer
an implicit Serializer for the type V
- Definition Classes
- ProducerOps
- def withRunningKafka[T](body: => T)(implicit config: EmbeddedKafkaConfig): T
Starts a ZooKeeper instance and a Kafka broker (and performs additional logic, if any), then executes the body passed as a parameter.
Starts a ZooKeeper instance and a Kafka broker (and performs additional logic, if any), then executes the body passed as a parameter.
- body
the function to execute
- config
an implicit EmbeddedKafkaConfig
- Definition Classes
- EmbeddedKafkaSupport
- def withRunningKafkaOnFoundPort[T](config: EmbeddedKafkaConfig)(body: (EmbeddedKafkaConfig) => T): T
Starts a ZooKeeper instance and a Kafka broker (and performs additional logic, if any), then executes the body passed as a parameter.
Starts a ZooKeeper instance and a Kafka broker (and performs additional logic, if any), then executes the body passed as a parameter. The actual ports of the servers will be detected and inserted into a copied version of the EmbeddedKafkaConfig that gets passed to body. This is useful if you set any port to 0, which will listen on an arbitrary available port.
- config
the user-defined EmbeddedKafkaConfig
- body
the function to execute, given an EmbeddedKafkaConfig with the actual ports the servers are running on
- Definition Classes
- EmbeddedKafkaSupport
- val zkConnectionTimeout: FiniteDuration
- Attributes
- protected
- Definition Classes
- KafkaOps
- val zkConnectionTimeoutMs: Int
- Definition Classes
- AdminOps
- val zkSessionTimeoutMs: Int
- Definition Classes
- AdminOps