EmbeddedKafka

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
object aKafkaProducer

Definition Classes
ProducerOps
final def asInstanceOf[T0]: T0

Definition Classes
Any
val autoCreateTopics: Boolean

Attributes
protected
Definition Classes
KafkaOps
val brokerId: Short

Attributes
protected
Definition Classes
KafkaOps
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
def consumeFirstKeyedMessageFrom[K, V](topic: String, autoCommit: Boolean = false)(implicit config: EmbeddedKafkaConfig, keyDeserializer: Deserializer[K], valueDeserializer: Deserializer[V]): (K, V)

Consumes the first message available in a given topic, deserializing it as type V).
Consumes the first message available in a given topic, deserializing it as type V).
Only the message that is returned is committed if autoCommit is false. If autoCommit is true then all messages that were polled will be committed.
topic
the topic to consume a message from
autoCommit
if false, only the offset for the consumed message will be committed. if true, the offset for the last polled message will be committed instead. Defaulted to false.
config
an implicit EmbeddedKafkaConfig
keyDeserializer
an implicit org.apache.kafka.common.serialization.Deserializer for the type K
valueDeserializer
an implicit org.apache.kafka.common.serialization.Deserializer for the type V
returns
the first message consumed from the given topic, with a type V)

Definition Classes
ConsumerOps
Annotations
@throws( classOf[TimeoutException] ) @throws( classOf[KafkaUnavailableException] )
Exceptions thrown
KafkaUnavailableException if unable to connect to Kafka
TimeoutException if unable to consume a message within 5 seconds
def consumeFirstMessageFrom[V](topic: String, autoCommit: Boolean = false)(implicit config: EmbeddedKafkaConfig, valueDeserializer: Deserializer[V]): V

Consumes the first message available in a given topic, deserializing it as type V.
Consumes the first message available in a given topic, deserializing it as type V.
Only the message that is returned is committed if autoCommit is false. If autoCommit is true then all messages that were polled will be committed.
topic
the topic to consume a message from
autoCommit
if false, only the offset for the consumed message will be committed. if true, the offset for the last polled message will be committed instead. Defaulted to false.
config
an implicit EmbeddedKafkaConfig
valueDeserializer
an implicit org.apache.kafka.common.serialization.Deserializer for the type V
returns
the first message consumed from the given topic, with a type V

Definition Classes
ConsumerOps
Annotations
@throws( classOf[TimeoutException] ) @throws( classOf[KafkaUnavailableException] )
Exceptions thrown
KafkaUnavailableException if unable to connect to Kafka
TimeoutException if unable to consume a message within 5 seconds
def consumeFirstStringMessageFrom(topic: String, autoCommit: Boolean = false)(implicit config: EmbeddedKafkaConfig): String

Definition Classes
ConsumerOps
def consumeNumberKeyedMessagesFrom[K, V](topic: String, number: Int, autoCommit: Boolean = false)(implicit config: EmbeddedKafkaConfig, keyDeserializer: Deserializer[K], valueDeserializer: Deserializer[V]): List[(K, V)]

Definition Classes
ConsumerOps
def consumeNumberKeyedMessagesFromTopics[K, V](topics: Set[String], number: Int, autoCommit: Boolean = false, timeout: Duration = 5.seconds, resetTimeoutOnEachMessage: Boolean = true)(implicit config: EmbeddedKafkaConfig, keyDeserializer: Deserializer[K], valueDeserializer: Deserializer[V]): Map[String, List[(K, V)]]

Consumes the first n messages available in given topics, deserializes them as type V), and returns the n messages in a Map from topic name to List[(K, V)].
Consumes the first n messages available in given topics, deserializes them as type V), and returns the n messages in a Map from topic name to List[(K, V)].
Only the messages that are returned are committed if autoCommit is false. If autoCommit is true then all messages that were polled will be committed.
topics
the topics to consume messages from
number
the number of messages to consume in a batch
autoCommit
if false, only the offset for the consumed messages will be committed. if true, the offset for the last polled message will be committed instead. Defaulted to false.
timeout
the interval to wait for messages before throwing TimeoutException
resetTimeoutOnEachMessage
when true, throw TimeoutException if we have a silent period (no incoming messages) for the timeout interval; when false, throw TimeoutException after the timeout interval if we haven't received all of the expected messages
config
an implicit EmbeddedKafkaConfig
keyDeserializer
an implicit org.apache.kafka.common.serialization.Deserializer for the type K
valueDeserializer
an implicit org.apache.kafka.common.serialization.Deserializer for the type V
returns
the List of messages consumed from the given topics, each with a type V)

Definition Classes
ConsumerOps
Exceptions thrown
KafkaUnavailableException if unable to connect to Kafka
TimeoutException if unable to consume messages within specified timeout
def consumeNumberMessagesFrom[V](topic: String, number: Int, autoCommit: Boolean = false)(implicit config: EmbeddedKafkaConfig, valueDeserializer: Deserializer[V]): List[V]

Definition Classes
ConsumerOps
def consumeNumberMessagesFromTopics[V](topics: Set[String], number: Int, autoCommit: Boolean = false, timeout: Duration = 5.seconds, resetTimeoutOnEachMessage: Boolean = true)(implicit config: EmbeddedKafkaConfig, valueDeserializer: Deserializer[V]): Map[String, List[V]]

Consumes the first n messages available in given topics, deserializes them as type V, and returns the n messages in a Map from topic name to List[V].
Consumes the first n messages available in given topics, deserializes them as type V, and returns the n messages in a Map from topic name to List[V].
Only the messages that are returned are committed if autoCommit is false. If autoCommit is true then all messages that were polled will be committed.
topics
the topics to consume messages from
number
the number of messages to consume in a batch
autoCommit
if false, only the offset for the consumed messages will be committed. if true, the offset for the last polled message will be committed instead. Defaulted to false.
timeout
the interval to wait for messages before throwing TimeoutException
resetTimeoutOnEachMessage
when true, throw TimeoutException if we have a silent period (no incoming messages) for the timeout interval; when false, throw TimeoutException after the timeout interval if we haven't received all of the expected messages
config
an implicit EmbeddedKafkaConfig
valueDeserializer
an implicit org.apache.kafka.common.serialization.Deserializer for the type V
returns
the List of messages consumed from the given topics, each with a type V

Definition Classes
ConsumerOps
Exceptions thrown
KafkaUnavailableException if unable to connect to Kafka
TimeoutException if unable to consume messages within specified timeout
def consumeNumberStringMessagesFrom(topic: String, number: Int, autoCommit: Boolean = false)(implicit config: EmbeddedKafkaConfig): List[String]

Definition Classes
ConsumerOps
val consumerPollingTimeout: FiniteDuration

Attributes
protected
Definition Classes
ConsumerOps
def createCustomTopic(topic: String, topicConfig: Map[String, String] = Map.empty, partitions: Int = 1, replicationFactor: Int = 1)(implicit config: EmbeddedKafkaConfig): Unit

Creates a topic with a custom configuration.
Creates a topic with a custom configuration.
topic
the topic name
topicConfig
per topic configuration Map
partitions
number of partitions Int
replicationFactor
replication factor Int
config
an implicit EmbeddedKafkaConfig

Definition Classes
AdminOps
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
def hashCode(): Int

Definition Classes
AnyRef → Any
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
def isRunning: Boolean

Returns whether the in memory servers are running.
Returns whether the in memory servers are running.

Definition Classes
EmbeddedKafka → RunningServersOps
def kafkaConsumer[K, T](implicit config: EmbeddedKafkaConfig, keyDeserializer: Deserializer[K], deserializer: Deserializer[T]): KafkaConsumer[K, T]

Definition Classes
ConsumerOps
def kafkaProducer[K, T](topic: String, key: K, message: T)(implicit config: EmbeddedKafkaConfig, keySerializer: Serializer[K], serializer: Serializer[T]): KafkaProducer[K, T]

Definition Classes
ProducerOps
val logCleanerDedupeBufferSize: Int

Attributes
protected
Definition Classes
KafkaOps
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
val producerPublishTimeout: FiniteDuration

Attributes
protected
Definition Classes
ProducerOps
def publishStringMessageToKafka(topic: String, message: String)(implicit config: EmbeddedKafkaConfig): Unit

Publishes synchronously a message of type String to the running Kafka broker.
Publishes synchronously a message of type String to the running Kafka broker.
topic
the topic to which publish the message (it will be auto-created)
message
the String message to publish
config
an implicit EmbeddedKafkaConfig

Definition Classes
ProducerOps
Exceptions thrown
KafkaUnavailableException if unable to connect to Kafka
See also
EmbeddedKafka#publishToKafka
def publishToKafka[K, T](topic: String, messages: Seq[(K, T)])(implicit config: EmbeddedKafkaConfig, keySerializer: Serializer[K], serializer: Serializer[T]): Unit

Publishes synchronously a batch of message to the running Kafka broker.
Publishes synchronously a batch of message to the running Kafka broker.
topic
the topic to which publish the message (it will be auto-created)
messages
the keys and messages of type T) to publish
config
an implicit EmbeddedKafkaConfig
keySerializer
an implicit Serializer for the type K
serializer
an implicit Serializer for the type T

Definition Classes
ProducerOps
Annotations
@throws( classOf[KafkaUnavailableException] )
Exceptions thrown
KafkaUnavailableException if unable to connect to Kafka
def publishToKafka[K, T](topic: String, key: K, message: T)(implicit config: EmbeddedKafkaConfig, keySerializer: Serializer[K], serializer: Serializer[T]): Unit

Publishes synchronously a message to the running Kafka broker.
Publishes synchronously a message to the running Kafka broker.
topic
the topic to which publish the message (it will be auto-created)
key
the key of type K to publish
message
the message of type T to publish
config
an implicit EmbeddedKafkaConfig
serializer
an implicit Serializer for the type T

Definition Classes
ProducerOps
Annotations
@throws( classOf[KafkaUnavailableException] )
Exceptions thrown
KafkaUnavailableException if unable to connect to Kafka
def publishToKafka[T](producerRecord: ProducerRecord[String, T])(implicit config: EmbeddedKafkaConfig, serializer: Serializer[T]): Unit

Publishes synchronously a message to the running Kafka broker.
Publishes synchronously a message to the running Kafka broker.
producerRecord
the producerRecord of type T to publish
config
an implicit EmbeddedKafkaConfig
serializer
an implicit Serializer for the type T

Definition Classes
ProducerOps
Annotations
@throws( classOf[KafkaUnavailableException] )
Exceptions thrown
KafkaUnavailableException if unable to connect to Kafka
def publishToKafka[T](topic: String, message: T)(implicit config: EmbeddedKafkaConfig, serializer: Serializer[T]): Unit

Publishes synchronously a message to the running Kafka broker.
Publishes synchronously a message to the running Kafka broker.
topic
the topic to which publish the message (it will be auto-created)
message
the message of type T to publish
config
an implicit EmbeddedKafkaConfig
serializer
an implicit Serializer for the type T

Definition Classes
ProducerOps
Annotations
@throws( classOf[KafkaUnavailableException] )
Exceptions thrown
KafkaUnavailableException if unable to connect to Kafka
def start()(implicit config: EmbeddedKafkaConfig): EmbeddedK

Starts in memory servers, using temporary directories for storing logs.
Starts in memory servers, using temporary directories for storing logs. The log directories will be cleaned after calling the EmbeddedServer.stop() method or on JVM exit, whichever happens earlier.
config
an implicit EmbeddedKafkaConfig

Definition Classes
EmbeddedKafka → ServerStarter
def startKafka(kafkaLogsDir: Directory, factory: Option[EmbeddedZ] = None)(implicit config: EmbeddedKafkaConfig): EmbeddedK

Starts a Kafka broker in memory, storing logs in a specific location.
Starts a Kafka broker in memory, storing logs in a specific location.
kafkaLogsDir
the path for the Kafka logs
factory
an EmbeddedZ server
config
an implicit EmbeddedKafkaConfig
returns
an EmbeddedK server

Definition Classes
RunningKafkaOps
def startKafka(config: EmbeddedKafkaConfig, kafkaLogDir: Directory): KafkaServer

Definition Classes
KafkaOps
def startZooKeeper(zkLogsDir: Directory)(implicit config: EmbeddedKafkaConfig): EmbeddedZ

Starts a Zookeeper instance in memory, storing logs in a specific location.
Starts a Zookeeper instance in memory, storing logs in a specific location.
zkLogsDir
the path for the Zookeeper logs
config
an implicit EmbeddedKafkaConfig
returns
an EmbeddedZ server

Definition Classes
RunningZooKeeperOps
def startZooKeeper(zooKeeperPort: Int, zkLogsDir: Directory): ServerCnxnFactory

Definition Classes
ZooKeeperOps
def stop(server: EmbeddedServer): Unit

Stops a specific EmbeddedServer instance, and deletes the log directory.
Stops a specific EmbeddedServer instance, and deletes the log directory.
server
the EmbeddedServer to be stopped.

Definition Classes
RunningServersOps
def stop(): Unit

Stops all in memory servers and deletes the log directories.
Stops all in memory servers and deletes the log directories.

Definition Classes
RunningServersOps
def stopKafka(): Unit

Stops all in memory Kafka instances, preserving the logs directories.
Stops all in memory Kafka instances, preserving the logs directories.

Definition Classes
RunningKafkaOps
def stopZooKeeper(): Unit

Stops all in memory Zookeeper instances, preserving the logs directories.
Stops all in memory Zookeeper instances, preserving the logs directories.

Definition Classes
RunningZooKeeperOps
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toString(): String

Definition Classes
AnyRef → Any
val topicCreationTimeout: FiniteDuration

Attributes
protected
Definition Classes
AdminOps
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
def withRunningKafka[T](body: ⇒ T)(implicit config: EmbeddedKafkaConfig): T

Starts a ZooKeeper instance and a Kafka broker (and performs additional logic, if any), then executes the body passed as a parameter.
Starts a ZooKeeper instance and a Kafka broker (and performs additional logic, if any), then executes the body passed as a parameter.
body
the function to execute
config
an implicit EmbeddedKafkaConfig

Definition Classes
EmbeddedKafkaSupport
def withRunningKafkaOnFoundPort[T](config: EmbeddedKafkaConfig)(body: (EmbeddedKafkaConfig) ⇒ T): T

Starts a ZooKeeper instance and a Kafka broker (and performs additional logic, if any), then executes the body passed as a parameter.
Starts a ZooKeeper instance and a Kafka broker (and performs additional logic, if any), then executes the body passed as a parameter. The actual ports of the servers will be detected and inserted into a copied version of the EmbeddedKafkaConfig that gets passed to body. This is useful if you set any port to 0, which will listen on an arbitrary available port.
config
the user-defined EmbeddedKafkaConfig
body
the function to execute, given an EmbeddedKafkaConfig with the actual ports the servers are running on

Definition Classes
EmbeddedKafkaSupport
val zkConnectionTimeout: FiniteDuration

Attributes
protected
Definition Classes
KafkaOps
val zkConnectionTimeoutMs: Int

Definition Classes
AdminOps
val zkSessionTimeoutMs: Int

Definition Classes
AdminOps

Related Docs: trait EmbeddedKafka | package embeddedkafka

object EmbeddedKafka extends EmbeddedKafka with RunningEmbeddedKafkaOps[EmbeddedKafkaConfig, EmbeddedK]

Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

object aKafkaProducer

final def asInstanceOf[T0]: T0

val autoCreateTopics: Boolean

val brokerId: Short

def clone(): AnyRef

def consumeFirstKeyedMessageFrom[K, V](topic: String, autoCommit: Boolean = false)(implicit config: EmbeddedKafkaConfig, keyDeserializer: Deserializer[K], valueDeserializer: Deserializer[V]): (K, V)

def consumeFirstMessageFrom[V](topic: String, autoCommit: Boolean = false)(implicit config: EmbeddedKafkaConfig, valueDeserializer: Deserializer[V]): V

def consumeFirstStringMessageFrom(topic: String, autoCommit: Boolean = false)(implicit config: EmbeddedKafkaConfig): String

def consumeNumberKeyedMessagesFrom[K, V](topic: String, number: Int, autoCommit: Boolean = false)(implicit config: EmbeddedKafkaConfig, keyDeserializer: Deserializer[K], valueDeserializer: Deserializer[V]): List[(K, V)]

def consumeNumberMessagesFrom[V](topic: String, number: Int, autoCommit: Boolean = false)(implicit config: EmbeddedKafkaConfig, valueDeserializer: Deserializer[V]): List[V]

def consumeNumberMessagesFromTopics[V](topics: Set[String], number: Int, autoCommit: Boolean = false, timeout: Duration = 5.seconds, resetTimeoutOnEachMessage: Boolean = true)(implicit config: EmbeddedKafkaConfig, valueDeserializer: Deserializer[V]): Map[String, List[V]]

def consumeNumberStringMessagesFrom(topic: String, number: Int, autoCommit: Boolean = false)(implicit config: EmbeddedKafkaConfig): List[String]

val consumerPollingTimeout: FiniteDuration

def createCustomTopic(topic: String, topicConfig: Map[String, String] = Map.empty, partitions: Int = 1, replicationFactor: Int = 1)(implicit config: EmbeddedKafkaConfig): Unit

final def eq(arg0: AnyRef): Boolean

def equals(arg0: Any): Boolean

def finalize(): Unit

final def getClass(): Class[_]

def hashCode(): Int

final def isInstanceOf[T0]: Boolean

def isRunning: Boolean

def kafkaConsumer[K, T](implicit config: EmbeddedKafkaConfig, keyDeserializer: Deserializer[K], deserializer: Deserializer[T]): KafkaConsumer[K, T]

def kafkaProducer[K, T](topic: String, key: K, message: T)(implicit config: EmbeddedKafkaConfig, keySerializer: Serializer[K], serializer: Serializer[T]): KafkaProducer[K, T]

val logCleanerDedupeBufferSize: Int

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

val producerPublishTimeout: FiniteDuration

def publishStringMessageToKafka(topic: String, message: String)(implicit config: EmbeddedKafkaConfig): Unit

def publishToKafka[K, T](topic: String, messages: Seq[(K, T)])(implicit config: EmbeddedKafkaConfig, keySerializer: Serializer[K], serializer: Serializer[T]): Unit

def publishToKafka[K, T](topic: String, key: K, message: T)(implicit config: EmbeddedKafkaConfig, keySerializer: Serializer[K], serializer: Serializer[T]): Unit

def publishToKafka[T](producerRecord: ProducerRecord[String, T])(implicit config: EmbeddedKafkaConfig, serializer: Serializer[T]): Unit

def publishToKafka[T](topic: String, message: T)(implicit config: EmbeddedKafkaConfig, serializer: Serializer[T]): Unit

def start()(implicit config: EmbeddedKafkaConfig): EmbeddedK

def startKafka(kafkaLogsDir: Directory, factory: Option[EmbeddedZ] = None)(implicit config: EmbeddedKafkaConfig): EmbeddedK

def startKafka(config: EmbeddedKafkaConfig, kafkaLogDir: Directory): KafkaServer

def startZooKeeper(zkLogsDir: Directory)(implicit config: EmbeddedKafkaConfig): EmbeddedZ

def startZooKeeper(zooKeeperPort: Int, zkLogsDir: Directory): ServerCnxnFactory

def stop(server: EmbeddedServer): Unit

def stop(): Unit

def stopKafka(): Unit

def stopZooKeeper(): Unit

final def synchronized[T0](arg0: ⇒ T0): T0

def toString(): String

val topicCreationTimeout: FiniteDuration

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

def withRunningKafka[T](body: ⇒ T)(implicit config: EmbeddedKafkaConfig): T

def withRunningKafkaOnFoundPort[T](config: EmbeddedKafkaConfig)(body: (EmbeddedKafkaConfig) ⇒ T): T

val zkConnectionTimeout: FiniteDuration

val zkConnectionTimeoutMs: Int

val zkSessionTimeoutMs: Int

Inherited from RunningEmbeddedKafkaOps[EmbeddedKafkaConfig, EmbeddedK]

Inherited from RunningKafkaOps

Inherited from RunningZooKeeperOps

Inherited from ServerStarter[EmbeddedKafkaConfig, EmbeddedK]

Inherited from RunningServersOps

Inherited from EmbeddedKafka

Inherited from EmbeddedKafkaOps[EmbeddedKafkaConfig, EmbeddedK]

Inherited from KafkaOps

Inherited from ZooKeeperOps

Inherited from ProducerOps[EmbeddedKafkaConfig]

Inherited from ConsumerOps[EmbeddedKafkaConfig]

Inherited from AdminOps[EmbeddedKafkaConfig]

Inherited from EmbeddedKafkaSupport[EmbeddedKafkaConfig]

Inherited from AnyRef

Inherited from Any

Ungrouped