kafka4m
package kafka4m
The high-level API space for kafka consumers
- Alphabetic
- By Inheritance
- kafka4m
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Package Members
Value Members
- def adminTopic(config: Config): String
- config
the root configuration
- returns
the admin topic as per the config
- def byteArrayProducer[A](config: Config): RichKafkaProducer[Key, Bytes]
- def closeOnComplete(config: Config): Boolean
- config
the kafka4m config
- returns
true if observables should be closed when complete
- def consumerTopics(config: Config): Set[String]
- config
the root configuration
- returns
the consumer topic as per the config
- def continueOnError(config: Config): Boolean
- def ensureTopicBlocking(config: Config)(implicit ec: ExecutionContext): Option[String]
Kafka Streams will fail if the topic does not yet exist.
Kafka Streams will fail if the topic does not yet exist. This way we can provide a means to 'getOrCreate' a topic if that's how it's configured.
- def fireAndForget(config: Config): Boolean
- def kafkaConsumer(config: Config)(implicit scheduler: Scheduler): RichKafkaConsumer[String, Array[Byte]]
- config
the kafka4m config
- returns
A RichKafkaConsumer from the given kafka4m configuration
- def kafkaProducer[A, K, V](config: Config): RichKafkaProducer[K, V]
- config
the kafka4m config
- returns
a RichKafkaProducer for the given config
- def producerTopic(config: Config): String
- config
the root configuration
- returns
the producer topic as per the config
- def read(config: Config = ConfigFactory.load())(implicit scheduler: Scheduler): Observable[AckableRecord[ConsumerRecord[String, Array[Byte]]]]
- config
the kafka4m configuration which contains the 'kafka4m.consumer' values
- returns
an Observable of AckableRecords from kafka. The offsets, etc will be controlled by the kafka4m.consumer configuration, which includes default offset strategy, etc.
- def readRecords(config: Config = ConfigFactory.load())(implicit scheduler: Scheduler): Observable[ConsumerRecord[Key, Bytes]]
- config
the kafka4m configuration which contains the 'kafka4m.consumer' values
- returns
an Observable of data coming from kafka. The offsets, etc will be controlled by the kafka4m.consumer configuration, which includes default offset strategy, etc.
- def richAdmin(config: Config): RichKafkaAdmin
- def write[A](config: Config = ConfigFactory.load())(implicit ev: Aux[A, Key, Bytes]): Consumer[A, Long]
- A
any type A which can be converted into a kafka ProducerRecord
- config
the kafka4m configuration
- returns
a consumer of the 'A' values and produce the number written
- def writeKeyAndBytes(config: Config = ConfigFactory.load()): Consumer[(String, Array[Byte]), Long]
- config
the kafka4m configuration
- returns
a consumer which will consume a stream of key/byte-array values into kafka and return the number written
- def writeText(config: Config = ConfigFactory.load()): Consumer[String, Long]
- config
the kafka4m configuration
- returns
a consumer which will consume raw text data and write it with null keys
- object Kafka4mApp extends ConfigApp with StrictLogging
An ETL entry point to read data into or out of kafka