fs2-kafka/fs2.kafka/KafkaProducer

KafkaProducer

abstract class KafkaProducer[F[_], K, V]

KafkaProducer represents a producer of Kafka records, with the ability to produce ProducerRecords using produce. Records are wrapped in ProducerRecords which allow an arbitrary value, that is a passthrough, to be included in the result. Most often this is used for keeping the CommittableOffsets, in order to commit offsets, but any value can be used as passthrough value.

KafkaProducer represents a producer of Kafka records, with the ability to produce ProducerRecords using produce. Records are wrapped in ProducerRecords which allow an arbitrary value, that is a passthrough, to be included in the result. Most often this is used for keeping the CommittableOffsets, in order to commit offsets, but any value can be used as passthrough value.

Companion
object
class Object
trait Matchable
class Any
class Metrics[F, K, V]

Value members

Abstract methods

def produce[P](records: ProducerRecords[K, V, P]): F[F[ProducerResult[K, V, P]]]

Produces the specified ProducerRecords in two steps: the first effect puts the records in the buffer of the producer, and the second effect waits for the records to send.

Produces the specified ProducerRecords in two steps: the first effect puts the records in the buffer of the producer, and the second effect waits for the records to send.

It's possible to flatten the result from this function to have an effect which both sends the records and waits for them to finish sending.

Waiting for individual records to send can substantially limit performance. In some cases, this is necessary, and so we might want to consider the following alternatives.

  • Wait for the produced records in batches, improving the rate at which records are produced, but loosing the guarantee where produce >> otherAction means otherAction executes after the record has been sent.
  • Run several produce.flatten >> otherAction concurrently, improving the rate at which records are produced, and still have otherAction execute after records have been sent, but losing the order of produced records.