kafka

Type Members

1. trait KafkaKeyGen[K] extends AnyRef

   

   An instance of KafkaKeyGen is used to generate a key from the row to be used by the KafkaSink when it produces a kafka message.

   An instance of KafkaKeyGen is used to generate a key from the row to be used by the KafkaSink when it produces a kafka message.

   If the key to be used is an Int or a String then a default HashCodeKeyGen is provided which will use the row objects hashcode if no more specific KafkaKeyGen is provided.

   K

   the key type used by the KafkaProducer passed into the KafkaSink.

2. trait KafkaPartitioner[+V] extends AnyRef

   

   A KafkaPartitioner is used to generate an optional partition key for each final value.

   A KafkaPartitioner is used to generate an optional partition key for each final value. If the KafkaPartitioner returns None then no partition is used when the KafkaSink proicuces a kafka message.

   By default a no-op is supplied which will be used if another more specific partition is not provided.

3. trait KafkaRowConverter[V] extends AnyRef

   

   An instance of KafkaRowConverter is used to convert an eel-sdk Row into another type that is compatible with the KafkaProducer used in the KafkaSink.

   An instance of KafkaRowConverter is used to convert an eel-sdk Row into another type that is compatible with the KafkaProducer used in the KafkaSink.

   Note that the KafkaSink can write to a KafkaProducer[K, Row] in which case the KafkaRowConverter can be a no-op, which will be supplied automatically.

   V

   the final type that will be written to the KafkaProducer

4. case class KafkaSink[K, V](topic: String, producer: KafkaProducer[K, V])(implicit partitioner: KafkaPartitioner[V], converter: KafkaRowConverter[V], keygen: KafkaKeyGen[K]) extends Sink with Logging with Product with Serializable