Type Members

1.  case class KafkaContinuousInputPartition(topicPartition: TopicPartition, startOffset: Long, kafkaParams: Map[String, AnyRef], pollTimeoutMs: Long, failOnDataLoss: Boolean, includeHeaders: Boolean) extends InputPartition with Product with Serializable

   An input partition for continuous Kafka processing.

   An input partition for continuous Kafka processing. This will be serialized and transformed into a full reader on executors.

   topicPartition

   The (topic, partition) pair this task is responsible for.

   startOffset

   The offset to start reading from within the partition.

   kafkaParams

   Kafka consumer params to use.

   pollTimeoutMs

   The timeout for Kafka consumer polling.

   failOnDataLoss

   Flag indicating whether data reader should fail if some offsets are skipped.

   includeHeaders

   Flag indicating whether to include Kafka records' headers.

2.  class KafkaContinuousPartitionReader extends ContinuousPartitionReader[InternalRow]

   A per-task data reader for continuous Kafka processing.

3.  class KafkaContinuousStream extends ContinuousStream with Logging

   A ContinuousStream for data from kafka.

4.  type PartitionOffsetMap = Map[TopicPartition, Long]