Packages

c

org.apache.spark.sql.kafka010

KafkaContinuousPartitionReader

class KafkaContinuousPartitionReader extends ContinuousPartitionReader[InternalRow]

A per-task data reader for continuous Kafka processing.

Linear Supertypes
ContinuousPartitionReader[InternalRow], PartitionReader[InternalRow], Closeable, AutoCloseable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. KafkaContinuousPartitionReader
  2. ContinuousPartitionReader
  3. PartitionReader
  4. Closeable
  5. AutoCloseable
  6. AnyRef
  7. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Instance Constructors

  1. new KafkaContinuousPartitionReader(topicPartition: TopicPartition, startOffset: Long, kafkaParams: Map[String, AnyRef], pollTimeoutMs: Long, failOnDataLoss: Boolean, includeHeaders: Boolean)

    topicPartition

    The (topic, partition) pair this data reader is responsible for.

    startOffset

    The offset to start reading from within the partition.

    kafkaParams

    Kafka consumer params to use.

    pollTimeoutMs

    The timeout for Kafka consumer polling.

    failOnDataLoss

    Flag indicating whether data reader should fail if some offsets are skipped.

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##: Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
  6. def close(): Unit
    Definition Classes
    KafkaContinuousPartitionReader → Closeable → AutoCloseable
  7. def currentMetricsValues(): Array[CustomTaskMetric]
    Definition Classes
    PartitionReader
  8. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  9. def equals(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef → Any
  10. def get(): UnsafeRow
    Definition Classes
    KafkaContinuousPartitionReader → PartitionReader
  11. final def getClass(): Class[_ <: AnyRef]
    Definition Classes
    AnyRef → Any
    Annotations
    @IntrinsicCandidate() @native()
  12. def getOffset(): KafkaSourcePartitionOffset
    Definition Classes
    KafkaContinuousPartitionReader → ContinuousPartitionReader
  13. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @IntrinsicCandidate() @native()
  14. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  15. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  16. def next(): Boolean
    Definition Classes
    KafkaContinuousPartitionReader → PartitionReader
  17. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @IntrinsicCandidate() @native()
  18. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @IntrinsicCandidate() @native()
  19. final def synchronized[T0](arg0: => T0): T0
    Definition Classes
    AnyRef
  20. def toString(): String
    Definition Classes
    AnyRef → Any
  21. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  22. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException]) @native()
  23. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.Throwable]) @Deprecated
    Deprecated

    (Since version 9)

Inherited from ContinuousPartitionReader[InternalRow]

Inherited from PartitionReader[InternalRow]

Inherited from Closeable

Inherited from AutoCloseable

Inherited from AnyRef

Inherited from Any

Ungrouped