Type Members

1.  class ContinuousDataSourceRDD extends RDD[InternalRow]

   The bottom-most RDD of a continuous processing read task.

   The bottom-most RDD of a continuous processing read task. Wraps a ContinuousQueuedDataReader to read from the remote source, and polls that queue for incoming rows.

   Note that continuous processing calls compute() multiple times, and the same ContinuousQueuedDataReader instance will/must be shared between each call for the same split.

2.  class ContinuousDataSourceRDDPartition extends Partition with Serializable
3.  class ContinuousExecution extends StreamExecution
4.  class ContinuousQueuedDataReader extends Closeable

   A wrapper for a continuous processing data reader, including a reading queue and epoch markers.

   A wrapper for a continuous processing data reader, including a reading queue and epoch markers.

   This will be instantiated once per partition - successive calls to compute() in the ContinuousDataSourceRDD will reuse the same reader. This is required to get continuity of offsets across epochs. Each compute() should call the next() method here until null is returned.

5.  class ContinuousTaskRetryException extends SparkException

   An exception thrown when a continuous processing task runs with a nonzero attempt ID.

6.  class ContinuousWriteRDD extends RDD[Unit]

   The RDD writing to a sink in continuous processing.

   The RDD writing to a sink in continuous processing.

   Within each task, we repeatedly call prev.compute(). Each resulting iterator contains the data to be written for one epoch, which we commit and forward to the driver.

   We keep repeating prev.compute() and writing new epochs until the query is shut down.

7.  case class RateStreamContinuousInputPartition(startValue: Long, startTimeMs: Long, partitionIndex: Int, increment: Long, rowsPerSecond: Double) extends InputPartition with Product with Serializable
8.  class RateStreamContinuousPartitionReader extends ContinuousPartitionReader[InternalRow]
9.  class RateStreamContinuousStream extends ContinuousStream
10.  case class RateStreamPartitionOffset(partition: Int, currentValue: Long, currentTimeMs: Long) extends PartitionOffset with Product with Serializable
11.  case class SetWriterPartitions(numPartitions: Int) extends EpochCoordinatorMessage with Product with Serializable
12.  case class TextSocketContinuousInputPartition(driverEndpointName: String, partitionId: Int, startOffset: Int, includeTimestamp: Boolean) extends InputPartition with Product with Serializable

    Continuous text socket input partition.

13.  class TextSocketContinuousPartitionReader extends ContinuousPartitionReader[InternalRow]

    Continuous text socket input partition reader.

    Continuous text socket input partition reader.

    Polls the driver endpoint for new records.

14.  class TextSocketContinuousStream extends ContinuousStream with Logging

    A ContinuousStream that reads text lines through a TCP socket, designed only for tutorials and debugging.

    A ContinuousStream that reads text lines through a TCP socket, designed only for tutorials and debugging. This ContinuousStream will *not* work in production applications due to multiple reasons, including no support for fault recovery.

    The driver maintains a socket connection to the host-port, keeps the received messages in buckets and serves the messages to the executors via a RPC endpoint.

15.  case class TextSocketOffset(offsets: List[Int]) extends connector.read.streaming.Offset with Product with Serializable
16.  case class WriteToContinuousDataSource(write: StreamingWrite, query: LogicalPlan) extends LogicalPlan with Product with Serializable

    The logical plan for writing data in a continuous stream.

17.  case class WriteToContinuousDataSourceExec(write: StreamingWrite, query: SparkPlan) extends SparkPlan with UnaryExecNode with Logging with Product with Serializable

    The physical plan for writing data into a continuous processing StreamingWrite.