Type Members

1.  final class DataStreamReader extends Logging

   Interface used to load a streaming Dataset from external storage systems (e.g.

   Interface used to load a streaming Dataset from external storage systems (e.g. file systems, key-value stores, etc). Use SparkSession.readStream to access this.

   Annotations

   @Evolving()

   Since

   3.5.0

2.  final class DataStreamWriter[T] extends Logging

   Interface used to write a streaming Dataset to external storage systems (e.g.

   Interface used to write a streaming Dataset to external storage systems (e.g. file systems, key-value stores, etc). Use Dataset.writeStream to access this.

   Annotations

   @Evolving()

   Since

   3.5.0

3.  class RemoteStreamingQuery extends StreamingQuery
4.  class SinkProgress extends Serializable

   Information about progress made for a sink in the execution of a StreamingQuery during a trigger.

   Information about progress made for a sink in the execution of a StreamingQuery during a trigger. See StreamingQueryProgress for more information.

   Annotations

   @Evolving()

   Since

   3.5.0

5.  class SourceProgress extends Serializable

   Information about progress made for a source in the execution of a StreamingQuery during a trigger.

   Information about progress made for a source in the execution of a StreamingQuery during a trigger. See StreamingQueryProgress for more information.

   Annotations

   @Evolving()

   Since

   3.5.0

6.  class StateOperatorProgress extends Serializable

   Information about updates made to stateful operators in a StreamingQuery during a trigger.

   Information about updates made to stateful operators in a StreamingQuery during a trigger.

   Annotations

   @Evolving()

7.  trait StreamingQuery extends AnyRef

   A handle to a query that is executing continuously in the background as new data arrives.

   A handle to a query that is executing continuously in the background as new data arrives. All these methods are thread-safe.

   Annotations

   @Evolving()

   Since

   3.5.0

8.  class StreamingQueryException extends Exception with SparkThrowable

   Exception that stopped a StreamingQuery in Spark Connect.

   Exception that stopped a StreamingQuery in Spark Connect. Currently not all fields in the original StreamingQueryException are supported.

   Annotations

   @Evolving()

   Since

   3.5.0

9.  abstract class StreamingQueryListener extends Serializable

   Interface for listening to events related to StreamingQueries.

   Interface for listening to events related to StreamingQueries.

   Annotations

   @Evolving()

   Since

   3.5.0

   Note

   The methods are not thread-safe as they may be called from different threads.

10.  class StreamingQueryManager extends Logging

    A class to manage all the StreamingQuery active in a SparkSession.

    A class to manage all the StreamingQuery active in a SparkSession.

    Annotations

    @Evolving()

    Since

    3.5.0

11.  class StreamingQueryProgress extends Serializable

    Information about progress made in the execution of a StreamingQuery during a trigger.

    Information about progress made in the execution of a StreamingQuery during a trigger. Each event relates to processing done for a single trigger of the streaming query. Events are emitted even when no new data is available to be processed.

    Annotations

    @Evolving()

    Since

    3.5.0

12.  class StreamingQueryStatus extends Serializable

    Reports information about the instantaneous status of a streaming query.

    Reports information about the instantaneous status of a streaming query.

    Annotations

    @Evolving()

    Since

    3.5.0