org.apache.spark.sql.execution.streaming
StreamingQueryWrapper
Related Doc:
package streaming
Waits for the termination of this query, either by query.stop() or by an exception.
Waits for the termination of this query, either by query.stop() or by an exception.
If the query has terminated with an exception, then the exception will be thrown.
Otherwise, it returns whether the query has terminated or not within the timeoutMs
milliseconds.
If the query has terminated, then all subsequent calls to this method will either return
true immediately (if the query was terminated by stop()), or throw the exception
immediately (if the query has terminated with exception).
2.0.0
StreamingQueryException if the query has terminated with an exception
Waits for the termination of this query, either by query.stop() or by an exception.
Waits for the termination of this query, either by query.stop() or by an exception.
If the query has terminated with an exception, then the exception will be thrown.
If the query has terminated, then all subsequent calls to this method will either return
immediately (if the query was terminated by stop()), or throw the exception
immediately (if the query has terminated with exception).
2.0.0
StreamingQueryException if the query has terminated with an exception.
Returns the StreamingQueryException if the query was terminated by an exception.
Returns the StreamingQueryException if the query was terminated by an exception.
2.0.0
Prints the physical plan to the console for debugging purposes.
Prints the physical plan to the console for debugging purposes.
whether to do extended explain or not
2.0.0
Prints the physical plan to the console for debugging purposes.
Prints the physical plan to the console for debugging purposes.
2.0.0
This method is called in Python.
This method is called in Python. Python cannot call "explain" directly as it outputs in the JVM process, which may not be visible in Python process.
Returns the unique id of this query that persists across restarts from checkpoint data.
Returns the unique id of this query that persists across restarts from checkpoint data. That is, this id is generated when a query is started for the first time, and will be the same every time it is restarted from checkpoint data. Also see runId.
2.1.0
Returns true if this query is actively running.
Returns true if this query is actively running.
2.0.0
Returns the most recent StreamingQueryProgress update of this streaming query.
Returns the most recent StreamingQueryProgress update of this streaming query.
2.1.0
Returns the user-specified name of the query, or null if not specified.
Returns the user-specified name of the query, or null if not specified.
This name can be specified in the org.apache.spark.sql.streaming.DataStreamWriter
as dataframe.writeStream.queryName("query").start().
This name, if set, must be unique across all active queries.
2.0.0
Blocks until all available data in the source has been processed and committed to the sink.
Blocks until all available data in the source has been processed and committed to the sink.
This method is intended for testing. Note that in the case of continually arriving data, this
method may block forever. Additionally, this method is only guaranteed to block until data that
has been synchronously appended data to a org.apache.spark.sql.execution.streaming.Source
prior to invocation. (i.e. getOffset must immediately reflect the addition).
2.0.0
Returns an array of the most recent StreamingQueryProgress updates for this query.
Returns an array of the most recent StreamingQueryProgress updates for this query.
The number of progress updates retained for each stream is configured by Spark session
configuration spark.sql.streaming.numRecentProgressUpdates.
2.1.0
Returns the unique id of this run of the query.
Returns the unique id of this run of the query. That is, every start/restart of a query will generated a unique runId. Therefore, every time a query is restarted from checkpoint, it will have the same id but different runIds.
Returns the SparkSession associated with this.
Returns the SparkSession associated with this.
2.0.0
Returns the current status of the query.
Returns the current status of the query.
2.0.2
Stops the execution of this query if it is running.
Stops the execution of this query if it is running. This method blocks until the threads performing execution has stopped.
2.0.0
Wrap non-serializable StreamExecution to make the query serializable as it's easy to for it to get captured with normal usage. It's safe to capture the query but not use it in executors. However, if the user tries to call its methods, it will throw
IllegalStateException.