org.apache.spark.sql.execution.streaming

IncrementalExecution

class IncrementalExecution extends QueryExecution with Logging

A variant of QueryExecution that allows the execution of the given LogicalPlan plan incrementally. Possibly preserving state in between each execution.

Linear Supertypes
Logging, QueryExecution, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. IncrementalExecution
  2. Logging
  3. QueryExecution
  4. AnyRef
  5. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new IncrementalExecution(sparkSession: SparkSession, logicalPlan: LogicalPlan, outputMode: OutputMode, checkpointLocation: String, currentBatchId: Long, currentEventTimeWatermark: Long)

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any

  3. final def ##(): Int

    Definition Classes
    AnyRef → Any

  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any

  6. lazy val analyzed: LogicalPlan

    Definition Classes
    QueryExecution

  7. final def asInstanceOf[T0]: T0

    Definition Classes
    Any

  8. def assertAnalyzed(): Unit

    Definition Classes
    QueryExecution

  9. def assertSupported(): Unit

    No need assert supported, as this check has already been done

    No need assert supported, as this check has already been done

    Definition Classes
    IncrementalExecution → QueryExecution

  10. val checkpointLocation: String

  11. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  12. val currentBatchId: Long

  13. val currentEventTimeWatermark: Long

  14. object debug

    A special namespace for commands that can be used to debug query execution.

  15. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  16. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any

  17. lazy val executedPlan: SparkPlan

    Definition Classes
    QueryExecution

  18. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  19. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any

  20. def hashCode(): Int

    Definition Classes
    AnyRef → Any

  21. def hiveResultString(): Seq[String]

    Returns the result as a hive compatible sequence of strings.

    Returns the result as a hive compatible sequence of strings. For native commands, the execution is simply passed back to Hive.

    Definition Classes
    QueryExecution

  22. def initializeLogIfNecessary(isInterpreter: Boolean): Unit

    Attributes
    protected
    Definition Classes
    Logging

  23. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any

  24. def isTraceEnabled(): Boolean

    Attributes
    protected
    Definition Classes
    Logging

  25. def log: Logger

    Attributes
    protected
    Definition Classes
    Logging

  26. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  27. def logDebug(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  28. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  29. def logError(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  30. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  31. def logInfo(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  32. def logName: String

    Attributes
    protected
    Definition Classes
    Logging

  33. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  34. def logTrace(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  35. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  36. def logWarning(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  37. val logical: LogicalPlan

    Definition Classes
    QueryExecution

  38. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  39. final def notify(): Unit

    Definition Classes
    AnyRef

  40. final def notifyAll(): Unit

    Definition Classes
    AnyRef

  41. lazy val optimizedPlan: LogicalPlan

    See [SPARK-18339] Walk the optimized logical plan and replace CurrentBatchTimestamp with the desired literal

    See [SPARK-18339] Walk the optimized logical plan and replace CurrentBatchTimestamp with the desired literal

    Definition Classes
    IncrementalExecution → QueryExecution

  42. val outputMode: OutputMode

  43. def planner: SparkPlanner

    Definition Classes
    IncrementalExecution → QueryExecution

  44. def preparations: Seq[Rule[SparkPlan]]

    A sequence of rules that will be applied in order to the physical plan before execution.

    A sequence of rules that will be applied in order to the physical plan before execution.

    Definition Classes
    IncrementalExecution → QueryExecution

  45. def prepareForExecution(plan: SparkPlan): SparkPlan

    Prepares a planned SparkPlan for execution by inserting shuffle operations and internal row format conversions as needed.

    Prepares a planned SparkPlan for execution by inserting shuffle operations and internal row format conversions as needed.

    Attributes
    protected
    Definition Classes
    QueryExecution

  46. def simpleString: String

    Definition Classes
    QueryExecution

  47. lazy val sparkPlan: SparkPlan

    Definition Classes
    QueryExecution

  48. val state: Rule[SparkPlan]

    Locates save/restore pairs surrounding aggregation.

  49. val stateStrategy: Seq[Strategy]

  50. def stringOrError[A](f: ⇒ A): String

    Attributes
    protected
    Definition Classes
    QueryExecution

  51. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef

  52. lazy val toRdd: RDD[InternalRow]

    Internal version of the RDD.

    Internal version of the RDD. Avoids copies and has no schema

    Definition Classes
    QueryExecution

  53. def toString(): String

    Definition Classes
    QueryExecution → AnyRef → Any

  54. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  55. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  56. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  57. lazy val withCachedData: LogicalPlan

    Definition Classes
    QueryExecution

Inherited from Logging

Inherited from QueryExecution

Inherited from AnyRef

Inherited from Any

Ungrouped