Packages

  • package root
    Definition Classes
    root
  • package org
    Definition Classes
    root
  • package apache
    Definition Classes
    org
  • package spark
    Definition Classes
    apache
  • package sql

    Allows the execution of relational queries, including those expressed in SQL using Spark.

    Allows the execution of relational queries, including those expressed in SQL using Spark.

    Definition Classes
    spark
  • package execution

    The physical execution component of Spark SQL.

    The physical execution component of Spark SQL. Note that this is a private package. All classes in catalyst are considered an internal API to Spark SQL and are subject to change between minor releases.

    Definition Classes
    sql
  • package debug

    Contains methods for debugging query execution.

    Contains methods for debugging query execution.

    Usage:

    import org.apache.spark.sql.execution.debug._
sql("SELECT 1").debug()
sql("SELECT 1").debugCodegen()

    or for streaming case (structured streaming):

    import org.apache.spark.sql.execution.debug._
val query = df.writeStream.<...>.start()
query.debugCodegen()

    Note that debug in structured streaming is not supported, because it doesn't make sense for streaming to execute batch once while main query is running concurrently.

    Definition Classes
    execution
  • case class DebugExec(child: SparkPlan) extends SparkPlan with UnaryExecNode with CodegenSupport with Product with Serializable
    Definition Classes
    debug
  • ColumnMetrics
  • SetAccumulator

class SetAccumulator[T] extends AccumulatorV2[T, Set[T]]

Linear Supertypes
AccumulatorV2[T, Set[T]], Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SetAccumulator
  2. AccumulatorV2
  3. Serializable
  4. Serializable
  5. AnyRef
  6. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new SetAccumulator()

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. def add(v: T): Unit
    Definition Classes
    SetAccumulator → AccumulatorV2
  5. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  6. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  7. def copy(): AccumulatorV2[T, Set[T]]
    Definition Classes
    SetAccumulator → AccumulatorV2
  8. def copyAndReset(): AccumulatorV2[T, Set[T]]
    Definition Classes
    AccumulatorV2
  9. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  10. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  11. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  12. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  13. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  14. final def id: Long
    Definition Classes
    AccumulatorV2
  15. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  16. final def isRegistered: Boolean
    Definition Classes
    AccumulatorV2
  17. def isZero: Boolean
    Definition Classes
    SetAccumulator → AccumulatorV2
  18. def merge(other: AccumulatorV2[T, Set[T]]): Unit
    Definition Classes
    SetAccumulator → AccumulatorV2
  19. final def name: Option[String]
    Definition Classes
    AccumulatorV2
  20. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  21. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  22. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  23. def reset(): Unit
    Definition Classes
    SetAccumulator → AccumulatorV2
  24. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  25. def toString(): String
    Definition Classes
    AccumulatorV2 → AnyRef → Any
  26. def value: Set[T]
    Definition Classes
    SetAccumulator → AccumulatorV2
  27. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  28. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  29. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  30. final def writeReplace(): Any
    Attributes
    protected
    Definition Classes
    AccumulatorV2

Inherited from AccumulatorV2[T, Set[T]]

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped