Packages

package debug

Contains methods for debugging query execution.

Usage:

import org.apache.spark.sql.execution.debug._
sql("SELECT 1").debug()
sql("SELECT 1").debugCodegen()

or for streaming case (structured streaming):

import org.apache.spark.sql.execution.debug._
val query = df.writeStream.<...>.start()
query.debugCodegen()

Note that debug in structured streaming is not supported, because it doesn't make sense for streaming to execute batch once while main query is running concurrently.

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. debug
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Type Members

  1. case class DebugExec(child: SparkPlan) extends SparkPlan with UnaryExecNode with CodegenSupport with Product with Serializable
  2. implicit class DebugQuery extends Logging

    Augments Datasets with debug methods.

  3. implicit class DebugStreamQuery extends Logging

Value Members

  1. def codegenString(query: StreamingQuery): String

    Get WholeStageCodegenExec subtrees and the codegen in a query plan into one String

    Get WholeStageCodegenExec subtrees and the codegen in a query plan into one String

    query

    the streaming query for codegen

    returns

    single String containing all WholeStageCodegen subtrees and corresponding codegen

  2. def codegenString(plan: SparkPlan): String

    Get WholeStageCodegenExec subtrees and the codegen in a query plan into one String

    Get WholeStageCodegenExec subtrees and the codegen in a query plan into one String

    plan

    the query plan for codegen

    returns

    single String containing all WholeStageCodegen subtrees and corresponding codegen

  3. def codegenStringSeq(query: StreamingQuery): Seq[(String, String, ByteCodeStats)]

    Get WholeStageCodegenExec subtrees and the codegen in a query plan

    Get WholeStageCodegenExec subtrees and the codegen in a query plan

    query

    the streaming query for codegen

    returns

    Sequence of WholeStageCodegen subtrees and corresponding codegen

  4. def codegenStringSeq(plan: SparkPlan): Seq[(String, String, ByteCodeStats)]

    Get WholeStageCodegenExec subtrees and the codegen in a query plan

    Get WholeStageCodegenExec subtrees and the codegen in a query plan

    plan

    the query plan for codegen

    returns

    Sequence of WholeStageCodegen subtrees and corresponding codegen

  5. def writeCodegen(append: (String) => Unit, plan: SparkPlan): Unit

Inherited from AnyRef

Inherited from Any

Ungrouped