Packages

package root

Definition Classes: root

package org

Definition Classes: root

package apache

Definition Classes: org

package spark

Definition Classes: apache

package sql

Allows the execution of relational queries, including those expressed in SQL using Spark.

Definition Classes: spark

package execution

The physical execution component of Spark SQL.

The physical execution component of Spark SQL. Note that this is a private package. All classes in catalyst are considered an internal API to Spark SQL and are subject to change between minor releases.

Definition Classes: sql

package adaptive

Definition Classes: execution

package aggregate

Definition Classes: execution

package analysis

Definition Classes: execution

package arrow

Definition Classes: execution

package columnar

Definition Classes: execution

package command

Definition Classes: execution

package datasources

Definition Classes: execution

package debug

Contains methods for debugging query execution.

Usage:

import org.apache.spark.sql.execution.debug._
sql("SELECT 1").debug()
sql("SELECT 1").debugCodegen()

or for streaming case (structured streaming):

import org.apache.spark.sql.execution.debug._
val query = df.writeStream.<...>.start()
query.debugCodegen()

Note that debug in structured streaming is not supported, because it doesn't make sense for streaming to execute batch once while main query is running concurrently.

Definition Classes: execution

package dynamicpruning

Definition Classes: execution

package exchange

Definition Classes: execution

package joins

Physical execution operators for join operations.

Definition Classes: execution

package metric

Definition Classes: execution

SQLMetric

SQLMetricInfo

SQLMetrics

SQLShuffleReadMetricsReporter

SQLShuffleWriteMetricsReporter

package python

Definition Classes: execution

package r

Definition Classes: execution

package stat

Definition Classes: execution

package streaming

Definition Classes: execution

package ui

Definition Classes: execution

package vectorized

Definition Classes: execution

package window

Definition Classes: execution

org.apache.spark.sql.execution

metric

package metric

Ordering

Alphabetic

Visibility

Public
All

Type Members

class SQLMetric extends AccumulatorV2[Long, Long]
A metric used in a SQL query plan.
A metric used in a SQL query plan. This is implemented as an AccumulatorV2. Updates on the executor side are automatically propagated and shown in the SQL UI through metrics. Updates on the driver side must be explicitly posted using SQLMetrics.postDriverMetricUpdates().
class SQLMetricInfo extends AnyRef
:: DeveloperApi :: Stores information about a SQL Metric.
:: DeveloperApi :: Stores information about a SQL Metric.

Annotations
@DeveloperApi()
class SQLShuffleReadMetricsReporter extends TempShuffleReadMetrics
A shuffle metrics reporter for SQL exchange operators.
class SQLShuffleWriteMetricsReporter extends ShuffleWriteMetricsReporter
A shuffle write metrics reporter for SQL exchange operators.

Packages

metric

package metric

Type Members

Value Members

Ungrouped

Packages

metric 

package metric

Type Members

Value Members

Ungrouped

metric