org.apache.spark.sql.execution

python

package python

Visibility
  1. Public
  2. All

Type Members

  1. case class BatchEvalPythonExec(udfs: Seq[PythonUDF], output: Seq[Attribute], child: SparkPlan) extends SparkPlan with Product with Serializable

    A physical plan that evaluates a PythonUDF, one partition of tuples at a time.

  2. case class PythonUDF(name: String, func: PythonFunction, dataType: DataType, children: Seq[Expression]) extends Expression with Unevaluable with NonSQLExpression with Product with Serializable

    A serialized version of a Python lambda function.

  3. case class UserDefinedPythonFunction(name: String, func: PythonFunction, dataType: DataType) extends Product with Serializable

    A user-defined Python function.

Value Members

  1. object EvaluatePython

  2. object ExtractPythonUDFFromAggregate extends Rule[LogicalPlan]

    Extracts all the Python UDFs in logical aggregate, which depends on aggregate expression or grouping key, evaluate them after aggregate.

  3. object ExtractPythonUDFs extends Rule[SparkPlan] with PredicateHelper

    Extracts PythonUDFs from operators, rewriting the query plan so that the UDF can be evaluated alone in a batch.

Ungrouped