execution

Type Members

1. case class Aggregate(partial: Boolean, groupingExpressions: Seq[Expression], aggregateExpressions: Seq[NamedExpression], child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

   :: DeveloperApi :: Groups input data by groupingExpressions and computes the aggregateExpressions for each group.

2. case class AggregateEvaluation(schema: Seq[Attribute], initialValues: Seq[Expression], update: Seq[Expression], result: Expression) extends Product with Serializable

3. case class BatchPythonEvaluation(udf: PythonUDF, output: Seq[Attribute], child: SparkPlan) extends SparkPlan with Product with Serializable

   :: DeveloperApi :: Uses PythonRDD to evaluate a PythonUDF, one partition of tuples at a time.

4. case class CacheTableCommand(tableName: String, plan: Option[LogicalPlan], isLazy: Boolean) extends LogicalPlan with RunnableCommand with Product with Serializable

   :: DeveloperApi ::

5. case class DescribeCommand(child: SparkPlan, output: Seq[Attribute], isExtended: Boolean) extends LogicalPlan with RunnableCommand with Product with Serializable

   :: DeveloperApi ::

6. case class Distinct(partial: Boolean, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

   :: DeveloperApi :: Computes the set of distinct input rows using a HashSet.

7. case class EvaluatePython(udf: PythonUDF, child: LogicalPlan, resultAttribute: AttributeReference) extends catalyst.plans.logical.UnaryNode with Product with Serializable

   :: DeveloperApi :: Evaluates a PythonUDF, appending the result to the end of the input tuple.

8. case class Except(left: SparkPlan, right: SparkPlan) extends SparkPlan with BinaryNode with Product with Serializable

   :: DeveloperApi :: Returns a table with the elements from left that are not in right using the built-in spark subtract function.

9. case class Exchange(newPartitioning: Partitioning, newOrdering: Seq[SortOrder], child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

   :: DeveloperApi :: Performs a shuffle that will result in the desired newPartitioning.

10. case class Expand(projections: Seq[GroupExpression], output: Seq[Attribute], child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    Apply the all of the GroupExpressions to every input row, hence we will get multiple output rows for a input row.

11. case class ExplainCommand(logicalPlan: LogicalPlan, output: Seq[Attribute] = ..., extended: Boolean = false) extends LogicalPlan with RunnableCommand with Product with Serializable

    An explain command for users to see how a command will be executed.

12. case class ExternalSort(sortOrder: Seq[SortOrder], global: Boolean, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: Performs a sort, spilling to disk as needed.

13. case class Filter(condition: Expression, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi ::

14. case class Generate(generator: Generator, join: Boolean, outer: Boolean, output: Seq[Attribute], child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: Applies a Generator to a stream of input rows, combining the output of each into a new stream of rows.

15. case class GeneratedAggregate(partial: Boolean, groupingExpressions: Seq[Expression], aggregateExpressions: Seq[NamedExpression], unsafeEnabled: Boolean, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: Alternate version of aggregation that leverages projection and thus code generation.

16. case class Intersect(left: SparkPlan, right: SparkPlan) extends SparkPlan with BinaryNode with Product with Serializable

    :: DeveloperApi :: Returns the rows in left that also appear in right using the built in spark intersection function.

17. case class Limit(limit: Int, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: Take the first limit elements.

18. case class OutputFaker(output: Seq[Attribute], child: SparkPlan) extends SparkPlan with Product with Serializable

    :: DeveloperApi :: A plan node that does nothing but lie about the output of its child.

19. case class Project(projectList: Seq[NamedExpression], child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi ::

20. class QueryExecutionException extends Exception

21. case class Repartition(numPartitions: Int, shuffle: Boolean, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: Return a new RDD that has exactly numPartitions partitions.

22. case class Sample(lowerBound: Double, upperBound: Double, withReplacement: Boolean, seed: Long, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: Sample the dataset.

23. case class SetCommand(kv: Option[(String, Option[String])], output: Seq[Attribute]) extends LogicalPlan with RunnableCommand with Logging with Product with Serializable

    :: DeveloperApi ::

24. case class ShowTablesCommand(databaseName: Option[String]) extends LogicalPlan with RunnableCommand with Product with Serializable

    A command for users to get tables in the given database.

25. case class Sort(sortOrder: Seq[SortOrder], global: Boolean, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: Performs a sort on-heap.

26. abstract class SparkPlan extends QueryPlan[SparkPlan] with Logging with Serializable

    :: DeveloperApi ::

27. case class TakeOrdered(limit: Int, sortOrder: Seq[SortOrder], child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: Take the first limit elements as defined by the sortOrder.

28. case class UncacheTableCommand(tableName: String) extends LogicalPlan with RunnableCommand with Product with Serializable

    :: DeveloperApi ::

29. case class Union(children: Seq[SparkPlan]) extends SparkPlan with Product with Serializable

    :: DeveloperApi ::

30. case class Window(projectList: Seq[Attribute], windowExpression: Seq[NamedExpression], windowSpec: WindowSpecDefinition, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: For every row, evaluates windowExpression containing Window Functions and attaches the results with other regular expressions (presented by projectList).