execution

Type Members

1. case class Aggregate(partial: Boolean, groupingExpressions: Seq[Expression], aggregateExpressions: Seq[NamedExpression], child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

   :: DeveloperApi :: Groups input data by groupingExpressions and computes the aggregateExpressions for each group.

2. case class BatchPythonEvaluation(udf: PythonUDF, output: Seq[Attribute], child: SparkPlan) extends SparkPlan with Product with Serializable

   :: DeveloperApi :: Uses PythonRDD to evaluate a PythonUDF, one partition of tuples at a time.

3. case class CacheTableCommand(tableName: String, plan: Option[LogicalPlan], isLazy: Boolean) extends LogicalPlan with RunnableCommand with Product with Serializable

   :: DeveloperApi ::

4. case class ConvertToSafe(child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

   :: DeveloperApi :: Converts UnsafeRows back into Java-object-based rows.

5. case class ConvertToUnsafe(child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

   :: DeveloperApi :: Converts Java-object-based rows into UnsafeRows.

6. case class DescribeCommand(child: SparkPlan, output: Seq[Attribute], isExtended: Boolean) extends LogicalPlan with RunnableCommand with Product with Serializable

   :: DeveloperApi ::

7. case class DescribeFunction(functionName: String, isExtended: Boolean) extends LogicalPlan with RunnableCommand with Product with Serializable

   A command for users to get the usage of a registered function.

8. case class EvaluatePython(udf: PythonUDF, child: LogicalPlan, resultAttribute: AttributeReference) extends catalyst.plans.logical.UnaryNode with Product with Serializable

   :: DeveloperApi :: Evaluates a PythonUDF, appending the result to the end of the input tuple.

9. case class Except(left: SparkPlan, right: SparkPlan) extends SparkPlan with BinaryNode with Product with Serializable

   :: DeveloperApi :: Returns a table with the elements from left that are not in right using the built-in spark subtract function.

10. case class Exchange(newPartitioning: Partitioning, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: Performs a shuffle that will result in the desired newPartitioning.

11. case class Expand(projections: Seq[Seq[Expression]], output: Seq[Attribute], child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    Apply the all of the GroupExpressions to every input row, hence we will get multiple output rows for a input row.

12. case class ExplainCommand(logicalPlan: LogicalPlan, output: Seq[Attribute] = ..., extended: Boolean = false) extends LogicalPlan with RunnableCommand with Product with Serializable

    An explain command for users to see how a command will be executed.

13. case class ExternalSort(sortOrder: Seq[SortOrder], global: Boolean, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    Performs a sort, spilling to disk as needed.

14. case class Filter(condition: Expression, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi ::

15. case class Generate(generator: Generator, join: Boolean, outer: Boolean, output: Seq[Attribute], child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: Applies a Generator to a stream of input rows, combining the output of each into a new stream of rows.

16. case class Intersect(left: SparkPlan, right: SparkPlan) extends SparkPlan with BinaryNode with Product with Serializable

    :: DeveloperApi :: Returns the rows in left that also appear in right using the built in spark intersection function.

17. case class Limit(limit: Int, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: Take the first limit elements.

18. case class OutputFaker(output: Seq[Attribute], child: SparkPlan) extends SparkPlan with Product with Serializable

    :: DeveloperApi :: A plan node that does nothing but lie about the output of its child.

19. case class Project(projectList: Seq[NamedExpression], child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi ::

20. class QueryExecutionException extends Exception

21. case class Repartition(numPartitions: Int, shuffle: Boolean, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: Return a new RDD that has exactly numPartitions partitions.

22. case class Sample(lowerBound: Double, upperBound: Double, withReplacement: Boolean, seed: Long, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: Sample the dataset.

23. case class SetCommand(kv: Option[(String, Option[String])]) extends LogicalPlan with RunnableCommand with Logging with Product with Serializable

    :: DeveloperApi ::

24. case class ShowFunctions(db: Option[String], pattern: Option[String]) extends LogicalPlan with RunnableCommand with Product with Serializable

    A command for users to list all of the registered functions.

25. case class ShowTablesCommand(databaseName: Option[String]) extends LogicalPlan with RunnableCommand with Product with Serializable

    A command for users to get tables in the given database.

26. class ShuffledRowRDD extends RDD[InternalRow]

    This is a specialized version of org.apache.spark.rdd.ShuffledRDD that is optimized for shuffling rows instead of Java key-value pairs.

27. case class Sort(sortOrder: Seq[SortOrder], global: Boolean, child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    Performs a sort on-heap.

28. abstract class SparkPlan extends QueryPlan[SparkPlan] with Logging with Serializable

    :: DeveloperApi ::

29. case class TakeOrderedAndProject(limit: Int, sortOrder: Seq[SortOrder], projectList: Option[Seq[NamedExpression]], child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: Take the first limit elements as defined by the sortOrder, and do projection if needed.

30. case class TungstenProject(projectList: Seq[NamedExpression], child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    A variant of Project that returns UnsafeRows.

31. case class TungstenSort(sortOrder: Seq[SortOrder], global: Boolean, child: SparkPlan, testSpillFrequency: Int = 0) extends SparkPlan with UnaryNode with Product with Serializable

    Optimized version of ExternalSort that operates on binary data (implemented as part of Project Tungsten).

32. case class UncacheTableCommand(tableName: String) extends LogicalPlan with RunnableCommand with Product with Serializable

    :: DeveloperApi ::

33. case class Union(children: Seq[SparkPlan]) extends SparkPlan with Product with Serializable

    :: DeveloperApi ::

34. final class UnsafeFixedWidthAggregationMap extends AnyRef

35. final class UnsafeKVExternalSorter extends AnyRef

36. case class Window(projectList: Seq[Attribute], windowExpression: Seq[NamedExpression], partitionSpec: Seq[Expression], orderSpec: Seq[SortOrder], child: SparkPlan) extends SparkPlan with UnaryNode with Product with Serializable

    :: DeveloperApi :: This class calculates and outputs (windowed) aggregates over the rows in a single (sorted) partition.