org.apache.spark.sql.catalyst

optimizer

package optimizer

Visibility
  1. Public
  2. All

Type Members

  1. case class CheckCartesianProducts(conf: SQLConf) extends Rule[LogicalPlan] with PredicateHelper with Product with Serializable

    Check if there any cartesian products between joins of any type in the optimized plan tree.

  2. case class Cost(card: BigInt, size: BigInt) extends Product with Serializable

    This class defines the cost model for a plan.

  3. case class CostBasedJoinReorder(conf: SQLConf) extends Rule[LogicalPlan] with PredicateHelper with Product with Serializable

    Cost-based join reorder.

  4. case class DecimalAggregates(conf: SQLConf) extends Rule[LogicalPlan] with Product with Serializable

    Speeds up aggregates on fixed-precision decimals by executing them on unscaled Long values.

  5. case class EliminateOuterJoin(conf: SQLConf) extends Rule[LogicalPlan] with PredicateHelper with Product with Serializable

    Elimination of outer joins, if the predicates can restrict the result sets so that all null-supplying rows are eliminated

  6. case class GetCurrentDatabase(sessionCatalog: SessionCatalog) extends Rule[LogicalPlan] with Product with Serializable

    Replaces the expression of CurrentDatabase with the current database name.

  7. case class InferFiltersFromConstraints(conf: SQLConf) extends Rule[LogicalPlan] with PredicateHelper with Product with Serializable

    Generate a list of additional filters from an operator's existing constraint but remove those that are either already part of the operator's condition or are part of the operator's child constraints.

  8. case class JoinGraphInfo(starJoins: Set[Int], nonStarJoins: Set[Int]) extends Product with Serializable

    Helper class that keeps information about the join graph as sets of item/plan ids.

  9. case class LimitPushDown(conf: SQLConf) extends Rule[LogicalPlan] with Product with Serializable

    Pushes down LocalLimit beneath UNION ALL and beneath the streamed inputs of outer joins.

  10. case class NullPropagation(conf: SQLConf) extends Rule[LogicalPlan] with Product with Serializable

    Replaces Expressions that can be statically evaluated with equivalent Literal values.

  11. case class OptimizeCodegen(conf: SQLConf) extends Rule[LogicalPlan] with Product with Serializable

    Optimizes expressions by replacing according to CodeGen configuration.

  12. case class OptimizeIn(conf: SQLConf) extends Rule[LogicalPlan] with Product with Serializable

    Optimize IN predicates: 1.

  13. abstract class Optimizer extends RuleExecutor[LogicalPlan]

    Abstract class all optimizers should inherit of, contains the standard batches (extending Optimizers can override this.

  14. case class OrderedJoin(left: LogicalPlan, right: LogicalPlan, joinType: JoinType, condition: Option[Expression]) extends BinaryNode with Product with Serializable

    This is a mimic class for a join node that has been ordered.

  15. case class PruneFilters(conf: SQLConf) extends Rule[LogicalPlan] with PredicateHelper with Product with Serializable

    Removes filters that can be evaluated trivially.

  16. case class ReorderJoin(conf: SQLConf) extends Rule[LogicalPlan] with PredicateHelper with Product with Serializable

    Reorder the joins and push all the conditions into join, so that the bottom ones have at least one condition.

  17. class SimpleTestOptimizer extends Optimizer

  18. case class StarSchemaDetection(conf: SQLConf) extends PredicateHelper with Product with Serializable

    Encapsulates star-schema detection logic.

Value Members

  1. object BooleanSimplification extends Rule[LogicalPlan] with PredicateHelper

    Simplifies boolean expressions: 1.

  2. object CollapseProject extends Rule[LogicalPlan]

    Combines two adjacent Project operators into one and perform alias substitution, merging the expressions into one single expression.

  3. object CollapseRepartition extends Rule[LogicalPlan]

    Combines adjacent RepartitionOperation operators

  4. object CollapseWindow extends Rule[LogicalPlan]

    Collapse Adjacent Window Expression.

  5. object ColumnPruning extends Rule[LogicalPlan]

    Attempts to eliminate the reading of unneeded columns from the query plan.

  6. object CombineFilters extends Rule[LogicalPlan] with PredicateHelper

    Combines two adjacent Filter operators into one, merging the non-redundant conditions into one conjunctive predicate.

  7. object CombineLimits extends Rule[LogicalPlan]

    Combines two adjacent Limit operators into one, merging the expressions into one single expression.

  8. object CombineTypedFilters extends Rule[LogicalPlan]

    Combines two adjacent TypedFilters, which operate on same type object in condition, into one, merging the filter functions into one conjunctive function.

  9. object CombineUnions extends Rule[LogicalPlan]

    Combines all adjacent Union operators into a single Union.

  10. object ComputeCurrentTime extends Rule[LogicalPlan]

    Computes the current date and time to make sure we return the same result in a single query.

  11. object ConstantFolding extends Rule[LogicalPlan]

    Replaces Expressions that can be statically evaluated with equivalent Literal values.

  12. object ConvertToLocalRelation extends Rule[LogicalPlan]

    Converts local operations (i.

  13. object EliminateMapObjects extends Rule[LogicalPlan]

    Removes MapObjects when the following conditions are satisfied

  14. object EliminateSerialization extends Rule[LogicalPlan]

    Removes cases where we are unnecessarily going between the object and serialized (InternalRow) representation of data item.

  15. object EliminateSorts extends Rule[LogicalPlan]

    Removes no-op SortOrder from Sort

  16. object FoldablePropagation extends Rule[LogicalPlan]

    Propagate foldable expressions: Replace attributes with aliases of the original foldable expressions if possible.

  17. object JoinReorderDP extends PredicateHelper with Logging

    Reorder the joins using a dynamic programming algorithm.

  18. object JoinReorderDPFilters extends PredicateHelper

    Implements optional filters to reduce the search space for join enumeration.

  19. object LikeSimplification extends Rule[LogicalPlan]

    Simplifies LIKE expressions that do not need full regular expressions to evaluate the condition.

  20. object PropagateEmptyRelation extends Rule[LogicalPlan] with PredicateHelper

    Collapse plans consisting empty local relations generated by PruneFilters.

  21. object PullupCorrelatedPredicates extends Rule[LogicalPlan] with PredicateHelper

    Pull out all (outer) correlated predicates from a given subquery.

  22. object PushDownPredicate extends Rule[LogicalPlan] with PredicateHelper

    Pushes Filter operators through many operators iff: 1) the operator is deterministic 2) the predicate is deterministic and the operator will not change any of rows.

  23. object PushPredicateThroughJoin extends Rule[LogicalPlan] with PredicateHelper

    Pushes down Filter operators where the condition can be evaluated using only the attributes of the left or right side of a join.

  24. object PushProjectionThroughUnion extends Rule[LogicalPlan] with PredicateHelper

    Pushes Project operator to both sides of a Union operator.

  25. object RemoveDispensableExpressions extends Rule[LogicalPlan]

    Removes nodes that are not necessary.

  26. object RemoveLiteralFromGroupExpressions extends Rule[LogicalPlan]

    Removes literals from group expressions in Aggregate, as they have no effect to the result but only makes the grouping key bigger.

  27. object RemoveRedundantAliases extends Rule[LogicalPlan]

    Remove redundant aliases from a query plan.

  28. object RemoveRedundantProject extends Rule[LogicalPlan]

    Remove projections from the query plan that do not make any modifications.

  29. object RemoveRepetitionFromGroupExpressions extends Rule[LogicalPlan]

    Removes repetition from group expressions in Aggregate, as they have no effect to the result but only makes the grouping key bigger.

  30. object ReorderAssociativeOperator extends Rule[LogicalPlan]

    Reorder associative integral-type operators and fold all constants into one.

  31. object ReplaceDeduplicateWithAggregate extends Rule[LogicalPlan]

    Replaces logical Deduplicate operator with an Aggregate operator.

  32. object ReplaceDistinctWithAggregate extends Rule[LogicalPlan]

    Replaces logical Distinct operator with an Aggregate operator.

  33. object ReplaceExceptWithAntiJoin extends Rule[LogicalPlan]

    Replaces logical Except operator with a left-anti Join operator.

  34. object ReplaceExpressions extends Rule[LogicalPlan]

    Finds all RuntimeReplaceable expressions and replace them with the expressions that can be evaluated.

  35. object ReplaceIntersectWithSemiJoin extends Rule[LogicalPlan]

    Replaces logical Intersect operator with a left-semi Join operator.

  36. object RewriteCorrelatedScalarSubquery extends Rule[LogicalPlan]

    This rule rewrites correlated ScalarSubquery expressions into LEFT OUTER joins.

  37. object RewriteDistinctAggregates extends Rule[LogicalPlan]

    This rule rewrites an aggregate query with distinct aggregations into an expanded double aggregation in which the regular aggregation expressions and every distinct clause is aggregated in a separate group.

  38. object RewritePredicateSubquery extends Rule[LogicalPlan] with PredicateHelper

    This rule rewrites predicate sub-queries into left semi/anti joins.

  39. object SimpleTestOptimizer extends SimpleTestOptimizer

    An optimizer used in test code.

  40. object SimplifyBinaryComparison extends Rule[LogicalPlan] with PredicateHelper

    Simplifies binary comparisons with semantically-equal expressions: 1) Replace '<=>' with 'true' literal.

  41. object SimplifyCaseConversionExpressions extends Rule[LogicalPlan]

    Removes the inner case conversion expressions that are unnecessary because the inner conversion is overwritten by the outer one.

  42. object SimplifyCasts extends Rule[LogicalPlan]

    Removes Casts that are unnecessary because the input is already the correct type.

  43. object SimplifyConditionals extends Rule[LogicalPlan] with PredicateHelper

    Simplifies conditional expressions (if / case).

  44. object SimplifyCreateArrayOps extends Rule[LogicalPlan]

    push down operations into CreateArray.

  45. object SimplifyCreateMapOps extends Rule[LogicalPlan]

    push down operations into CreateMap.

  46. object SimplifyCreateStructOps extends Rule[LogicalPlan]

    push down operations into CreateNamedStructLike.

Ungrouped