org.apache.spark.sql.catalyst

optimizer

package optimizer

Visibility
  1. Public
  2. All

Type Members

  1. case class GetCurrentDatabase(sessionCatalog: SessionCatalog) extends Rule[LogicalPlan] with Product with Serializable

    Replaces the expression of CurrentDatabase with the current database name.

  2. case class OptimizeCodegen(conf: CatalystConf) extends Rule[LogicalPlan] with Product with Serializable

    Optimizes expressions by replacing according to CodeGen configuration.

  3. case class OptimizeIn(conf: CatalystConf) extends Rule[LogicalPlan] with Product with Serializable

    Replaces (value, seq[Literal]) with optimized version(value, HashSet[Literal]) which is much faster

  4. abstract class Optimizer extends RuleExecutor[LogicalPlan]

    Abstract class all optimizers should inherit of, contains the standard batches (extending Optimizers can override this.

  5. class SimpleTestOptimizer extends Optimizer

Value Members

  1. object BooleanSimplification extends Rule[LogicalPlan] with PredicateHelper

    Simplifies boolean expressions: 1.

  2. object CollapseProject extends Rule[LogicalPlan]

    Combines two adjacent Project operators into one and perform alias substitution, merging the expressions into one single expression.

  3. object CollapseRepartition extends Rule[LogicalPlan]

    Combines adjacent Repartition operators by keeping only the last one.

  4. object ColumnPruning extends Rule[LogicalPlan]

    Attempts to eliminate the reading of unneeded columns from the query plan.

  5. object CombineFilters extends Rule[LogicalPlan] with PredicateHelper

    Combines two adjacent Filter operators into one, merging the non-redundant conditions into one conjunctive predicate.

  6. object CombineLimits extends Rule[LogicalPlan]

    Combines two adjacent Limit operators into one, merging the expressions into one single expression.

  7. object CombineUnions extends Rule[LogicalPlan]

    Combines all adjacent Union operators into a single Union.

  8. object ComputeCurrentTime extends Rule[LogicalPlan]

    Computes the current date and time to make sure we return the same result in a single query.

  9. object ConstantFolding extends Rule[LogicalPlan]

    Replaces Expressions that can be statically evaluated with equivalent Literal values.

  10. object ConvertToLocalRelation extends Rule[LogicalPlan]

    Converts local operations (i.

  11. object DecimalAggregates extends Rule[LogicalPlan]

    Speeds up aggregates on fixed-precision decimals by executing them on unscaled Long values.

  12. object EliminateOuterJoin extends Rule[LogicalPlan] with PredicateHelper

    Elimination of outer joins, if the predicates can restrict the result sets so that all null-supplying rows are eliminated

  13. object EliminateSerialization extends Rule[LogicalPlan]

    Removes cases where we are unnecessarily going between the object and serialized (InternalRow) representation of data item.

  14. object EliminateSorts extends Rule[LogicalPlan]

    Removes no-op SortOrder from Sort

  15. object EmbedSerializerInFilter extends Rule[LogicalPlan]

    Typed Filter is by default surrounded by a DeserializeToObject beneath it and a SerializeFromObject above it.

  16. object FoldablePropagation extends Rule[LogicalPlan]

    Propagate foldable expressions: Replace attributes with aliases of the original foldable expressions if possible.

  17. object InferFiltersFromConstraints extends Rule[LogicalPlan] with PredicateHelper

    Generate a list of additional filters from an operator's existing constraint but remove those that are either already part of the operator's condition or are part of the operator's child constraints.

  18. object LikeSimplification extends Rule[LogicalPlan]

    Simplifies LIKE expressions that do not need full regular expressions to evaluate the condition.

  19. object LimitPushDown extends Rule[LogicalPlan]

    Pushes down LocalLimit beneath UNION ALL and beneath the streamed inputs of outer joins.

  20. object NullPropagation extends Rule[LogicalPlan]

    Replaces Expressions that can be statically evaluated with equivalent Literal values.

  21. object PruneFilters extends Rule[LogicalPlan] with PredicateHelper

    Removes filters that can be evaluated trivially.

  22. object PushDownPredicate extends Rule[LogicalPlan] with PredicateHelper

    Pushes Filter operators through many operators iff: 1) the operator is deterministic 2) the predicate is deterministic and the operator will not change any of rows.

  23. object PushPredicateThroughJoin extends Rule[LogicalPlan] with PredicateHelper

    Pushes down Filter operators where the condition can be evaluated using only the attributes of the left or right side of a join.

  24. object PushThroughSetOperations extends Rule[LogicalPlan] with PredicateHelper

    Pushes certain operations to both sides of a Union operator.

  25. object RemoveAliasOnlyProject extends Rule[LogicalPlan]

    Removes the Project only conducting Alias of its child node.

  26. object RemoveDispensableExpressions extends Rule[LogicalPlan]

    Removes nodes that are not necessary.

  27. object RemoveLiteralFromGroupExpressions extends Rule[LogicalPlan]

    Removes literals from group expressions in Aggregate, as they have no effect to the result but only makes the grouping key bigger.

  28. object RemoveRepetitionFromGroupExpressions extends Rule[LogicalPlan]

    Removes repetition from group expressions in Aggregate, as they have no effect to the result but only makes the grouping key bigger.

  29. object ReorderJoin extends Rule[LogicalPlan] with PredicateHelper

    Reorder the joins and push all the conditions into join, so that the bottom ones have at least one condition.

  30. object ReplaceDistinctWithAggregate extends Rule[LogicalPlan]

    Replaces logical Distinct operator with an Aggregate operator.

  31. object ReplaceExceptWithAntiJoin extends Rule[LogicalPlan]

    Replaces logical Except operator with a left-anti Join operator.

  32. object ReplaceExpressions extends Rule[LogicalPlan]

    Finds all RuntimeReplaceable expressions and replace them with the expressions that can be evaluated.

  33. object ReplaceIntersectWithSemiJoin extends Rule[LogicalPlan]

    Replaces logical Intersect operator with a left-semi Join operator.

  34. object RewriteCorrelatedScalarSubquery extends Rule[LogicalPlan]

    This rule rewrites correlated ScalarSubquery expressions into LEFT OUTER joins.

  35. object RewriteDistinctAggregates extends Rule[LogicalPlan]

    This rule rewrites an aggregate query with distinct aggregations into an expanded double aggregation in which the regular aggregation expressions and every distinct clause is aggregated in a separate group.

  36. object RewritePredicateSubquery extends Rule[LogicalPlan] with PredicateHelper

    This rule rewrites predicate sub-queries into left semi/anti joins.

  37. object SimpleTestOptimizer extends SimpleTestOptimizer

    An optimizer used in test code.

  38. object SimplifyBinaryComparison extends Rule[LogicalPlan] with PredicateHelper

    Simplifies binary comparisons with semantically-equal expressions: 1) Replace '<=>' with 'true' literal.

  39. object SimplifyCaseConversionExpressions extends Rule[LogicalPlan]

    Removes the inner case conversion expressions that are unnecessary because the inner conversion is overwritten by the outer one.

  40. object SimplifyCasts extends Rule[LogicalPlan]

    Removes Casts that are unnecessary because the input is already the correct type.

  41. object SimplifyConditionals extends Rule[LogicalPlan] with PredicateHelper

    Simplifies conditional expressions (if / case).

Ungrouped