org.apache.spark.sql.catalyst.analysis
A batch of rules.
A strategy that runs until fix point or maxIterations times, whichever comes first.
An execution strategy for rules that indicates the maximum number of executions.
Analyze cte definitions and substitute child plan with analyzed cte definitions.
Extracts Generator from the projectList of a Project operator and create Generate operator under Project.
Extracts WindowExpressions from the projectList of a Project operator and aggregateExpressions of an Aggregate operator and creates individual Window operators for every distinct WindowSpecDefinition.
Fixes nullability of Attributes in a resolved LogicalPlan by using the nullability of corresponding Attributes of its children output Attributes.
Turns projections that contain aggregate expressions into aggregations.
Correctly handle null primitive inputs for UDF by adding extra If expression to do the null check.
Checks whether a function identifier referenced by an UnresolvedFunction is defined in the function registry.
A strategy that only runs once.
Pulls out nondeterministic expressions from LogicalPlan which is not Project or Filter, put them into an inner Project and finally project them away at the outer Project.
Replace unresolved expressions in grouping keys with resolved ones in SELECT clauses.
This rule finds aggregate expressions that are not in an aggregate operator.
Replaces UnresolvedAliass with concrete aliases.
Replaces UnresolvedDeserializer with the deserialization expression that has been resolved to the given input attributes.
Replaces UnresolvedFunctions with concrete Expressions.
Rewrites table generating expressions that either need one or more of the following in order to be resolved:
In many dialects of SQL it is valid to sort by attributes that are not present in the SELECT clause.
Removes natural or using joins by calculating output columns based on output from two sides, Then apply a Project on a normal Join to eliminate natural or using join.
Resolves NewInstance by finding and adding the outer scope to it if the object being constructed is an inner class.
In many dialects of SQL it is valid to use ordinal positions in order/sort by and group by clauses.
Replaces UnresolvedAttributes with concrete AttributeReferences from a logical plan node's children.
Replaces UnresolvedRelations with concrete relations from the catalog.
This rule resolves and rewrites subqueries inside expressions.
Replace the UpCast expression by Cast, and throw exceptions if the cast may truncate.
Check and add proper window frames for all window functions.
Check and add order to AggregateWindowFunctions.
Substitute child plan with WindowSpecDefinitions.
Returns true if expr can be evaluated using only the output of plan.
Returns true if expr can be evaluated using only the output of plan. This method
can be used to determine when it is acceptable to move expression evaluation within a query
plan.
For example consider a join between two relations R(a, b) and S(c, d).
- canEvaluate(EqualTo(a,b), R) returns true
- canEvaluate(EqualTo(a,c), R) returns false
- canEvaluate(Literal(1), R) returns true as literals CAN be evaluated on any plan
Returns true iff expr could be evaluated as a condition within join.
Returns true iff expr could be evaluated as a condition within join.
Executes the batches of rules defined by the subclass.
Executes the batches of rules defined by the subclass. The batches are executed serially using the defined execution strategy. Within each batch, rules are also executed serially.
Override to provide additional checks for correct analysis.
Override to provide additional checks for correct analysis. These rules will be evaluated after our built-in check rules.
Override to provide additional rules for the "Resolution" batch.
Override to provide additional rules for the "Resolution" batch.
Override to provide rules to do post-hoc resolution.
Override to provide rules to do post-hoc resolution. Note that these rules will be executed in an individual batch. This batch is to run right after the normal resolution batch and execute its rules in one pass.
A trivial Analyzer with a dummy SessionCatalog and EmptyFunctionRegistry. Used for testing when all relations are already filled in and the analyzer needs only to resolve attribute references.