org.apache.spark.sql.catalyst.analysis
SimpleAnalyzer
Related Doc:
package analysis
A batch of rules.
A batch of rules.
A strategy that runs until fix point or maxIterations times, whichever comes first.
A strategy that runs until fix point or maxIterations times, whichever comes first.
An execution strategy for rules that indicates the maximum number of executions.
An execution strategy for rules that indicates the maximum number of executions. If the execution reaches fix point (i.e. converge) before maxIterations, it will stop.
Substitute child plan with cte definitions
Substitute child plan with cte definitions
Extracts WindowExpressions from the projectList of a Project operator and aggregateExpressions of an Aggregate operator and creates individual Window operators for every distinct WindowSpecDefinition.
Extracts WindowExpressions from the projectList of a Project operator and aggregateExpressions of an Aggregate operator and creates individual Window operators for every distinct WindowSpecDefinition.
This rule handles three cases:
For every case, the transformation works as follows: 1. For a list of Expressions (a projectList or an aggregateExpressions), partitions it two lists of Expressions, one for all WindowExpressions and another for all regular expressions. 2. For all WindowExpressions, groups them based on their WindowSpecDefinitions. 3. For every distinct WindowSpecDefinition, creates a Window operator and inserts it into the plan tree.
Turns projections that contain aggregate expressions into aggregations.
Turns projections that contain aggregate expressions into aggregations.
Correctly handle null primitive inputs for UDF by adding extra If expression to do the null check.
Correctly handle null primitive inputs for UDF by adding extra If expression to do the null check. When user defines a UDF with primitive parameters, there is no way to tell if the primitive parameter is null or not, so here we assume the primitive input is null-propagatable and we should return null if the input is null.
A strategy that only runs once.
A strategy that only runs once.
Pulls out nondeterministic expressions from LogicalPlan which is not Project or Filter, put them into an inner Project and finally project them away at the outer Project.
Pulls out nondeterministic expressions from LogicalPlan which is not Project or Filter, put them into an inner Project and finally project them away at the outer Project.
This rule finds aggregate expressions that are not in an aggregate operator.
This rule finds aggregate expressions that are not in an aggregate operator. For example, those in a HAVING clause or ORDER BY clause. These expressions are pushed down to the underlying aggregate operator and then projected away after the original operator.
Replaces UnresolvedAliass with concrete aliases.
Replaces UnresolvedAliass with concrete aliases.
Replaces UnresolvedFunctions with concrete Expressions.
Replaces UnresolvedFunctions with concrete Expressions.
Rewrites table generating expressions that either need one or more of the following in order to be resolved:
Rewrites table generating expressions that either need one or more of the following in order to be resolved:
Names for the output Attributes are extracted from Alias or MultiAlias expressions that wrap the Generator. If more than one Generator is found in a Project, an AnalysisException is throw.
Replaces UnresolvedAttributes with concrete AttributeReferences from a logical plan node's children.
Replaces UnresolvedAttributes with concrete AttributeReferences from a logical plan node's children.
Replaces UnresolvedRelations with concrete relations from the catalog.
Replaces UnresolvedRelations with concrete relations from the catalog.
In many dialects of SQL it is valid to sort by attributes that are not present in the SELECT clause.
In many dialects of SQL it is valid to sort by attributes that are not present in the SELECT clause. This rule detects such queries and adds the required attributes to the original projection, so that they will be available during sorting. Another projection is added to remove these attributes after sorting.
Substitute child plan with WindowSpecDefinitions.
Substitute child plan with WindowSpecDefinitions.
Defines a sequence of rule batches, to be overridden by the implementation.
Defines a sequence of rule batches, to be overridden by the implementation.
Executes the batches of rules defined by the subclass.
Executes the batches of rules defined by the subclass. The batches are executed serially using the defined execution strategy. Within each batch, rules are also executed serially.
Override to provide additional checks for correct analysis.
Override to provide additional checks for correct analysis. These rules will be evaluated after our built-in check rules.
Override to provide additional rules for the "Resolution" batch.
Override to provide additional rules for the "Resolution" batch.
A trivial Analyzer with an EmptyCatalog and EmptyFunctionRegistry. Used for testing when all relations are already filled in and the analyzer needs only to resolve attribute references.