org.apache.spark.sql.execution
OptimizeMetadataOnlyQuery
Related Doc:
package execution
A pattern that finds the partitioned table relation node inside the given plan, and returns a pair of the partition attributes and the table relation node.
A pattern that finds the partitioned table relation node inside the given plan, and returns a pair of the partition attributes and the table relation node.
It keeps traversing down the given plan tree if there is a Project or Filter with deterministic expressions, and returns result after reaching the partitioned table relation node.
This rule optimizes the execution of queries that can be answered by looking only at partition-level metadata. This applies when all the columns scanned are partition columns, and the query has an aggregate operator that satisfies the following conditions: 1. aggregate expression is partition columns. e.g. SELECT col FROM tbl GROUP BY col. 2. aggregate function on partition columns with DISTINCT. e.g. SELECT col1, count(DISTINCT col2) FROM tbl GROUP BY col1. 3. aggregate function on partition columns which have same result w or w/o DISTINCT keyword. e.g. SELECT col1, Max(col2) FROM tbl GROUP BY col1.