OffsetWindowFunctionMeta
Related Doc:
package shims
Call this method to record information about type conversions via DataTypeMeta.
Call this method to record information about type conversions via DataTypeMeta.
Returns true iff all of the data writing commands can be replaced.
Returns true iff all of the data writing commands can be replaced.
Returns true iff all of the expressions and their children could be replaced.
Returns true iff all of the expressions and their children could be replaced.
Returns true iff all of the partitioning can be replaced.
Returns true iff all of the partitioning can be replaced.
Returns true iff this could be replaced.
Returns true iff this could be replaced.
The wrapped data writing commands that should be examined
The wrapped data writing commands that should be examined
The wrapped expressions that should be examined
The wrapped expressions that should be examined
The wrapped partitioning that should be examined
The wrapped partitioning that should be examined
The wrapped plans that should be examined
The wrapped plans that should be examined
the config
the config
Keep this on the CPU, but possibly convert its children under it to run on the GPU if enabled.
Keep this on the CPU, but possibly convert its children under it to run on the GPU if enabled. By default this just returns what is wrapped by this. For some types of operators/stages, like SparkPlan, each part of the query can be converted independent of other parts. As such in a subclass this should be overridden to do the correct thing.
Returns the list of reasons the entire plan can't be replaced.
Returns the list of reasons the entire plan can't be replaced. An empty set means the entire plan is ok to be replaced, do the normal checking per exec and children.
Call this if there is a condition found that the entire plan is not allowed to run on the GPU.
Call this if there is a condition found that the entire plan is not allowed to run on the GPU.
Returns true iff this must be replaced because its children have already been replaced and this needs to also be replaced for compatibility.
Returns true iff this must be replaced because its children have already been replaced and this needs to also be replaced for compatibility.
Overrides the data type of the wrapped expression during type checking.
Overrides the data type of the wrapped expression during type checking.
NOTICE: This method will NOT modify the wrapped expression itself. Therefore, the actual transition on data type is still necessary when converting this expression to GPU.
the parent of this node, if there is one.
the parent of this node, if there is one.
Create a string representation of this in append.
Create a string representation of this in append.
where to place the string representation.
how far down the tree this is.
should all the data be printed or just what does not work on the GPU?
Create a string explanation for whether this expression tree can be converted to an AST
Create a string explanation for whether this expression tree can be converted to an AST
where to place the string representation.
how far down the tree this is.
should all the data be printed or just what does not work in the AST?
When converting this to a string should we include the string representation of what this wraps too? This is off by default.
When converting this to a string should we include the string representation of what this wraps too? This is off by default.
Recursively force a section of the plan back onto CPU, stopping once a plan is reached that is already on CPU.
Recursively force a section of the plan back onto CPU, stopping once a plan is reached that is already on CPU.
Returns true if this node should be removed.
Returns true if this node should be removed.
Called to verify that this expression will work on the GPU.
Called to verify that this expression will work on the GPU. For most expressions without extra checks all of the checks should have already been done.
Tag all of the children to see if they are GPU compatible first.
Tag all of the children to see if they are GPU compatible first. Do basic common verification for the operators, and then call tagSelfForGpu
Called to verify that this expression will work as a GPU AST expression.
Called to verify that this expression will work as a GPU AST expression.
Do any extra checks and tag yourself if you are compatible or not.
Do any extra checks and tag yourself if you are compatible or not. Be aware that this may already have been marked as incompatible for a number of reasons.
All of your children should have already been tagged so if there are situations where you may need to disqualify your children for various reasons you may do it here too.
Gets the DataTypeMeta of current BaseExprMeta, which is supposed to be called in the tag methods of expression-level type checks.
Gets the DataTypeMeta of current BaseExprMeta, which is supposed to be called in the tag methods of expression-level type checks.
By default, it simply returns the data type of wrapped expression. But for specific
expressions, they can easily override data type for type checking through calling the
method overrideDataType.
Call this to indicate that this should not be replaced with a GPU enabled version
Call this to indicate that this should not be replaced with a GPU enabled version
why it should not be replaced.
what we are wrapping
what we are wrapping
Spark 3.1.1-specific replacement for com.nvidia.spark.rapids.OffsetWindowFunctionMeta. This is required primarily for two reasons:
expr.inputis not recognized as an Expression.) 2. The semantics of offsets in LAG() are reversed/negated in Spark 3.1.1. E.g. The expressionLAG(col, 5)causes Lag.offset to be set to-5, as opposed to5, in prior versions of Spark. This class adjusts the LAG offset to use similar semantics to Spark 3.0.x.