Spark 3.1.1-specific replacement for com.nvidia.spark.rapids.OffsetWindowFunctionMeta.
Spark 3.1.1-specific replacement for com.nvidia.spark.rapids.OffsetWindowFunctionMeta.
This is required primarily for two reasons:
com.nvidia.spark.rapids.OffsetWindowFunctionMeta (compiled against Spark 3.0.x)
fails class load in Spark 3.1.x. (expr.input is not recognized as an Expression.)
2. The semantics of offsets in LAG() are reversed/negated in Spark 3.1.1.
E.g. The expression LAG(col, 5) causes Lag.offset to be set to -5,
as opposed to 5, in prior versions of Spark.
This class adjusts the LAG offset to use similar semantics to Spark 3.0.x.
Spark 3.1.1-specific replacement for com.nvidia.spark.rapids.OffsetWindowFunctionMeta. This is required primarily for two reasons:
expr.input
is not recognized as an Expression.) 2. The semantics of offsets in LAG() are reversed/negated in Spark 3.1.1. E.g. The expressionLAG(col, 5)
causes Lag.offset to be set to-5
, as opposed to5
, in prior versions of Spark. This class adjusts the LAG offset to use similar semantics to Spark 3.0.x.