Class

com.nvidia.spark.rapids

GpuBaseWindowExecMeta

Related Doc: package rapids

Permalink

abstract class GpuBaseWindowExecMeta[WindowExecType <: SparkPlan] extends SparkPlanMeta[WindowExecType] with Logging

Base class for GPU Execs that implement window functions. This abstracts the method by which the window function's input expressions, partition specs, order-by specs, etc. are extracted from the specific WindowExecType.

WindowExecType

The Exec class that implements window functions (E.g. o.a.s.sql.execution.window.WindowExec.)

Linear Supertypes
Logging, SparkPlanMeta[WindowExecType], RapidsMeta[WindowExecType, SparkPlan], AnyRef, Any
Known Subclasses
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. GpuBaseWindowExecMeta
  2. Logging
  3. SparkPlanMeta
  4. RapidsMeta
  5. AnyRef
  6. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new GpuBaseWindowExecMeta(windowExec: WindowExecType, conf: RapidsConf, parent: Option[RapidsMeta[_, _]], rule: DataFromReplacementRule)

    Permalink

Abstract Value Members

  1. abstract def getInputWindowExpressions: Seq[NamedExpression]

    Permalink

    Extracts window-expression from WindowExecType.

    Extracts window-expression from WindowExecType. The implementation varies, depending on the WindowExecType class.

  2. abstract def getOrderSpecs: Seq[SortOrder]

    Permalink

    Extracts order-by spec from WindowExecType.

    Extracts order-by spec from WindowExecType. The implementation varies, depending on the WindowExecType class.

  3. abstract def getPartitionSpecs: Seq[Expression]

    Permalink

    Extracts partition-spec from WindowExecType.

    Extracts partition-spec from WindowExecType. The implementation varies, depending on the WindowExecType class.

  4. abstract def getResultColumnsOnly: Boolean

    Permalink

    Indicates the output column semantics for the WindowExecType, i.e.

    Indicates the output column semantics for the WindowExecType, i.e. whether to only return the window-expression result columns (as in some Spark distributions) or also include the input columns (as in Apache Spark).

Concrete Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. final def addConvertedDataType(expression: Expression, typeMeta: DataTypeMeta): Unit

    Permalink

    Call this method to record information about type conversions via DataTypeMeta.

    Call this method to record information about type conversions via DataTypeMeta.

    Definition Classes
    RapidsMeta
  5. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  6. val availableRuntimeDataTransition: Boolean

    Permalink

    Whether there exists runtime data transition for the wrapped plan, if true, the overriding of output attributes will always work even when the wrapped plan can't be replaced by GPU overrides.

    Whether there exists runtime data transition for the wrapped plan, if true, the overriding of output attributes will always work even when the wrapped plan can't be replaced by GPU overrides.

    Definition Classes
    SparkPlanMeta
  7. def canDataWriteCmdsBeReplaced: Boolean

    Permalink

    Returns true iff all of the data writing commands can be replaced.

    Returns true iff all of the data writing commands can be replaced.

    Definition Classes
    RapidsMeta
  8. def canExprTreeBeReplaced: Boolean

    Permalink

    Returns true iff all of the expressions and their children could be replaced.

    Returns true iff all of the expressions and their children could be replaced.

    Definition Classes
    RapidsMeta
  9. def canPartsBeReplaced: Boolean

    Permalink

    Returns true iff all of the partitioning can be replaced.

    Returns true iff all of the partitioning can be replaced.

    Definition Classes
    RapidsMeta
  10. final def canThisBeReplaced: Boolean

    Permalink

    Returns true iff this could be replaced.

    Returns true iff this could be replaced.

    Definition Classes
    RapidsMeta
  11. var cannotBeReplacedReasons: Option[Set[String]]

    Permalink
    Attributes
    protected
    Definition Classes
    RapidsMeta
  12. var cannotRunOnGpuBecauseOfCost: Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    RapidsMeta
  13. var cannotRunOnGpuBecauseOfSparkPlan: Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    RapidsMeta
  14. def checkExistingTags(): Unit

    Permalink

    When AQE is enabled and we are planning a new query stage, we need to look at meta-data previously stored on the spark plan to determine whether this operator can run on GPU

    When AQE is enabled and we are planning a new query stage, we need to look at meta-data previously stored on the spark plan to determine whether this operator can run on GPU

    Definition Classes
    SparkPlanMeta
  15. def checkTimeZoneId(timeZoneId: Option[String]): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    RapidsMeta
  16. val childDataWriteCmds: Seq[DataWritingCommandMeta[_]]

    Permalink

    The wrapped data writing commands that should be examined

    The wrapped data writing commands that should be examined

    Definition Classes
    SparkPlanMetaRapidsMeta
  17. val childExprs: Seq[BaseExprMeta[_]]

    Permalink

    The wrapped expressions that should be examined

    The wrapped expressions that should be examined

    Definition Classes
    SparkPlanMetaRapidsMeta
  18. val childParts: Seq[PartMeta[_]]

    Permalink

    The wrapped partitioning that should be examined

    The wrapped partitioning that should be examined

    Definition Classes
    SparkPlanMetaRapidsMeta
  19. val childPlans: Seq[SparkPlanMeta[SparkPlan]]

    Permalink

    The wrapped plans that should be examined

    The wrapped plans that should be examined

    Definition Classes
    SparkPlanMetaRapidsMeta
  20. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  21. val conf: RapidsConf

    Permalink

    the config

    the config

    Definition Classes
    RapidsMeta
  22. def confKey: String

    Permalink
    Definition Classes
    RapidsMeta
  23. final def convertIfNeeded(): SparkPlan

    Permalink

    If this is enabled to be converted to a GPU version convert it and return the result, else do what is needed to possibly convert the rest of the plan.

    If this is enabled to be converted to a GPU version convert it and return the result, else do what is needed to possibly convert the rest of the plan.

    Definition Classes
    SparkPlanMeta
  24. def convertToCpu(): SparkPlan

    Permalink

    Keep this on the CPU, but possibly convert its children under it to run on the GPU if enabled.

    Keep this on the CPU, but possibly convert its children under it to run on the GPU if enabled. By default this just returns what is wrapped by this. For some types of operators/stages, like SparkPlan, each part of the query can be converted independent of other parts. As such in a subclass this should be overridden to do the correct thing.

    Definition Classes
    SparkPlanMetaRapidsMeta
  25. final def costPreventsRunningOnGpu(): Unit

    Permalink
    Definition Classes
    RapidsMeta
  26. var cpuCost: Double

    Permalink
    Definition Classes
    SparkPlanMeta
  27. final val disabledMsg: Option[String]

    Permalink
    Definition Classes
    RapidsMeta
  28. final def entirePlanExcludedReasons: Seq[String]

    Permalink

    Returns the list of reasons the entire plan can't be replaced.

    Returns the list of reasons the entire plan can't be replaced. An empty set means the entire plan is ok to be replaced, do the normal checking per exec and children.

    Definition Classes
    RapidsMeta
  29. final def entirePlanWillNotWork(because: String): Unit

    Permalink

    Call this if there is a condition found that the entire plan is not allowed to run on the GPU.

    Call this if there is a condition found that the entire plan is not allowed to run on the GPU.

    Definition Classes
    RapidsMeta
  30. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  31. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  32. var estimatedOutputRows: Option[BigInt]

    Permalink
    Definition Classes
    SparkPlanMeta
  33. def explain(all: Boolean): String

    Permalink
    Definition Classes
    RapidsMeta
  34. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  35. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  36. def getReasonsNotToReplaceEntirePlan: Seq[String]

    Permalink
    Definition Classes
    SparkPlanMeta
  37. var gpuCost: Double

    Permalink
    Definition Classes
    SparkPlanMeta
  38. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  39. final val incompatDoc: Option[String]

    Permalink
    Definition Classes
    RapidsMeta
  40. def indent(append: StringBuilder, depth: Int): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    RapidsMeta
  41. def initReasons(): Unit

    Permalink
    Definition Classes
    RapidsMeta
  42. def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  43. def initializeLogIfNecessary(isInterpreter: Boolean): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  44. lazy val inputFields: Seq[BaseExprMeta[Attribute]]

    Permalink
  45. final def inputFilePreventsRunningOnGpu(): Unit

    Permalink
    Definition Classes
    RapidsMeta
  46. def isDisabledByDefault: Boolean

    Permalink
    Definition Classes
    RapidsMeta
  47. def isIncompat: Boolean

    Permalink
    Definition Classes
    RapidsMeta
  48. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  49. def isTraceEnabled(): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  50. def log: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  51. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  52. def logDebug(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  53. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  54. def logError(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  55. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  56. def logInfo(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  57. def logName: String

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  58. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  59. def logTrace(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  60. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  61. def logWarning(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  62. final def mustBeReplaced(because: String): Unit

    Permalink
    Definition Classes
    RapidsMeta
  63. final def mustThisBeReplaced: Boolean

    Permalink

    Returns true iff this must be replaced because its children have already been replaced and this needs to also be replaced for compatibility.

    Returns true iff this must be replaced because its children have already been replaced and this needs to also be replaced for compatibility.

    Definition Classes
    RapidsMeta
  64. def namedChildExprs: Map[String, Seq[BaseExprMeta[_]]]

    Permalink
    Definition Classes
    GpuBaseWindowExecMetaSparkPlanMeta
  65. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  66. def noReplacementPossibleMessage(reasons: String): String

    Permalink
    Definition Classes
    RapidsMeta
  67. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  68. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  69. final val operationName: String

    Permalink
    Definition Classes
    RapidsMeta
  70. val orderSpec: Seq[BaseExprMeta[SortOrder]]

    Permalink
  71. def outputAttributes: Seq[Attribute]

    Permalink

    Gets output attributes of current SparkPlanMeta, which is supposed to be called during type checking for the current plan.

    Gets output attributes of current SparkPlanMeta, which is supposed to be called during type checking for the current plan.

    By default, it simply returns the output of wrapped plan. For specific plans, they can override outputTypeMetas to apply custom conversions on the output of wrapped plan. For plans which just pass through the schema of childPlan, they can set useOutputAttributesOfChild to true, in order to propagate the custom conversions of childPlan if they exist.

    Definition Classes
    SparkPlanMeta
  72. lazy val outputTypeMetas: Option[Seq[DataTypeMeta]]

    Permalink

    Overrides this method to implement custom conversions for specific plans.

    Overrides this method to implement custom conversions for specific plans.

    Attributes
    protected
    Definition Classes
    SparkPlanMeta
  73. val parent: Option[RapidsMeta[_, _]]

    Permalink

    the parent of this node, if there is one.

    the parent of this node, if there is one.

    Definition Classes
    RapidsMeta
  74. val partitionSpec: Seq[BaseExprMeta[Expression]]

    Permalink
  75. def print(strBuilder: StringBuilder, depth: Int, all: Boolean): Unit

    Permalink

    Create a string representation of this in append.

    Create a string representation of this in append.

    strBuilder

    where to place the string representation.

    depth

    how far down the tree this is.

    all

    should all the data be printed or just what does not work on the GPU?

    Attributes
    protected
    Definition Classes
    RapidsMeta
  76. val printWrapped: Boolean

    Permalink

    When converting this to a string should we include the string representation of what this wraps too? This is off by default.

    When converting this to a string should we include the string representation of what this wraps too? This is off by default.

    Attributes
    protected
    Definition Classes
    RapidsMeta
  77. final def recursiveCostPreventsRunningOnGpu(): Unit

    Permalink

    Recursively force a section of the plan back onto CPU, stopping once a plan is reached that is already on CPU.

    Recursively force a section of the plan back onto CPU, stopping once a plan is reached that is already on CPU.

    Definition Classes
    RapidsMeta
  78. final def recursiveSparkPlanPreventsRunningOnGpu(): Unit

    Permalink
    Definition Classes
    RapidsMeta
  79. final def recursiveSparkPlanRemoved(): Unit

    Permalink
    Definition Classes
    RapidsMeta
  80. def replaceMessage: String

    Permalink
    Definition Classes
    RapidsMeta
  81. def requireAstForGpuOn(exprMeta: BaseExprMeta[_]): Unit

    Permalink
    Definition Classes
    SparkPlanMeta
  82. def runAfterTagRules(): Unit

    Permalink

    Run rules that happen for the entire tree after it has been tagged initially.

    Run rules that happen for the entire tree after it has been tagged initially.

    Definition Classes
    SparkPlanMeta
  83. final def shouldBeRemoved(because: String): Unit

    Permalink
    Definition Classes
    RapidsMeta
  84. final def shouldThisBeRemoved: Boolean

    Permalink

    Returns true if this node should be removed.

    Returns true if this node should be removed.

    Definition Classes
    RapidsMeta
  85. def suppressWillWorkOnGpuInfo: Boolean

    Permalink
    Definition Classes
    RapidsMeta
  86. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  87. def tagForExplain(): Unit

    Permalink
    Definition Classes
    SparkPlanMeta
  88. final def tagForGpu(): Unit

    Permalink

    Tag all of the children to see if they are GPU compatible first.

    Tag all of the children to see if they are GPU compatible first. Do basic common verification for the operators, and then call tagSelfForGpu

    Definition Classes
    RapidsMeta
  89. def tagPlanForGpu(): Unit

    Permalink

    Called to verify that this plan will work on the GPU.

    Called to verify that this plan will work on the GPU. Generic checks will have already been done. In general this method should only tag this operator as bad. If it needs to tag one of its children please take special care to update the comment inside tagSelfForGpu so we don't end up with something that could be cyclical.

    Definition Classes
    GpuBaseWindowExecMetaSparkPlanMeta
  90. final def tagSelfForGpu(): Unit

    Permalink

    Do any extra checks and tag yourself if you are compatible or not.

    Do any extra checks and tag yourself if you are compatible or not. Be aware that this may already have been marked as incompatible for a number of reasons.

    All of your children should have already been tagged so if there are situations where you may need to disqualify your children for various reasons you may do it here too.

    Definition Classes
    SparkPlanMetaRapidsMeta
  91. def toString(): String

    Permalink
    Definition Classes
    RapidsMeta → AnyRef → Any
  92. val useOutputAttributesOfChild: Boolean

    Permalink

    Whether to pass through the outputAttributes of childPlan's meta, only for UnaryPlan

    Whether to pass through the outputAttributes of childPlan's meta, only for UnaryPlan

    Attributes
    protected
    Definition Classes
    SparkPlanMeta
  93. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  94. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  95. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  96. final def willNotWorkOnGpu(because: String): Unit

    Permalink

    Call this to indicate that this should not be replaced with a GPU enabled version

    Call this to indicate that this should not be replaced with a GPU enabled version

    because

    why it should not be replaced.

    Definition Classes
    RapidsMeta
  97. val windowExpressions: Seq[BaseExprMeta[NamedExpression]]

    Permalink
  98. val wrapped: WindowExecType

    Permalink

    what we are wrapping

    what we are wrapping

    Definition Classes
    RapidsMeta

Inherited from Logging

Inherited from SparkPlanMeta[WindowExecType]

Inherited from RapidsMeta[WindowExecType, SparkPlan]

Inherited from AnyRef

Inherited from Any

Ungrouped