rapids

Type Members

1. abstract class AggExprMeta[INPUT <: AggregateFunction] extends ExprMeta[INPUT]

   

   Base class for metadata around AggregateFunction.

2. abstract class BaseExprMeta[INPUT <: Expression] extends RapidsMeta[INPUT, Expression]

   

   Base class for metadata around Expression.

3. abstract class BinaryAstExprMeta[INPUT <: BinaryExpression] extends BinaryExprMeta[INPUT]

   

   Base metadata class for binary expressions that support conversion to AST

4. abstract class BinaryExprMeta[INPUT <: BinaryExpression] extends ExprMeta[INPUT]

   

   Base class for metadata around BinaryExpression.

5. class CastChecks extends ExprChecks

   

6. final class CastExprMeta[INPUT <: Cast] extends UnaryExprMeta[INPUT]

   

   Meta-data for cast and ansi_cast.

7. abstract class ComplexTypeMergingExprMeta[INPUT <: ComplexTypeMergingExpression] extends ExprMeta[INPUT]

   

   Base class for metadata around ComplexTypeMergingExpression.

8. class ConfBuilder extends AnyRef

   

9. abstract class ConfEntry[T] extends AnyRef

   

10. class ConfEntryWithDefault[T] extends ConfEntry[T]

    

11. case class ContextChecks(outputCheck: TypeSig, sparkOutputSig: TypeSig, paramCheck: Seq[ParamCheck] = Seq.empty, repeatingParamCheck: Option[RepeatingParamCheck] = None) extends TypeChecks[Map[String, SupportLevel]] with Product with Serializable

    

    Checks an expression that have input parameters and a single output.

    Checks an expression that have input parameters and a single output. This is intended to be given for a specific ExpressionContext. If your expression does not meet this pattern you may need to create a custom ExprChecks instance.

12. final class CreateDataSourceTableAsSelectCommandMeta extends DataWritingCommandMeta[CreateDataSourceTableAsSelectCommand]

    

13. class CudfRegexTranspiler extends AnyRef

    

    Transpile Java/Spark regular expression to a format that cuDF supports, or throw an exception if this is not possible.

14. trait DataFromReplacementRule extends AnyRef

    

15. class DataTypeMeta extends AnyRef

    

    The metadata around DataType, which records the original data type, the desired data type for GPU overrides, and the reason of potential conversion.

    The metadata around DataType, which records the original data type, the desired data type for GPU overrides, and the reason of potential conversion. The metadata is to ensure TypeChecks tagging the actual data types for GPU runtime, since data types of GPU overrides may slightly differ from original CPU counterparts.

16. abstract class DataWritingCommandMeta[INPUT <: DataWritingCommand] extends RapidsMeta[INPUT, DataWritingCommand]

    

    Base class for metadata around DataWritingCommand.

17. class DataWritingCommandRule[INPUT <: DataWritingCommand] extends ReplacementRule[INPUT, DataWritingCommand, DataWritingCommandMeta[INPUT]]

    

    Holds everything that is needed to replace a DataWritingCommand with a GPU enabled version.

18. final class DoNotReplaceOrWarnSparkPlanMeta[INPUT <: SparkPlan] extends SparkPlanMeta[INPUT]

    

    Metadata for SparkPlan that should not be replaced or have any kind of warning for

19. class ExecChecks extends TypeChecks[Map[String, SupportLevel]]

    

    Checks the input and output types supported by a SparkPlan node.

    Checks the input and output types supported by a SparkPlan node. We don't currently separate input checks from output checks. We can add this in if something needs it.

    The namedChecks map can be used to provide checks for specific groups of expressions.

20. class ExecRule[INPUT <: SparkPlan] extends ReplacementRule[INPUT, SparkPlan, SparkPlanMeta[INPUT]]

    

    Holds everything that is needed to replace a SparkPlan with a GPU enabled version.

21. trait ExplainPlanBase extends AnyRef

    

22. abstract class ExprChecks extends TypeChecks[Map[ExpressionContext, Map[String, SupportLevel]]]

    

    Base class all Expression checks must follow.

23. case class ExprChecksImpl(contexts: Map[ExpressionContext, ContextChecks]) extends ExprChecks with Product with Serializable

    

24. abstract class ExprMeta[INPUT <: Expression] extends BaseExprMeta[INPUT]

    

25. class ExprRule[INPUT <: Expression] extends ReplacementRule[INPUT, Expression, BaseExprMeta[INPUT]]

    

    Holds everything that is needed to replace an Expression with a GPU enabled version.

26. sealed abstract class ExpressionContext extends AnyRef

    

27. class FileFormatChecks extends TypeChecks[SupportLevel]

    

    Checks for either a read or a write of a given file format.

28. sealed trait FileFormatOp extends AnyRef

    

29. sealed trait FileFormatType extends AnyRef

    

30. abstract class GeneratorExprMeta[INPUT <: Generator] extends ExprMeta[INPUT]

    

31. abstract class GpuBaseWindowExecMeta[WindowExecType <: SparkPlan] extends SparkPlanMeta[WindowExecType] with Logging

    

    Base class for GPU Execs that implement window functions.

    Base class for GPU Execs that implement window functions. This abstracts the method by which the window function's input expressions, partition specs, order-by specs, etc. are extracted from the specific WindowExecType.

    WindowExecType

    The Exec class that implements window functions (E.g. o.a.s.sql.execution.window.WindowExec.)

32. abstract class GpuBroadcastJoinMeta[INPUT <: SparkPlan] extends SparkPlanMeta[INPUT]

    

33. class GpuExpandExecMeta extends SparkPlanMeta[ExpandExec]

    

34. class GpuGenerateExecSparkPlanMeta extends SparkPlanMeta[GenerateExec]

    

35. class GpuHashAggregateMeta extends SparkPlanMeta[HashAggregateExec]

    

36. trait GpuOverridesListener extends AnyRef

    

37. class GpuProjectExecMeta extends SparkPlanMeta[ProjectExec] with Logging

    

38. class GpuSampleExecMeta extends SparkPlanMeta[SampleExec] with Logging

    

39. class GpuSortAggregateExecMeta extends SparkPlanMeta[SortAggregateExec]

    

40. class GpuSortMeta extends SparkPlanMeta[SortExec]

    

41. class GpuWindowExecMeta extends GpuBaseWindowExecMeta[WindowExec]

    

    Specialization of GpuBaseWindowExecMeta for org.apache.spark.sql.window.WindowExec.

    Specialization of GpuBaseWindowExecMeta for org.apache.spark.sql.window.WindowExec. This class implements methods to extract the window-expressions, partition columns, order-by columns, etc. from WindowExec.

42. abstract class ImperativeAggExprMeta[INPUT <: ImperativeAggregate] extends AggExprMeta[INPUT]

    

    Base class for metadata around ImperativeAggregate.

43. case class InputCheck(cudf: TypeSig, spark: TypeSig, notes: List[String] = List.empty) extends Product with Serializable

    

    Checks a set of named inputs to an SparkPlan node against a TypeSig

44. final class InsertIntoHadoopFsRelationCommandMeta extends DataWritingCommandMeta[InsertIntoHadoopFsRelationCommand]

    

45. class LiteralExprMeta extends ExprMeta[Literal]

    

46. final class NoRuleDataFromReplacementRule extends DataFromReplacementRule

    

    A version of DataFromReplacementRule that is used when no replacement rule can be found.

47. sealed abstract class Optimization extends AnyRef

    

48. class OptionalConfEntry[T] extends ConfEntry[Option[T]]

    

49. case class ParamCheck(name: String, cudf: TypeSig, spark: TypeSig) extends Product with Serializable

    

    Checks a single parameter by position against a TypeSig

50. abstract class PartChecks extends TypeChecks[Map[String, SupportLevel]]

    

    Base class all Partition checks must follow

51. case class PartChecksImpl(paramCheck: Seq[ParamCheck] = Seq.empty, repeatingParamCheck: Option[RepeatingParamCheck] = None) extends PartChecks with Product with Serializable

    

52. abstract class PartMeta[INPUT <: Partitioning] extends RapidsMeta[INPUT, Partitioning]

    

    Base class for metadata around Partitioning.

53. class PartRule[INPUT <: Partitioning] extends ReplacementRule[INPUT, Partitioning, PartMeta[INPUT]]

    

    Holds everything that is needed to replace a Partitioning with a GPU enabled version.

54. class PartiallySupported extends SupportLevel

    

    The plugin partially supports this type.

55. sealed case class QuantifierFixedLength(length: Int) extends RegexQuantifier with Product with Serializable

    

56. sealed case class QuantifierVariableLength(minLength: Int, maxLength: Option[Int]) extends RegexQuantifier with Product with Serializable

    

57. class RapidsConf extends Logging

    

58. abstract class RapidsMeta[INPUT <: BASE, BASE] extends AnyRef

    

    Holds metadata about a stage in the physical plan that is separate from the plan itself.

    Holds metadata about a stage in the physical plan that is separate from the plan itself. This is helpful in deciding when to replace part of the plan with a GPU enabled version.

    INPUT

    the exact type of the class we are wrapping.

    BASE

    the generic base class for this type of stage, i.e. SparkPlan, Expression, etc.

59. sealed trait RegexAST extends AnyRef

    

60. sealed case class RegexChar(a: Char) extends RegexCharacterClassComponent with Product with Serializable

    

61. sealed case class RegexCharacterClass(negated: Boolean, characters: ListBuffer[RegexCharacterClassComponent]) extends RegexAST with Product with Serializable

    

62. sealed trait RegexCharacterClassComponent extends RegexAST

    

63. sealed case class RegexCharacterRange(start: Char, end: Char) extends RegexCharacterClassComponent with Product with Serializable

    

64. sealed case class RegexChoice(a: RegexAST, b: RegexAST) extends RegexAST with Product with Serializable

    

65. sealed case class RegexEscaped(a: Char) extends RegexCharacterClassComponent with Product with Serializable

    

66. sealed case class RegexGroup(capture: Boolean, term: RegexAST) extends RegexAST with Product with Serializable

    

67. sealed case class RegexHexDigit(a: String) extends RegexCharacterClassComponent with Product with Serializable

    

68. sealed case class RegexOctalChar(a: String) extends RegexCharacterClassComponent with Product with Serializable

    

69. class RegexParser extends AnyRef

    

    Regular expression parser based on a Pratt Parser design.

    Regular expression parser based on a Pratt Parser design.

    The goal of this parser is to build a minimal AST that allows us to validate that we can support the expression on the GPU. The goal is not to parse with the level of detail that would be required if we were building an evaluation engine. For example, operator precedence is largely ignored but could be added if we need it later.

    The Java and cuDF regular expression documentation has been used as a reference:

    Java regex: https://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html cuDF regex: https://docs.rapids.ai/api/libcudf/stable/md_regex.html

    The following blog posts provide some background on Pratt Parsers and parsing regex.

    - https://journal.stuffwithstuff.com/2011/03/19/pratt-parsers-expression-parsing-made-easy/ - https://matt.might.net/articles/parsing-regex-with-recursive-descent/

70. sealed trait RegexQuantifier extends RegexAST

    

71. sealed case class RegexRepetition(a: RegexAST, quantifier: RegexQuantifier) extends RegexAST with Product with Serializable

    

72. sealed case class RegexSequence(parts: ListBuffer[RegexAST]) extends RegexAST with Product with Serializable

    

73. class RegexUnsupportedException extends SQLException

    

74. case class RepeatingParamCheck(name: String, cudf: TypeSig, spark: TypeSig) extends Product with Serializable

    

    Checks the type signature for a parameter that repeats (Can only be used at the end of a list of position parameters)

75. abstract class ReplacementRule[INPUT <: BASE, BASE, WRAP_TYPE <: RapidsMeta[INPUT, BASE]] extends DataFromReplacementRule

    

    Base class for all ReplacementRules

    Base class for all ReplacementRules

    INPUT

    the exact type of the class we are wrapping.

    BASE

    the generic base class for this type of stage, i.e. SparkPlan, Expression, etc.

    WRAP_TYPE

    base class that should be returned by doWrap.

76. abstract class ReplicateRowsExprMeta[INPUT <: ReplicateRows] extends GeneratorExprMeta[INPUT]

    

    Base class for metadata around GeneratorExprMeta.

77. final class RuleNotFoundDataWritingCommandMeta[INPUT <: DataWritingCommand] extends DataWritingCommandMeta[INPUT]

    

    Metadata for DataWritingCommand with no rule found

78. final class RuleNotFoundExprMeta[INPUT <: Expression] extends ExprMeta[INPUT]

    

    Metadata for Expression with no rule found

79. final class RuleNotFoundPartMeta[INPUT <: Partitioning] extends PartMeta[INPUT]

    

    Metadata for Partitioning with no rule found

80. final class RuleNotFoundSparkPlanMeta[INPUT <: SparkPlan] extends SparkPlanMeta[INPUT]

    

    Metadata for SparkPlan with no rule found

81. sealed case class SimpleQuantifier(ch: Char) extends RegexQuantifier with Product with Serializable

    

82. sealed trait SortExecType extends Serializable

    

83. abstract class SparkPlanMeta[INPUT <: SparkPlan] extends RapidsMeta[INPUT, SparkPlan]

    

    Base class for metadata around SparkPlan.

84. abstract class String2TrimExpressionMeta[INPUT <: String2TrimExpression] extends ExprMeta[INPUT]

    

85. sealed abstract class SupportLevel extends AnyRef

    

    The level of support that the plugin has for a given type.

    The level of support that the plugin has for a given type. Used for documentation generation.

86. class Supported extends SupportLevel

    

    Both Spark and the plugin support this.

87. abstract class TernaryExprMeta[INPUT <: TernaryExpression] extends ExprMeta[INPUT]

    

    Base class for metadata around TernaryExpression.

88. abstract class TypeChecks[RET] extends AnyRef

    

89. final class TypeSig extends AnyRef

    

    A type signature.

    A type signature. This is a bit limited in what it supports right now, but can express a set of base types and a separate set of types that can be nested under the base types (child types). It can also express if a particular base type has to be a literal or not.

90. trait TypeSigUtilBase extends AnyRef

    

    Trait of TypeSigUtil for different spark versions

91. class TypedConfBuilder[T] extends AnyRef

    

92. abstract class TypedImperativeAggExprMeta[INPUT <: TypedImperativeAggregate[_]] extends ImperativeAggExprMeta[INPUT]

    

    Base class for metadata around TypedImperativeAggregate.

93. abstract class UnaryAstExprMeta[INPUT <: UnaryExpression] extends UnaryExprMeta[INPUT]

    

    Base metadata class for unary expressions that support conversion to AST as well

94. abstract class UnaryExprMeta[INPUT <: UnaryExpression] extends ExprMeta[INPUT]

    

    Base class for metadata around UnaryExpression.