Trait

io.projectglow.sql.expressions

AggregateByIndex

Related Doc: package expressions

Permalink

trait AggregateByIndex extends DeclarativeAggregate with HigherOrderFunction

An expression that allows users to aggregate over all array elements at a specific index in an array column. For example, this expression can be used to compute per-sample summary statistics from a genotypes column.

The user must provide the following arguments: - The array for aggregation - The initialValue for each element in the per-index buffer - An update function to update the buffer with a new element - A merge function to combine two buffers

The user may optionally provide an evaluate function. If it's not provided, the identity function is used.

Example usage to calculate average depth across all sites for a sample: aggregate_by_index( genotypes, named_struct('sum', 0l, 'count', 0l), (buf, genotype) -> named_struct('sum', buf.sum + genotype.depth, 'count', buf.count + 1), (buf1, buf2) -> named_struct('sum', buf1.sum + buf2.sum, 'count', buf1.count + buf2.count), buf -> buf.sum / buf.count)

Linear Supertypes
HigherOrderFunction, ExpectsInputTypes, DeclarativeAggregate, Unevaluable, Serializable, Serializable, AggregateFunction, Expression, TreeNode[Expression], Product, Equals, AnyRef, Any
Known Subclasses
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. AggregateByIndex
  2. HigherOrderFunction
  3. ExpectsInputTypes
  4. DeclarativeAggregate
  5. Unevaluable
  6. Serializable
  7. Serializable
  8. AggregateFunction
  9. Expression
  10. TreeNode
  11. Product
  12. Equals
  13. AnyRef
  14. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. implicit class RichAttribute extends AnyRef

    Permalink
    Definition Classes
    DeclarativeAggregate

Abstract Value Members

  1. abstract def arr: Expression

    Permalink

    The array over which we're aggregating

  2. abstract def canEqual(that: Any): Boolean

    Permalink
    Definition Classes
    Equals
  3. abstract def evaluate: Expression

    Permalink

    Function to turn a buffer into the actual output

  4. abstract def initialValue: Expression

    Permalink

    The initial value for each element in the aggregation buffer

  5. abstract def merge: Expression

    Permalink

    Function to merge two buffers' elements

  6. abstract def productArity: Int

    Permalink
    Definition Classes
    Product
  7. abstract def productElement(n: Int): Any

    Permalink
    Definition Classes
    Product
  8. abstract def update: Expression

    Permalink

    Function to update an element of the buffer with a new input element

  9. abstract def withBoundExprs(newUpdate: Expression, newMerge: Expression, newEvaluate: Expression): AggregateByIndex

    Permalink

Concrete Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. def aggBufferAttributes: Seq[AttributeReference]

    Permalink
    Definition Classes
    AggregateByIndex → AggregateFunction
  5. final def aggBufferSchema: StructType

    Permalink
    Definition Classes
    DeclarativeAggregate → AggregateFunction
  6. def apply(number: Int): TreeNode[_]

    Permalink
    Definition Classes
    TreeNode
  7. def argString: String

    Permalink
    Definition Classes
    TreeNode
  8. def argumentTypes: Seq[ADT]

    Permalink
    Definition Classes
    AggregateByIndex → HigherOrderFunction
  9. def arguments: Seq[Expression]

    Permalink
    Definition Classes
    AggregateByIndex → HigherOrderFunction
  10. lazy val argumentsResolved: Boolean

    Permalink
    Definition Classes
    HigherOrderFunction
  11. def asCode: String

    Permalink
    Definition Classes
    TreeNode
  12. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  13. def bind(f: (Expression, Seq[(DataType, Boolean)]) ⇒ LambdaFunction): AggregateByIndex

    Permalink

    To create bound lambda functions, we bind each of the child higher order functions, extract their bound lambda functions, and then copy.

    To create bound lambda functions, we bind each of the child higher order functions, extract their bound lambda functions, and then copy.

    Definition Classes
    AggregateByIndex → HigherOrderFunction
  14. val buffer: AttributeReference

    Permalink
    Attributes
    protected
  15. lazy val canonicalized: Expression

    Permalink
    Definition Classes
    Expression
  16. def checkArgumentDataTypes(): TypeCheckResult

    Permalink
    Definition Classes
    HigherOrderFunction
  17. def checkInputDataTypes(): TypeCheckResult

    Permalink
    Definition Classes
    ExpectsInputTypes → Expression
  18. def children: Seq[Expression]

    Permalink
    Definition Classes
    HigherOrderFunction → TreeNode
  19. def childrenResolved: Boolean

    Permalink
    Definition Classes
    Expression
  20. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  21. def collect[B](pf: PartialFunction[Expression, B]): Seq[B]

    Permalink
    Definition Classes
    TreeNode
  22. def collectFirst[B](pf: PartialFunction[Expression, B]): Option[B]

    Permalink
    Definition Classes
    TreeNode
  23. def collectLeaves(): Seq[Expression]

    Permalink
    Definition Classes
    TreeNode
  24. lazy val containsChild: Set[TreeNode[_]]

    Permalink
    Definition Classes
    TreeNode
  25. def dataType: DataType

    Permalink
    Definition Classes
    AggregateByIndex → Expression
  26. def defaultResult: Option[Literal]

    Permalink
    Definition Classes
    AggregateFunction
  27. lazy val deterministic: Boolean

    Permalink
    Definition Classes
    Expression
  28. final def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode

    Permalink
    Attributes
    protected
    Definition Classes
    Unevaluable → Expression
  29. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  30. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  31. final def eval(input: InternalRow): Any

    Permalink
    Definition Classes
    Unevaluable → Expression
  32. lazy val evaluateExpression: Expression

    Permalink
    Definition Classes
    AggregateByIndex → DeclarativeAggregate
  33. def fastEquals(other: TreeNode[_]): Boolean

    Permalink
    Definition Classes
    TreeNode
  34. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  35. def find(f: (Expression) ⇒ Boolean): Option[Expression]

    Permalink
    Definition Classes
    TreeNode
  36. def flatArguments: Iterator[Any]

    Permalink
    Attributes
    protected
    Definition Classes
    Expression
  37. def flatMap[A](f: (Expression) ⇒ TraversableOnce[A]): Seq[A]

    Permalink
    Definition Classes
    TreeNode
  38. final def foldable: Boolean

    Permalink
    Definition Classes
    AggregateFunction → Expression
  39. def foreach(f: (Expression) ⇒ Unit): Unit

    Permalink
    Definition Classes
    TreeNode
  40. def foreachUp(f: (Expression) ⇒ Unit): Unit

    Permalink
    Definition Classes
    TreeNode
  41. def functionTypes: Seq[ADT]

    Permalink
    Definition Classes
    AggregateByIndex → HigherOrderFunction
  42. def functions: Seq[Expression]

    Permalink
    Definition Classes
    AggregateByIndex → HigherOrderFunction
  43. lazy val functionsForEval: Seq[Expression]

    Permalink
    Definition Classes
    HigherOrderFunction
  44. def genCode(ctx: CodegenContext): ExprCode

    Permalink
    Definition Classes
    Expression
  45. def generateTreeString(depth: Int, lastChildren: Seq[Boolean], builder: StringBuilder, verbose: Boolean, prefix: String, addSuffix: Boolean): StringBuilder

    Permalink
    Definition Classes
    TreeNode
  46. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  47. def hashCode(): Int

    Permalink
    Definition Classes
    TreeNode → AnyRef → Any
  48. lazy val initialValues: Seq[Expression]

    Permalink
    Definition Classes
    AggregateByIndex → DeclarativeAggregate
  49. def innerChildren: Seq[TreeNode[_]]

    Permalink
    Attributes
    protected
    Definition Classes
    TreeNode
  50. final lazy val inputAggBufferAttributes: Seq[AttributeReference]

    Permalink
    Definition Classes
    DeclarativeAggregate → AggregateFunction
  51. def inputTypes: Seq[AbstractDataType]

    Permalink
    Definition Classes
    HigherOrderFunction → ExpectsInputTypes
  52. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  53. def jsonFields: List[JField]

    Permalink
    Attributes
    protected
    Definition Classes
    TreeNode
  54. def makeCopy(newArgs: Array[AnyRef]): Expression

    Permalink
    Definition Classes
    TreeNode
  55. def map[A](f: (Expression) ⇒ A): Seq[A]

    Permalink
    Definition Classes
    TreeNode
  56. def mapChildren(f: (Expression) ⇒ Expression): Expression

    Permalink
    Definition Classes
    TreeNode
  57. def mapProductIterator[B](f: (Any) ⇒ B)(implicit arg0: ClassTag[B]): Array[B]

    Permalink
    Attributes
    protected
    Definition Classes
    TreeNode
  58. lazy val mergeExpressions: Seq[Expression]

    Permalink
    Definition Classes
    AggregateByIndex → DeclarativeAggregate
  59. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  60. def nodeName: String

    Permalink
    Definition Classes
    TreeNode
  61. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  62. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  63. def nullable: Boolean

    Permalink
    Definition Classes
    AggregateByIndex → HigherOrderFunction → Expression
  64. def numberedTreeString: String

    Permalink
    Definition Classes
    TreeNode
  65. val origin: Origin

    Permalink
    Definition Classes
    TreeNode
  66. def otherCopyArgs: Seq[AnyRef]

    Permalink
    Attributes
    protected
    Definition Classes
    TreeNode
  67. def p(number: Int): Expression

    Permalink
    Definition Classes
    TreeNode
  68. def prettyJson: String

    Permalink
    Definition Classes
    TreeNode
  69. def prettyName: String

    Permalink
    Definition Classes
    AggregateByIndex → Expression
  70. def productIterator: Iterator[Any]

    Permalink
    Definition Classes
    Product
  71. def productPrefix: String

    Permalink
    Definition Classes
    Product
  72. def references: AttributeSet

    Permalink
    Definition Classes
    Expression
  73. lazy val resolved: Boolean

    Permalink
    Definition Classes
    HigherOrderFunction → Expression
  74. def semanticEquals(other: Expression): Boolean

    Permalink
    Definition Classes
    Expression
  75. def semanticHash(): Int

    Permalink
    Definition Classes
    Expression
  76. def simpleString: String

    Permalink
    Definition Classes
    Expression → TreeNode
  77. def sql(isDistinct: Boolean): String

    Permalink
    Definition Classes
    AggregateFunction
  78. def sql: String

    Permalink
    Definition Classes
    Expression
  79. def stringArgs: Iterator[Any]

    Permalink
    Attributes
    protected
    Definition Classes
    TreeNode
  80. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  81. def toAggString(isDistinct: Boolean): String

    Permalink
    Definition Classes
    AggregateFunction
  82. def toAggregateExpression(isDistinct: Boolean): AggregateExpression

    Permalink
    Definition Classes
    AggregateFunction
  83. def toAggregateExpression(): AggregateExpression

    Permalink
    Definition Classes
    AggregateFunction
  84. def toJSON: String

    Permalink
    Definition Classes
    TreeNode
  85. def toString(): String

    Permalink
    Definition Classes
    Expression → TreeNode → AnyRef → Any
  86. def transform(rule: PartialFunction[Expression, Expression]): Expression

    Permalink
    Definition Classes
    TreeNode
  87. def transformDown(rule: PartialFunction[Expression, Expression]): Expression

    Permalink
    Definition Classes
    TreeNode
  88. def transformUp(rule: PartialFunction[Expression, Expression]): Expression

    Permalink
    Definition Classes
    TreeNode
  89. def treeString(verbose: Boolean, addSuffix: Boolean): String

    Permalink
    Definition Classes
    TreeNode
  90. def treeString: String

    Permalink
    Definition Classes
    TreeNode
  91. lazy val updateExpressions: Seq[Expression]

    Permalink
    Definition Classes
    AggregateByIndex → DeclarativeAggregate
  92. final def verboseString: String

    Permalink
    Definition Classes
    Expression → TreeNode
  93. def verboseStringWithSuffix: String

    Permalink
    Definition Classes
    TreeNode
  94. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  95. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  96. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  97. def withNewChildren(newChildren: Seq[Expression]): Expression

    Permalink
    Definition Classes
    TreeNode

Inherited from HigherOrderFunction

Inherited from ExpectsInputTypes

Inherited from DeclarativeAggregate

Inherited from Unevaluable

Inherited from Serializable

Inherited from Serializable

Inherited from AggregateFunction

Inherited from Expression

Inherited from TreeNode[Expression]

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped