object aggregate extends AggregateFunctions
- Alphabetic
- By Inheritance
- aggregate
- AggregateFunctions
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- def approxCountDistinct[T](column: TypedColumn[T, _], rsd: Double): TypedAggregate[T, Long]
Aggregate function: returns the approximate number of distinct items in a group.
Aggregate function: returns the approximate number of distinct items in a group.
- rsd
maximum estimation error allowed (default = 0.05) apache/spark
- Definition Classes
- AggregateFunctions
- def approxCountDistinct[T](column: TypedColumn[T, _]): TypedAggregate[T, Long]
Aggregate function: returns the approximate number of distinct items in a group.
Aggregate function: returns the approximate number of distinct items in a group.
- Definition Classes
- AggregateFunctions
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def avg[A, T, Out](column: TypedColumn[T, A])(implicit averageable: CatalystAverageable[A, Out], oencoder: TypedEncoder[Out]): TypedAggregate[T, Out]
Aggregate function: returns the average of the values in a group.
Aggregate function: returns the average of the values in a group.
apache/spark
- Definition Classes
- AggregateFunctions
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- def collectList[T, A](column: TypedColumn[T, A])(implicit arg0: TypedEncoder[A]): TypedAggregate[T, Vector[A]]
Aggregate function: returns a list of objects with duplicates.
Aggregate function: returns a list of objects with duplicates.
apache/spark
- Definition Classes
- AggregateFunctions
- def collectSet[T, A](column: TypedColumn[T, A])(implicit arg0: TypedEncoder[A]): TypedAggregate[T, Vector[A]]
Aggregate function: returns a set of objects with duplicate elements eliminated.
Aggregate function: returns a set of objects with duplicate elements eliminated.
apache/spark
- Definition Classes
- AggregateFunctions
- def corr[A, B, T](column1: TypedColumn[T, A], column2: TypedColumn[T, B])(implicit i0: CatalystCast[A, Double], i1: CatalystCast[B, Double]): TypedAggregate[T, Option[Double]]
Aggregate function: returns the Pearson Correlation Coefficient for two columns.
Aggregate function: returns the Pearson Correlation Coefficient for two columns.
- Definition Classes
- AggregateFunctions
- Note
In Spark corr always returns Double https://github.com/apache/spark/blob/4a3c09601ba69f7d49d1946bb6f20f5cfe453031/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Corr.scala#L95 apache/spark
- def count[T](column: TypedColumn[T, _]): TypedAggregate[T, Long]
Aggregate function: returns the number of items in a group for which the selected column is not null.
Aggregate function: returns the number of items in a group for which the selected column is not null.
apache/spark
- Definition Classes
- AggregateFunctions
- def count[T](): TypedAggregate[T, Long]
Aggregate function: returns the number of items in a group.
Aggregate function: returns the number of items in a group.
apache/spark
- Definition Classes
- AggregateFunctions
- def countDistinct[T](column: TypedColumn[T, _]): TypedAggregate[T, Long]
Aggregate function: returns the number of distinct items in a group.
Aggregate function: returns the number of distinct items in a group.
apache/spark
- Definition Classes
- AggregateFunctions
- def covarPop[A, B, T](column1: TypedColumn[T, A], column2: TypedColumn[T, B])(implicit i0: CatalystCast[A, Double], i1: CatalystCast[B, Double]): TypedAggregate[T, Option[Double]]
Aggregate function: returns the covariance of two collumns.
Aggregate function: returns the covariance of two collumns.
- Definition Classes
- AggregateFunctions
- Note
In Spark covar_pop always returns Double https://github.com/apache/spark/blob/4a3c09601ba69f7d49d1946bb6f20f5cfe453031/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Covariance.scala#L82 apache/spark
- def covarSamp[A, B, T](column1: TypedColumn[T, A], column2: TypedColumn[T, B])(implicit i0: CatalystCast[A, Double], i1: CatalystCast[B, Double]): TypedAggregate[T, Option[Double]]
Aggregate function: returns the covariance of two columns.
Aggregate function: returns the covariance of two columns.
- Definition Classes
- AggregateFunctions
- Note
In Spark covar_samp always returns Double https://github.com/apache/spark/blob/4a3c09601ba69f7d49d1946bb6f20f5cfe453031/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Covariance.scala#L93 apache/spark
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- def first[A, T](column: TypedColumn[T, A]): TypedAggregate[T, A]
Aggregate function: returns the first value in a group.
Aggregate function: returns the first value in a group.
The function by default returns the first values it sees. It will return the first non-null value it sees when ignoreNulls is set to true. If all values are null, then null is returned.
apache/spark
- Definition Classes
- AggregateFunctions
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def kurtosis[A, T](column: TypedColumn[T, A])(implicit ev: CatalystCast[A, Double]): TypedAggregate[T, Option[Double]]
Aggregate function: returns the kurtosis of a column.
Aggregate function: returns the kurtosis of a column.
- Definition Classes
- AggregateFunctions
- Note
In Spark kurtosis always returns Double https://github.com/apache/spark/blob/4a3c09601ba69f7d49d1946bb6f20f5cfe453031/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/CentralMomentAgg.scala#L220 apache/spark
- def last[A, T](column: TypedColumn[T, A]): TypedAggregate[T, A]
Aggregate function: returns the last value in a group.
Aggregate function: returns the last value in a group.
The function by default returns the last values it sees. It will return the last non-null value it sees when ignoreNulls is set to true. If all values are null, then null is returned.
apache/spark
- Definition Classes
- AggregateFunctions
- def max[A, T](column: TypedColumn[T, A])(implicit arg0: CatalystOrdered[A]): TypedAggregate[T, A]
Aggregate function: returns the maximum value of the column in a group.
Aggregate function: returns the maximum value of the column in a group.
apache/spark
- Definition Classes
- AggregateFunctions
- def min[A, T](column: TypedColumn[T, A])(implicit arg0: CatalystOrdered[A]): TypedAggregate[T, A]
Aggregate function: returns the minimum value of the column in a group.
Aggregate function: returns the minimum value of the column in a group.
apache/spark
- Definition Classes
- AggregateFunctions
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- def skewness[A, T](column: TypedColumn[T, A])(implicit ev: CatalystCast[A, Double]): TypedAggregate[T, Option[Double]]
Aggregate function: returns the skewness of a column.
Aggregate function: returns the skewness of a column.
- Definition Classes
- AggregateFunctions
- Note
In Spark skewness always returns Double https://github.com/apache/spark/blob/4a3c09601ba69f7d49d1946bb6f20f5cfe453031/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/CentralMomentAgg.scala#L200 apache/spark
- def stddev[A, T](column: TypedColumn[T, A])(implicit arg0: CatalystVariance[A]): TypedAggregate[T, Double]
Aggregate function: returns the sample standard deviation.
Aggregate function: returns the sample standard deviation.
- Definition Classes
- AggregateFunctions
- Note
In Spark stddev always returns Double https://github.com/apache/spark/blob/4a3c09601ba69f7d49d1946bb6f20f5cfe453031/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/CentralMomentAgg.scala#155 apache/spark
- def stddevPop[A, T](column: TypedColumn[T, A])(implicit ev: CatalystCast[A, Double]): TypedAggregate[T, Option[Double]]
Aggregate function: returns the standard deviation of a column by population.
Aggregate function: returns the standard deviation of a column by population.
- Definition Classes
- AggregateFunctions
- Note
In Spark stddev always returns Double https://github.com/apache/spark/blob/4a3c09601ba69f7d49d1946bb6f20f5cfe453031/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/CentralMomentAgg.scala#L143 apache/spark
- def stddevSamp[A, T](column: TypedColumn[T, A])(implicit ev: CatalystCast[A, Double]): TypedAggregate[T, Option[Double]]
Aggregate function: returns the standard deviation of a column by sample.
Aggregate function: returns the standard deviation of a column by sample.
- Definition Classes
- AggregateFunctions
- Note
In Spark stddev always returns Double https://github.com/apache/spark/blob/4a3c09601ba69f7d49d1946bb6f20f5cfe453031/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/CentralMomentAgg.scala#L160 apache/spark
- def sum[A, T, Out](column: TypedColumn[T, A])(implicit summable: CatalystSummable[A, Out], oencoder: TypedEncoder[Out], aencoder: TypedEncoder[A]): TypedAggregate[T, Out]
Aggregate function: returns the sum of all values in the given column.
Aggregate function: returns the sum of all values in the given column.
apache/spark
- Definition Classes
- AggregateFunctions
- def sumDistinct[A, T, Out](column: TypedColumn[T, A])(implicit summable: CatalystSummable[A, Out], oencoder: TypedEncoder[Out], aencoder: TypedEncoder[A]): TypedAggregate[T, Out]
Aggregate function: returns the sum of distinct values in the column.
Aggregate function: returns the sum of distinct values in the column.
apache/spark
- Definition Classes
- AggregateFunctions
- Annotations
- @nowarn()
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- def variance[A, T](column: TypedColumn[T, A])(implicit arg0: CatalystVariance[A]): TypedAggregate[T, Double]
Aggregate function: returns the unbiased variance of the values in a group.
Aggregate function: returns the unbiased variance of the values in a group.
- Definition Classes
- AggregateFunctions
- Note
In Spark variance always returns Double https://github.com/apache/spark/blob/4a3c09601ba69f7d49d1946bb6f20f5cfe453031/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/CentralMomentAgg.scala#186 apache/spark
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()