nonAggregate

object nonAggregate extends NonAggregateFunctions

Linear Supertypes

NonAggregateFunctions, AnyRef, Any

Ordering

Alphabetic
By Inheritance

Inherited

nonAggregate
NonAggregateFunctions
AnyRef
Any

Hide All
Show All

Visibility

Public
Protected

Type Members

class When[T, A] extends AnyRef
Definition Classes
NonAggregateFunctions

Value Members

final def !=(arg0: Any): Boolean
Definition Classes
AnyRef → Any
final def ##: Int
Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean
Definition Classes
AnyRef → Any
def abs[A, B, T](column: AbstractTypedColumn[T, A])(implicit i0: CatalystNumericWithJavaBigDecimal[A, B], i1: TypedEncoder[B]): ThisType[T, B]
Non-Aggregate function: returns the absolute value of a numeric column
Non-Aggregate function: returns the absolute value of a numeric column
apache/spark
Definition Classes
NonAggregateFunctions
def acos[A, T](column: AbstractTypedColumn[T, A])(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Non-Aggregate function: returns the acos of a numeric column
Non-Aggregate function: returns the acos of a numeric column
Spark will expect a Double value for this expression. See: https://github.com/apache/spark/blob/4a3c09601ba69f7d49d1946bb6f20f5cfe453031/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala#L67 apache/spark
Definition Classes
NonAggregateFunctions
def arrayContains[C[_], A, T](column: AbstractTypedColumn[T, C[A]], value: A)(implicit arg0: CatalystCollection[C]): ThisType[T, Boolean]
Non-Aggregate function: returns true if value is contained with in the array in the specified column
Non-Aggregate function: returns true if value is contained with in the array in the specified column
apache/spark
Definition Classes
NonAggregateFunctions
final def asInstanceOf[T0]: T0
Definition Classes
Any
def ascii[T](column: AbstractTypedColumn[T, String]): ThisType[T, Int]
Non-Aggregate function: takes the first letter of a string column and returns the ascii int value in a new column
Non-Aggregate function: takes the first letter of a string column and returns the ascii int value in a new column
apache/spark
Definition Classes
NonAggregateFunctions
def asin[A, T](column: AbstractTypedColumn[T, A])(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Non-Aggregate function: returns the asin of a numeric column
Non-Aggregate function: returns the asin of a numeric column
Spark will expect a Double value for this expression. See: https://github.com/apache/spark/blob/4a3c09601ba69f7d49d1946bb6f20f5cfe453031/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala#L67 apache/spark
Definition Classes
NonAggregateFunctions
def atan[A, T](column: AbstractTypedColumn[T, A])(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Non-Aggregate function: returns the atan of a numeric column
Non-Aggregate function: returns the atan of a numeric column
Spark will expect a Double value for this expression. See: https://github.com/apache/spark/blob/4a3c09601ba69f7d49d1946bb6f20f5cfe453031/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala#L67 apache/spark
Definition Classes
NonAggregateFunctions
def atan2[A, T](l: TypedAggregate[T, A], r: Double)(implicit i0: CatalystCast[A, Double]): TypedAggregate[T, Double]
Definition Classes
NonAggregateFunctions
def atan2[B, T](l: Double, r: TypedAggregate[T, B])(implicit i0: CatalystCast[B, Double]): TypedAggregate[T, Double]
Definition Classes
NonAggregateFunctions
def atan2[A, T](l: TypedColumn[T, A], r: Double)(implicit i0: CatalystCast[A, Double]): TypedColumn[T, Double]
Definition Classes
NonAggregateFunctions
def atan2[B, T](l: Double, r: TypedColumn[T, B])(implicit i0: CatalystCast[B, Double]): TypedColumn[T, Double]
Definition Classes
NonAggregateFunctions
def atan2[A, B, T](l: TypedAggregate[T, A], r: TypedAggregate[T, B])(implicit i0: CatalystCast[A, Double], i1: CatalystCast[B, Double]): TypedAggregate[T, Double]
Non-Aggregate function: returns the angle theta from the conversion of rectangular coordinates (x, y) to polar coordinates (r, theta).
Non-Aggregate function: returns the angle theta from the conversion of rectangular coordinates (x, y) to polar coordinates (r, theta).
Spark will expect a Double value for this expression. See: https://github.com/apache/spark/blob/4a3c09601ba69f7d49d1946bb6f20f5cfe453031/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala#L67 apache/spark
Definition Classes
NonAggregateFunctions
def atan2[A, B, T](l: TypedColumn[T, A], r: TypedColumn[T, B])(implicit i0: CatalystCast[A, Double], i1: CatalystCast[B, Double]): TypedColumn[T, Double]
Non-Aggregate function: returns the angle theta from the conversion of rectangular coordinates (x, y) to polar coordinates (r, theta).
Non-Aggregate function: returns the angle theta from the conversion of rectangular coordinates (x, y) to polar coordinates (r, theta).
Spark will expect a Double value for this expression. See: https://github.com/apache/spark/blob/4a3c09601ba69f7d49d1946bb6f20f5cfe453031/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala#L67 apache/spark
Definition Classes
NonAggregateFunctions
def base64[T](column: AbstractTypedColumn[T, Array[Byte]]): ThisType[T, String]
Non-Aggregate function: Computes the BASE64 encoding of a binary column and returns it as a string column.
Non-Aggregate function: Computes the BASE64 encoding of a binary column and returns it as a string column. This is the reverse of unbase64.
apache/spark
Definition Classes
NonAggregateFunctions
def bin[T](column: AbstractTypedColumn[T, Long]): ThisType[T, String]
Non-Aggregate function: Returns the string representation of the binary value of the given long column.
Non-Aggregate function: Returns the string representation of the binary value of the given long column. For example, bin("12") returns "1100".
apache/spark
Definition Classes
NonAggregateFunctions
def bitwiseNOT[A, T](column: AbstractTypedColumn[T, A])(implicit arg0: CatalystBitwise[A]): ThisType[T, A]
Non-Aggregate function: Computes bitwise NOT.
Non-Aggregate function: Computes bitwise NOT.
apache/spark
Definition Classes
NonAggregateFunctions
Annotations
@nowarn()
def bround[A, B, T](column: AbstractTypedColumn[T, A], scale: Int)(implicit i0: CatalystNumericWithJavaBigDecimal[A, B], i1: TypedEncoder[B]): ThisType[T, B]
Non-Aggregate function: Bankers Rounding - returns the rounded to scale decimal places value with HALF_EVEN round mode of a numeric column.
Non-Aggregate function: Bankers Rounding - returns the rounded to scale decimal places value with HALF_EVEN round mode of a numeric column. If scale is greater than or equal to 0 or at integral part when scale is less than 0.
apache/spark
Definition Classes
NonAggregateFunctions
def bround[A, B, T](column: AbstractTypedColumn[T, A])(implicit i0: CatalystNumericWithJavaBigDecimal[A, B], i1: TypedEncoder[B]): ThisType[T, B]
Non-Aggregate function: Bankers Rounding - returns the rounded to 0 decimal places value with HALF_EVEN round mode of a numeric column.
Non-Aggregate function: Bankers Rounding - returns the rounded to 0 decimal places value with HALF_EVEN round mode of a numeric column.
apache/spark
Definition Classes
NonAggregateFunctions
def cbrt[A, T](column: AbstractTypedColumn[T, A])(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Non-Aggregate function: returns the cubic root value of a numeric column.
Non-Aggregate function: returns the cubic root value of a numeric column.
apache/spark
Definition Classes
NonAggregateFunctions
def ceil[A, B, T](column: AbstractTypedColumn[T, A])(implicit i0: CatalystRound[A, B], i1: TypedEncoder[B]): ThisType[T, B]
Non-Aggregate function: returns the ceiling of a numeric column
Non-Aggregate function: returns the ceiling of a numeric column
apache/spark
Definition Classes
NonAggregateFunctions
def clone(): AnyRef
Attributes
protected[lang]
Definition Classes
AnyRef
Annotations
@throws(classOf[java.lang.CloneNotSupportedException]) @native()
def concat[T](columns: TypedAggregate[T, String]*): TypedAggregate[T, String]
Non-Aggregate function: Concatenates multiple input string columns together into a single string column.
Non-Aggregate function: Concatenates multiple input string columns together into a single string column.
Definition Classes
NonAggregateFunctions
Note
varargs make it harder to generalize so we overload the method for TypedColumn and TypedAggregate apache/spark
def concat[T](columns: TypedColumn[T, String]*): TypedColumn[T, String]
Non-Aggregate function: Concatenates multiple input string columns together into a single string column.
Non-Aggregate function: Concatenates multiple input string columns together into a single string column.
Definition Classes
NonAggregateFunctions
Note
varargs make it harder to generalize so we overload the method for TypedColumn and TypedAggregate apache/spark
def concatWs[T](sep: String, columns: TypedColumn[T, String]*): TypedColumn[T, String]
Non-Aggregate function: Concatenates multiple input string columns together into a single string column, using the given separator.
Non-Aggregate function: Concatenates multiple input string columns together into a single string column, using the given separator.
Definition Classes
NonAggregateFunctions
Note
varargs make it harder to generalize so we overload the method for TypedColumn and TypedAggregate apache/spark
def concatWs[T](sep: String, columns: TypedAggregate[T, String]*): TypedAggregate[T, String]
Non-Aggregate function: Concatenates multiple input string columns together into a single string column, using the given separator.
Non-Aggregate function: Concatenates multiple input string columns together into a single string column, using the given separator.
Definition Classes
NonAggregateFunctions
Note
varargs make it harder to generalize so we overload the method for TypedColumn and TypedAggregate apache/spark
def conv[T](column: AbstractTypedColumn[T, String], fromBase: Int, toBase: Int): ThisType[T, String]
Non-Aggregate function: Convert a number in a string column from one base to another.
Non-Aggregate function: Convert a number in a string column from one base to another.
apache/spark
Definition Classes
NonAggregateFunctions
def cos[A, T](column: AbstractTypedColumn[T, A])(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Non-Aggregate function: Computes the cosine of the given value.
Non-Aggregate function: Computes the cosine of the given value.
Spark will expect a Double value for this expression. See: https://github.com/apache/spark/blob/4a3c09601ba69f7d49d1946bb6f20f5cfe453031/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala#L67 apache/spark
Definition Classes
NonAggregateFunctions
def cosh[A, T](column: AbstractTypedColumn[T, A])(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Non-Aggregate function: Computes the hyperbolic cosine of the given value.
Non-Aggregate function: Computes the hyperbolic cosine of the given value.
Spark will expect a Double value for this expression. See: https://github.com/apache/spark/blob/4a3c09601ba69f7d49d1946bb6f20f5cfe453031/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala#L67 apache/spark
Definition Classes
NonAggregateFunctions
def crc32[T](column: AbstractTypedColumn[T, Array[Byte]]): ThisType[T, Long]
Non-Aggregate function: returns a cyclic redundancy check value of a binary column as long.
Non-Aggregate function: returns a cyclic redundancy check value of a binary column as long.
apache/spark
Definition Classes
NonAggregateFunctions
def dayofmonth[T](str: AbstractTypedColumn[T, String]): ThisType[T, Option[Int]]
Non-Aggregate function: Extracts the day of the month as an integer from a given date/timestamp/string.
Non-Aggregate function: Extracts the day of the month as an integer from a given date/timestamp/string.
Differs from Column#dayofmonth by wrapping it's result into an Option.
apache/spark
Definition Classes
NonAggregateFunctions
def dayofweek[T](str: AbstractTypedColumn[T, String]): ThisType[T, Option[Int]]
Non-Aggregate function: Extracts the day of the week as an integer from a given date/timestamp/string.
Non-Aggregate function: Extracts the day of the week as an integer from a given date/timestamp/string.
Differs from Column#dayofweek by wrapping it's result into an Option.
apache/spark
Definition Classes
NonAggregateFunctions
def dayofyear[T](str: AbstractTypedColumn[T, String]): ThisType[T, Option[Int]]
Non-Aggregate function: Extracts the day of the year as an integer from a given date/timestamp/string.
Non-Aggregate function: Extracts the day of the year as an integer from a given date/timestamp/string.
Differs from Column#dayofyear by wrapping it's result into an Option.
apache/spark
Definition Classes
NonAggregateFunctions
def degrees[A, T](column: AbstractTypedColumn[T, A]): ThisType[T, Double]
Non-Aggregate function: Converts an angle measured in radians to an approximately equivalent angle measured in degrees.
Non-Aggregate function: Converts an angle measured in radians to an approximately equivalent angle measured in degrees.
apache/spark
Definition Classes
NonAggregateFunctions
final def eq(arg0: AnyRef): Boolean
Definition Classes
AnyRef
def equals(arg0: AnyRef): Boolean
Definition Classes
AnyRef → Any
def exp[A, T](column: AbstractTypedColumn[T, A])(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Non-Aggregate function: returns the exponential value of a numeric column.
Non-Aggregate function: returns the exponential value of a numeric column.
apache/spark
Definition Classes
NonAggregateFunctions
def factorial[T](column: AbstractTypedColumn[T, Long])(implicit i0: TypedEncoder[Long]): ThisType[T, Long]
Computes the factorial of the given value.
Computes the factorial of the given value.
apache/spark
Definition Classes
NonAggregateFunctions
def finalize(): Unit
Attributes
protected[lang]
Definition Classes
AnyRef
Annotations
@throws(classOf[java.lang.Throwable])
def floor[A, B, T](column: AbstractTypedColumn[T, A])(implicit i0: CatalystRound[A, B], i1: TypedEncoder[B]): ThisType[T, B]
Non-Aggregate function: returns the floor of a numeric column
Non-Aggregate function: returns the floor of a numeric column
apache/spark
Definition Classes
NonAggregateFunctions
final def getClass(): Class[_ <: AnyRef]
Definition Classes
AnyRef → Any
Annotations
@native()
def hashCode(): Int
Definition Classes
AnyRef → Any
Annotations
@native()
def hour[T](str: AbstractTypedColumn[T, String]): ThisType[T, Option[Int]]
Non-Aggregate function: Extracts the hours as an integer from a given date/timestamp/string.
Non-Aggregate function: Extracts the hours as an integer from a given date/timestamp/string.
Differs from Column#hour by wrapping it's result into an Option.
apache/spark
Definition Classes
NonAggregateFunctions
def hypot[A, T](l: Double, column: AbstractTypedColumn[T, A])(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Computes sqrt(a² + b²) without intermediate overflow or underflow.
Computes sqrt(a² + b²) without intermediate overflow or underflow.
apache/spark
Definition Classes
NonAggregateFunctions
def hypot[A, T](column: AbstractTypedColumn[T, A], l: Double)(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Computes sqrt(a² + b²) without intermediate overflow or underflow.
Computes sqrt(a² + b²) without intermediate overflow or underflow.
apache/spark
Definition Classes
NonAggregateFunctions
def hypot[A, T](column: AbstractTypedColumn[T, A], column2: AbstractTypedColumn[T, A])(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Computes sqrt(a² + b²) without intermediate overflow or underflow.
Computes sqrt(a² + b²) without intermediate overflow or underflow.
apache/spark
Definition Classes
NonAggregateFunctions
def inputFileName[T](): TypedColumn[T, String]
Non-Aggregate function: file name of the current Spark task.
Non-Aggregate function: file name of the current Spark task. Empty string if row did not originate from a file
apache/spark
Definition Classes
NonAggregateFunctions
def instr[T](str: AbstractTypedColumn[T, String], substring: String): ThisType[T, Int]
Non-Aggregate function: Locates the position of the first occurrence of substring column in given string
Non-Aggregate function: Locates the position of the first occurrence of substring column in given string
Definition Classes
NonAggregateFunctions
Note
The position is not zero based, but 1 based index. Returns 0 if substr could not be found in str. apache/spark
final def isInstanceOf[T0]: Boolean
Definition Classes
Any
def length[T](str: AbstractTypedColumn[T, String]): ThisType[T, Int]
Non-Aggregate function: Computes the length of a given string.
Non-Aggregate function: Computes the length of a given string.
apache/spark
Definition Classes
NonAggregateFunctions
def levenshtein[T](l: TypedAggregate[T, String], r: TypedAggregate[T, String]): TypedAggregate[T, Int]
Non-Aggregate function: Computes the Levenshtein distance of the two given string columns.
Non-Aggregate function: Computes the Levenshtein distance of the two given string columns.
apache/spark
Definition Classes
NonAggregateFunctions
def levenshtein[T](l: TypedColumn[T, String], r: TypedColumn[T, String]): TypedColumn[T, Int]
Non-Aggregate function: Computes the Levenshtein distance of the two given string columns.
Non-Aggregate function: Computes the Levenshtein distance of the two given string columns.
apache/spark
Definition Classes
NonAggregateFunctions
def log[A, T](base: Double, column: AbstractTypedColumn[T, A])(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Returns the first argument-base logarithm of the second argument.
Returns the first argument-base logarithm of the second argument.
apache/spark
Definition Classes
NonAggregateFunctions
def log[A, T](column: AbstractTypedColumn[T, A])(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Computes the natural logarithm of the given value.
Computes the natural logarithm of the given value.
apache/spark
Definition Classes
NonAggregateFunctions
def log10[A, T](column: AbstractTypedColumn[T, A])(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Computes the logarithm of the given column in base 10.
Computes the logarithm of the given column in base 10.
apache/spark
Definition Classes
NonAggregateFunctions
def log1p[A, T](column: AbstractTypedColumn[T, A])(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Computes the natural logarithm of the given value plus one.
Computes the natural logarithm of the given value plus one.
apache/spark
Definition Classes
NonAggregateFunctions
def log2[A, T](column: AbstractTypedColumn[T, A])(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Computes the logarithm of the given column in base 2.
Computes the logarithm of the given column in base 2.
apache/spark
Definition Classes
NonAggregateFunctions
def lower[T](str: AbstractTypedColumn[T, String]): ThisType[T, String]
Non-Aggregate function: Converts a string column to lower case.
Non-Aggregate function: Converts a string column to lower case.
apache/spark
Definition Classes
NonAggregateFunctions
def lpad[T](str: AbstractTypedColumn[T, String], len: Int, pad: String): ThisType[T, String]
Non-Aggregate function: Left-pad the string column with pad to a length of len.
Non-Aggregate function: Left-pad the string column with pad to a length of len. If the string column is longer than len, the return value is shortened to len characters.
apache/spark
Definition Classes
NonAggregateFunctions
def ltrim[T](str: AbstractTypedColumn[T, String]): ThisType[T, String]
Non-Aggregate function: Trim the spaces from left end for the specified string value.
Non-Aggregate function: Trim the spaces from left end for the specified string value.
apache/spark
Definition Classes
NonAggregateFunctions
def md5[T, A](column: AbstractTypedColumn[T, A])(implicit i0: TypedEncoder[A]): ThisType[T, String]
Calculates the MD5 digest of a binary column and returns the value as a 32 character hex string.
Calculates the MD5 digest of a binary column and returns the value as a 32 character hex string.
apache/spark
Definition Classes
NonAggregateFunctions
def minute[T](str: AbstractTypedColumn[T, String]): ThisType[T, Option[Int]]
Non-Aggregate function: Extracts the minutes as an integer from a given date/timestamp/string.
Non-Aggregate function: Extracts the minutes as an integer from a given date/timestamp/string.
Differs from Column#minute by wrapping it's result into an Option.
apache/spark
Definition Classes
NonAggregateFunctions
def monotonicallyIncreasingId[T](): TypedColumn[T, Long]
Non-Aggregate function: generates monotonically increasing id
Non-Aggregate function: generates monotonically increasing id
apache/spark
Definition Classes
NonAggregateFunctions
def month[T](str: AbstractTypedColumn[T, String]): ThisType[T, Option[Int]]
Non-Aggregate function Extracts the month as an integer from a given date/timestamp/string.
Non-Aggregate function Extracts the month as an integer from a given date/timestamp/string.
Differs from Column#month by wrapping it's result into an Option.
apache/spark
Definition Classes
NonAggregateFunctions
final def ne(arg0: AnyRef): Boolean
Definition Classes
AnyRef
def negate[A, B, T](column: AbstractTypedColumn[T, A])(implicit i0: CatalystNumericWithJavaBigDecimal[A, B], i1: TypedEncoder[B]): ThisType[T, B]
Non-Aggregate function: returns the negated value of column.
Non-Aggregate function: returns the negated value of column.
apache/spark
Definition Classes
NonAggregateFunctions
def not[T](column: AbstractTypedColumn[T, Boolean]): ThisType[T, Boolean]
Non-Aggregate function: logical not.
Non-Aggregate function: logical not.
apache/spark
Definition Classes
NonAggregateFunctions
final def notify(): Unit
Definition Classes
AnyRef
Annotations
@native()
final def notifyAll(): Unit
Definition Classes
AnyRef
Annotations
@native()
def pmod[A, T](column: AbstractTypedColumn[T, A], column2: AbstractTypedColumn[T, A])(implicit i0: TypedEncoder[A]): ThisType[T, A]
Returns the positive value of dividend mod divisor.
Returns the positive value of dividend mod divisor.
apache/spark
Definition Classes
NonAggregateFunctions
def pow[A, T](l: Double, column: AbstractTypedColumn[T, A])(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Returns the value of the first argument raised to the power of the second argument.
Returns the value of the first argument raised to the power of the second argument.
apache/spark
Definition Classes
NonAggregateFunctions
def pow[A, T](column: AbstractTypedColumn[T, A], l: Double)(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Returns the value of the first argument raised to the power of the second argument.
Returns the value of the first argument raised to the power of the second argument.
apache/spark
Definition Classes
NonAggregateFunctions
def pow[A, T](column: AbstractTypedColumn[T, A], column2: AbstractTypedColumn[T, A])(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Returns the value of the first argument raised to the power of the second argument.
Returns the value of the first argument raised to the power of the second argument.
apache/spark
Definition Classes
NonAggregateFunctions
def quarter[T](str: AbstractTypedColumn[T, String]): ThisType[T, Option[Int]]
Non-Aggregate function: Extracts the quarter as an integer from a given date/timestamp/string.
Non-Aggregate function: Extracts the quarter as an integer from a given date/timestamp/string.
Differs from Column#quarter by wrapping it's result into an Option.
apache/spark
Definition Classes
NonAggregateFunctions
def regexpReplace[T](str: AbstractTypedColumn[T, String], pattern: Regex, replacement: String): ThisType[T, String]
Non-Aggregate function: Replace all substrings of the specified string value that match regexp with rep.
Non-Aggregate function: Replace all substrings of the specified string value that match regexp with rep.
apache/spark
Definition Classes
NonAggregateFunctions
def reverse[T](str: AbstractTypedColumn[T, String]): ThisType[T, String]
Non-Aggregate function: Reverses the string column and returns it as a new string column.
Non-Aggregate function: Reverses the string column and returns it as a new string column.
apache/spark
Definition Classes
NonAggregateFunctions
def round[A, B, T](column: AbstractTypedColumn[T, A], scale: Int)(implicit i0: CatalystNumericWithJavaBigDecimal[A, B], i1: TypedEncoder[B]): ThisType[T, B]
Non-Aggregate function: Round the value of e to scale decimal places with HALF_UP round mode if scale is greater than or equal to 0 or at integral part when scale is less than 0.
Non-Aggregate function: Round the value of e to scale decimal places with HALF_UP round mode if scale is greater than or equal to 0 or at integral part when scale is less than 0.
apache/spark
Definition Classes
NonAggregateFunctions
def round[A, B, T](column: AbstractTypedColumn[T, A])(implicit i0: CatalystNumericWithJavaBigDecimal[A, B], i1: TypedEncoder[B]): ThisType[T, B]
Non-Aggregate function: Returns the value of the column e rounded to 0 decimal places with HALF_UP round mode.
Non-Aggregate function: Returns the value of the column e rounded to 0 decimal places with HALF_UP round mode.
apache/spark
Definition Classes
NonAggregateFunctions
def rpad[T](str: AbstractTypedColumn[T, String], len: Int, pad: String): ThisType[T, String]
Non-Aggregate function: Right-pad the string column with pad to a length of len.
Non-Aggregate function: Right-pad the string column with pad to a length of len. If the string column is longer than len, the return value is shortened to len characters.
apache/spark
Definition Classes
NonAggregateFunctions
def rtrim[T](str: AbstractTypedColumn[T, String]): ThisType[T, String]
Non-Aggregate function: Trim the spaces from right end for the specified string value.
Non-Aggregate function: Trim the spaces from right end for the specified string value.
apache/spark
Definition Classes
NonAggregateFunctions
def second[T](str: AbstractTypedColumn[T, String]): ThisType[T, Option[Int]]
Non-Aggregate function: Extracts the seconds as an integer from a given date/timestamp/string.
Non-Aggregate function: Extracts the seconds as an integer from a given date/timestamp/string.
Differs from Column#second by wrapping it's result into an Option.
apache/spark
Definition Classes
NonAggregateFunctions
def sha1[T](column: AbstractTypedColumn[T, Array[Byte]]): ThisType[T, String]
Non-Aggregate function: calculates the SHA-1 digest of a binary column and returns the value as a 40 character hex string
Non-Aggregate function: calculates the SHA-1 digest of a binary column and returns the value as a 40 character hex string
apache/spark
Definition Classes
NonAggregateFunctions
def sha2[T](column: AbstractTypedColumn[T, Array[Byte]], numBits: Int): ThisType[T, String]
Non-Aggregate function: calculates the SHA-2 digest of a binary column and returns the value as a 40 character hex string
Non-Aggregate function: calculates the SHA-2 digest of a binary column and returns the value as a 40 character hex string
apache/spark
Definition Classes
NonAggregateFunctions
def shiftLeft[A, B, T](column: AbstractTypedColumn[T, A], numBits: Int)(implicit i0: CatalystBitShift[A, B], i1: TypedEncoder[B]): ThisType[T, B]
Non-Aggregate function: shift the the given value numBits left.
Non-Aggregate function: shift the the given value numBits left. If given long, will return long else it will return an integer.
apache/spark
Definition Classes
NonAggregateFunctions
Annotations
@nowarn()
def shiftRight[A, B, T](column: AbstractTypedColumn[T, A], numBits: Int)(implicit i0: CatalystBitShift[A, B], i1: TypedEncoder[B]): ThisType[T, B]
Non-Aggregate function: shift the the given value numBits right.
Non-Aggregate function: shift the the given value numBits right. If given long, will return long else it will return an integer.
apache/spark
Definition Classes
NonAggregateFunctions
Annotations
@nowarn()
def shiftRightUnsigned[A, B, T](column: AbstractTypedColumn[T, A], numBits: Int)(implicit i0: CatalystBitShift[A, B], i1: TypedEncoder[B]): ThisType[T, B]
Non-Aggregate function: unsigned shift the the given value numBits right.
Non-Aggregate function: unsigned shift the the given value numBits right. If given long, will return long else it will return an integer.
apache/spark
Definition Classes
NonAggregateFunctions
Annotations
@nowarn()
def signum[A, T](column: AbstractTypedColumn[T, A])(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Non-Aggregate function: Computes the signum of the given value.
Non-Aggregate function: Computes the signum of the given value.
Spark will expect a Double value for this expression. See: https://github.com/apache/spark/blob/4a3c09601ba69f7d49d1946bb6f20f5cfe453031/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala#L67 apache/spark
Definition Classes
NonAggregateFunctions
def sin[A, T](column: AbstractTypedColumn[T, A])(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Non-Aggregate function: Computes the sine of the given value.
Non-Aggregate function: Computes the sine of the given value.
Spark will expect a Double value for this expression. See: https://github.com/apache/spark/blob/4a3c09601ba69f7d49d1946bb6f20f5cfe453031/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala#L67 apache/spark
Definition Classes
NonAggregateFunctions
def sinh[A, T](column: AbstractTypedColumn[T, A])(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Non-Aggregate function: Computes the hyperbolic sine of the given value.
Non-Aggregate function: Computes the hyperbolic sine of the given value.
Spark will expect a Double value for this expression. See: https://github.com/apache/spark/blob/4a3c09601ba69f7d49d1946bb6f20f5cfe453031/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala#L67 apache/spark
Definition Classes
NonAggregateFunctions
def sqrt[A, T](column: AbstractTypedColumn[T, A])(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Non-Aggregate function: returns the square root value of a numeric column.
Non-Aggregate function: returns the square root value of a numeric column.
apache/spark
Definition Classes
NonAggregateFunctions
def substring[T](str: AbstractTypedColumn[T, String], pos: Int, len: Int): ThisType[T, String]
Non-Aggregate function: Substring starts at pos and is of length len
Non-Aggregate function: Substring starts at pos and is of length len
apache/spark
Definition Classes
NonAggregateFunctions
final def synchronized[T0](arg0: => T0): T0
Definition Classes
AnyRef
def tan[A, T](column: AbstractTypedColumn[T, A])(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Non-Aggregate function: Computes the tangent of the given column.
Non-Aggregate function: Computes the tangent of the given column.
Spark will expect a Double value for this expression. See: https://github.com/apache/spark/blob/4a3c09601ba69f7d49d1946bb6f20f5cfe453031/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala#L67 apache/spark
Definition Classes
NonAggregateFunctions
def tanh[A, T](column: AbstractTypedColumn[T, A])(implicit i0: CatalystCast[A, Double]): ThisType[T, Double]
Non-Aggregate function: Computes the hyperbolic tangent of the given value.
Non-Aggregate function: Computes the hyperbolic tangent of the given value.
Spark will expect a Double value for this expression. See: https://github.com/apache/spark/blob/4a3c09601ba69f7d49d1946bb6f20f5cfe453031/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala#L67 apache/spark
Definition Classes
NonAggregateFunctions
def toString(): String
Definition Classes
AnyRef → Any
def trim[T](str: AbstractTypedColumn[T, String]): ThisType[T, String]
Non-Aggregate function: Trim the spaces from both ends for the specified string column.
Non-Aggregate function: Trim the spaces from both ends for the specified string column.
apache/spark
Definition Classes
NonAggregateFunctions
def unbase64[T](column: AbstractTypedColumn[T, String]): ThisType[T, Array[Byte]]
Non-Aggregate function: Decodes a BASE64 encoded string column and returns it as a binary column.
Non-Aggregate function: Decodes a BASE64 encoded string column and returns it as a binary column. This is the reverse of base64.
apache/spark
Definition Classes
NonAggregateFunctions
def upper[T](str: AbstractTypedColumn[T, String]): ThisType[T, String]
Non-Aggregate function: Converts a string column to upper case.
Non-Aggregate function: Converts a string column to upper case.
apache/spark
Definition Classes
NonAggregateFunctions
final def wait(): Unit
Definition Classes
AnyRef
Annotations
@throws(classOf[java.lang.InterruptedException])
final def wait(arg0: Long, arg1: Int): Unit
Definition Classes
AnyRef
Annotations
@throws(classOf[java.lang.InterruptedException])
final def wait(arg0: Long): Unit
Definition Classes
AnyRef
Annotations
@throws(classOf[java.lang.InterruptedException]) @native()
def weekofyear[T](str: AbstractTypedColumn[T, String]): ThisType[T, Option[Int]]
Non-Aggregate function: Extracts the week number as an integer from a given date/timestamp/string.
Non-Aggregate function: Extracts the week number as an integer from a given date/timestamp/string.
Differs from Column#weekofyear by wrapping it's result into an Option.
apache/spark
Definition Classes
NonAggregateFunctions
def when[T, A](condition: AbstractTypedColumn[T, Boolean], value: AbstractTypedColumn[T, A]): When[T, A]
Non-Aggregate function: Evaluates a list of conditions and returns one of multiple possible result expressions.
Non-Aggregate function: Evaluates a list of conditions and returns one of multiple possible result expressions. If none match, otherwise is returned
```
when(ds('boolField), ds('a))
  .when(ds('otherBoolField), lit(123))
  .otherwise(ds('b))
```
apache/spark
Definition Classes
NonAggregateFunctions
def year[T](str: AbstractTypedColumn[T, String]): ThisType[T, Option[Int]]
Non-Aggregate function: Extracts the year as an integer from a given date/timestamp/string.
Non-Aggregate function: Extracts the year as an integer from a given date/timestamp/string.
Differs from Column#year by wrapping it's result into an Option.
apache/spark
Definition Classes
NonAggregateFunctions

Packages

nonAggregate

object nonAggregate extends NonAggregateFunctions

Type Members

Value Members

Inherited from NonAggregateFunctions

Inherited from AnyRef

Inherited from Any

Ungrouped

Packages

nonAggregate

object nonAggregate extends NonAggregateFunctions

Type Members

Value Members

Inherited from NonAggregateFunctions

Inherited from AnyRef

Inherited from Any

Ungrouped

nonAggregate