Type Members

1.  case class ScalarUserDefinedFunction extends UserDefinedFunction with Product with Serializable

   Holder class for a scalar user-defined function and it's input/output encoder(s).

2.  sealed abstract class UserDefinedFunction extends AnyRef

   A user-defined function.

   A user-defined function. To create one, use the udf functions in functions.

   As an example:

   // Define a UDF that returns true or false based on some numeric score.
   val predict = udf((score: Double) => score > 0.5)
   
   // Projects a column that adds a prediction column based on the score column.
   df.select( predict(df("score")) )

   Since

   3.4.0

3.  class Window extends AnyRef

   Utility functions for defining window in DataFrames.

   Utility functions for defining window in DataFrames.

   // PARTITION BY country ORDER BY date ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
   Window.partitionBy("country").orderBy("date")
     .rowsBetween(Window.unboundedPreceding, Window.currentRow)
   
   // PARTITION BY country ORDER BY date ROWS BETWEEN 3 PRECEDING AND 3 FOLLOWING
   Window.partitionBy("country").orderBy("date").rowsBetween(-3, 3)

   Annotations

   @Stable()

   Since

   3.4.0

4.  class WindowSpec extends AnyRef

   A window specification that defines the partitioning, ordering, and frame boundaries.

   A window specification that defines the partitioning, ordering, and frame boundaries.

   Use the static methods in Window to create a WindowSpec.

   Annotations

   @Stable()

   Since

   3.4.0