PythonUDFCreatorConfig

Configuration to register a Python UDF in the spark session of SmartDataLake. Define a python function with type hints i python code and register it in global configuration. The name of the function must match the name you use to declare it in GlobalConf. The Python function can then be used in Spark SQL expressions.

pythonFile: Optional pythonFile to use for python UDF.
pythonCode: Optional pythonCode to use for python UDF.
options: Options are available in your python code as variable options.

Linear Supertypes

Serializable, Serializable, Product, Equals, AnyRef, Any

Instance Constructors

new PythonUDFCreatorConfig(pythonFile: Option[String] = None, pythonCode: Option[String] = None, options: Option[Map[String, String]] = None)

pythonFile
Optional pythonFile to use for python UDF.
pythonCode
Optional pythonCode to use for python UDF.
options
Options are available in your python code as variable options.

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def asInstanceOf[T0]: T0

Definition Classes
Any
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
val options: Option[Map[String, String]]

Options are available in your python code as variable options.
val pythonCode: Option[String]

Optional pythonCode to use for python UDF.
val pythonFile: Option[String]

Optional pythonFile to use for python UDF.
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )

Related Doc: package customlogic

case class PythonUDFCreatorConfig(pythonFile: Option[String] = None, pythonCode: Option[String] = None, options: Option[Map[String, String]] = None) extends Product with Serializable

Instance Constructors

new PythonUDFCreatorConfig(pythonFile: Option[String] = None, pythonCode: Option[String] = None, options: Option[Map[String, String]] = None)

Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

final def asInstanceOf[T0]: T0

def clone(): AnyRef

final def eq(arg0: AnyRef): Boolean

def finalize(): Unit

final def getClass(): Class[_]

final def isInstanceOf[T0]: Boolean

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

val options: Option[Map[String, String]]

val pythonCode: Option[String]

val pythonFile: Option[String]

final def synchronized[T0](arg0: ⇒ T0): T0

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped