Function to define the transformation between several input and output DataFrames (n:m)
Function to define the transformation between several input and output DataFrames (n:m)
Spark Session
Options specified in the configuration for this transformation
DataFrames to be transformed
Transformed DataFrame
Optional function to define the transformation of input to output partition values.
Optional function to define the transformation of input to output partition values. For example this enables to implement aggregations where multiple input partitions are combined into one output partition. Note that the default value is input = output partition values, which should be correct for most use cases.
Options specified in the configuration for this transformation
partition values to be transformed
a map of input partition values to output partition values
Interface to define a custom Spark-DataFrame transformation (n:m) Same trait as CustomDfTransformer, but multiple input and outputs supported.