io.smartdatalake.workflow.action.customlogic
Optional class name implementing trait CustomDfCreator
Optional file where scala code for creator is loaded from. The scala code in the file needs to be a function of type fnExecType.
Optional scala code for creator. The scala code needs to be a function of type fnExecType.
Options to pass to the creator
Optional class name implementing trait CustomDfCreator
Options to pass to the creator
Optional scala code for creator.
Optional scala code for creator. The scala code needs to be a function of type fnExecType.
Optional file where scala code for creator is loaded from.
Optional file where scala code for creator is loaded from. The scala code in the file needs to be a function of type fnExecType.
Configuration of a custom Spark-DataFrame creator as part of CustomDfDataObject Define a exec function which receives a map of options and returns a DataFrame to be used as input. Optionally define a schema function to return a StructType used as schema in init-phase. See also trait CustomDfCreator.
Note that for now implementing CustomDfCreator.schema method is only possible with className configuration attribute.
Optional class name implementing trait CustomDfCreator
Optional file where scala code for creator is loaded from. The scala code in the file needs to be a function of type fnExecType.
Optional scala code for creator. The scala code needs to be a function of type fnExecType.
Options to pass to the creator