Given an instance of DataFrame returns a dataframe where all primitive fields are converted to String
Given an instance of DataFrame returns a dataframe where all primitive fields are converted to String
A dataframe
A new dataframe with all primitive fields as Strings.
Retrieves all executors available for the current job.
Given an instance of DataFrame returns a dataframe with flattened schema.
Given an instance of DataFrame returns a dataframe with flattened schema. All nested structures are flattened and arrays are projected as columns.
Note. The method checks the maximum size for each array and that could perform slowly, especially on a vary big dataframes.
A dataframe
When flattening a schema each field name will contain full path. You can override this behavior and use a short field names instead
A new dataframe with flat schema.
This object contains common Spark tools used for easier processing of dataframes originated from mainframes.