: Domain name
: Schema
: stage
: Storage Handler
Function to build the metrics save path
Function to build the metrics save path
: path where metrics are stored
: path where the metrics for the specified schema are stored
in the form [SinkType:[configName:]]viewName
(SinkType, configName, viewName)
Partition a dataset using dataset columns.
Partition a dataset using dataset columns. To partition the dataset using the ingestion time, use the reserved column names :
: Input dataset
: list of columns to use for partitioning.
The Spark session used to run this job
Just to force any spark job to implement its entry point using within the "run" method
Just to force any spark job to implement its entry point using within the "run" method
: Spark Session used for the job
Function Function that unifies discrete and continuous metrics dataframe, then write save the result to parquet
Function Function that unifies discrete and continuous metrics dataframe, then write save the result to parquet
: dataframe that contains all the discrete metrics
: dataframe that contains all the continuous metrics
: name of the domain
: schema of the initial data
: time which correspond to the ingestion
: stage (unit / global)