feed selector of the run
application name of the run
runId of the run. Stays 1 if recovery is not enabled.
attemptId of the run. Stays 1 if recovery is not enabled.
registry of all SmartDataLake objects parsed from the config
timestamp used as reference in certain actions (e.g. HistorizeAction)
the command line parameters parsed into a SmartDataLakeBuilderConfig object
start time of the run
start time of attempt
true if this is a simulation run
current execution phase
Counter how many times a DataFrame of a SparkSubFeed is reused by an Action later in the pipeline. The counter is increased during ExecutionPhase.Init when preparing the SubFeeds for an Action and it is decreased in ExecutionPhase.Exec to unpersist the DataFrame after there is no need for it anymore.
the command line parameters parsed into a SmartDataLakeBuilderConfig object
application name of the run
attemptId of the run.
attemptId of the run. Stays 1 if recovery is not enabled.
start time of attempt
Counter how many times a DataFrame of a SparkSubFeed is reused by an Action later in the pipeline.
Counter how many times a DataFrame of a SparkSubFeed is reused by an Action later in the pipeline. The counter is increased during ExecutionPhase.Init when preparing the SubFeeds for an Action and it is decreased in ExecutionPhase.Exec to unpersist the DataFrame after there is no need for it anymore.
feed selector of the run
registry of all SmartDataLake objects parsed from the config
current execution phase
timestamp used as reference in certain actions (e.g.
timestamp used as reference in certain actions (e.g. HistorizeAction)
runId of the run.
runId of the run. Stays 1 if recovery is not enabled.
start time of the run
true if this is a simulation run
ActionPipelineContext contains start and runtime information about a SmartDataLake run.
feed selector of the run
application name of the run
runId of the run. Stays 1 if recovery is not enabled.
attemptId of the run. Stays 1 if recovery is not enabled.
registry of all SmartDataLake objects parsed from the config
timestamp used as reference in certain actions (e.g. HistorizeAction)
the command line parameters parsed into a SmartDataLakeBuilderConfig object
start time of the run
start time of attempt
true if this is a simulation run
current execution phase
Counter how many times a DataFrame of a SparkSubFeed is reused by an Action later in the pipeline. The counter is increased during ExecutionPhase.Init when preparing the SubFeeds for an Action and it is decreased in ExecutionPhase.Exec to unpersist the DataFrame after there is no need for it anymore.