This methods fires at the end of the stage and collects metrics flattened into the stageMetricsData ListBuffer Note all times are in ms, cpu time and shuffle write time are originally in nanosec, thus in the code are divided by 1e6
This methods fires at the end of the stage and collects metrics flattened into the stageMetricsData ListBuffer Note all times are in ms, cpu time and shuffle write time are originally in nanosec, thus in the code are divided by 1e6