Interface CheckpointStatsTracker

  • All Known Implementing Classes:
    DefaultCheckpointStatsTracker, NoOpCheckpointStatsTracker

    public interface CheckpointStatsTracker
    Tracker for checkpoint statistics.

    This is tightly integrated with the CheckpointCoordinator in order to ease the gathering of fine-grained statistics.

    The tracked stats include summary counts, a detailed history of recent and in progress checkpoints as well as summaries about the size, duration and more of recent checkpoints.

    Data is gathered via callbacks in the CheckpointCoordinator and related classes like PendingCheckpoint and CompletedCheckpoint, which receive the raw stats data in the first place.

    The statistics are accessed via createSnapshot() and exposed via both the web frontend and the Metric system.

    • Method Detail

      • reportRestoredCheckpoint

        void reportRestoredCheckpoint​(long checkpointID,
                                      CheckpointProperties properties,
                                      String externalPath,
                                      long stateSize)
      • reportCompletedCheckpoint

        void reportCompletedCheckpoint​(CompletedCheckpointStats completed)
        Callback when a checkpoint completes.
        Parameters:
        completed - The completed checkpoint stats.
      • getPendingCheckpointStats

        @Nullable
        PendingCheckpointStats getPendingCheckpointStats​(long checkpointId)
      • reportInitializationStarted

        void reportInitializationStarted​(Set<ExecutionAttemptID> toInitialize,
                                         long initializationStartTs)
      • reportPendingCheckpoint

        @Nullable
        PendingCheckpointStats reportPendingCheckpoint​(long checkpointId,
                                                       long triggerTimestamp,
                                                       CheckpointProperties props,
                                                       Map<JobVertexID,​Integer> vertexToDop)
        Creates a new pending checkpoint tracker.
        Parameters:
        checkpointId - ID of the checkpoint.
        triggerTimestamp - Trigger timestamp of the checkpoint.
        props - The checkpoint properties.
        vertexToDop - mapping of JobVertexID to DOP
        Returns:
        Tracker for statistics gathering or null if no stats were tracked.
      • reportFailedCheckpointsWithoutInProgress

        void reportFailedCheckpointsWithoutInProgress()
        Callback when a checkpoint failure without in progress checkpoint. For example, it should be callback when triggering checkpoint failure before creating PendingCheckpoint.
      • createSnapshot

        CheckpointStatsSnapshot createSnapshot()
        Creates a new snapshot of the available stats.
        Returns:
        The latest statistics snapshot.