Adds actions that are preformed when all data is fully committed/moved into permanent storage.
Adds actions that are preformed when all data is fully committed/moved into permanent storage. Can be used to do cleanup operations.
logical name of the commit
A UUID generated at runtime unique to a commit name
labels that were committed
data flow to which to add finalise actions
data flow with finalise actions
Adds actions to the flow that move data to the permanent storage, simulating a wave commit
Adds actions to the flow that move data to the permanent storage, simulating a wave commit
logical name of the commit
A UUID generated at runtime unique to a commit name
labels to move
data flow to which move actions are added to
data flow with move actions
Adds cache actions to the flow.
Adds cache actions to the flow.
logical name of the commit
A UUID generated at runtime unique to a commit name
labels to cache
data flow to which the caching actions are added to
data flow with caching actions
Checks if data flow that will be used to cache, commit and finalise steps contains all necessary configurations.
Checks if data flow that will be used to cache, commit and finalise steps contains all necessary configurations.
data flow to validate
Defines phases of each data committers in the data flow, which are: 1) validate that committer is properly configured 2) cache the labels in temp area 3) when flow is successful, move all cached data into its permanent storage 4) finalise or cleanup after the committer had committed all of the data into permanent storage
Created by Alexei Perelighin