T
- batch execution context@Beta public abstract class BatchConfigurable<T extends BatchContext> extends Object implements PipelineConfigurable, SubmitterLifecycle<T>
Constructor and Description |
---|
BatchConfigurable() |
Modifier and Type | Method and Description |
---|---|
void |
configurePipeline(PipelineConfigurer pipelineConfigurer)
Configure an ETL pipeline, registering datasets and plugins that the stage needs.
|
void |
onRunFinish(boolean succeeded,
T context)
Invoked after the Batch run finishes.
|
abstract void |
prepareRun(T context)
Prepare the Batch run.
|
public void configurePipeline(PipelineConfigurer pipelineConfigurer)
PipelineConfigurable
ValidationException
if there are any
invalid config properties, or if the input or output schema is not compatible. Output schema should also be set.
This method is called many times during the lifecycle of a pipeline so it should not generate any side effects.
When the pipeline is being constructed, this is called in order to validate the pipeline and
propagate schema. Any datasets registered at this time will be ignored. Config properties that contain macros
will not have been evaluated yet.
When the pipeline is deployed, this is called in order to validate the pipeline and create any datasets that
are registered. Config properties that contain macros will not have been evaluated yet.configurePipeline
in interface PipelineConfigurable
pipelineConfigurer
- the configurer used to register required datasets and pluginspublic abstract void prepareRun(T context) throws Exception
prepareRun
in interface SubmitterLifecycle<T extends BatchContext>
context
- batch execution contextException
- if there's an error during this method invocationpublic void onRunFinish(boolean succeeded, T context)
onRunFinish
in interface SubmitterLifecycle<T extends BatchContext>
succeeded
- defines the result of batch execution: true if run succeeded, false otherwisecontext
- batch execution contextCopyright © 2021 Cask Data, Inc. Licensed under the Apache License, Version 2.0.