io.cdap.cdap.etl.api

Interface PipelineConfigurable

All Known Implementing Classes:

Action, AlertPublisher, BatchAggregator, BatchConfigurable, BatchSink, BatchSource, Condition, ErrorTransform, PostAction, Transform

@Beta
public interface PipelineConfigurable

Allows the stage to configure pipeline.

Method Summary

Modifier and Type	Method and Description
void	configurePipeline(PipelineConfigurer pipelineConfigurer) Configure an ETL pipeline, registering datasets and plugins that the stage needs.

Method Detail

configurePipeline

void configurePipeline(PipelineConfigurer pipelineConfigurer)

Configure an ETL pipeline, registering datasets and plugins that the stage needs. Validation should be performed in this method, throwing a ValidationException if there are any invalid config properties, or if the input or output schema is not compatible. Output schema should also be set. This method is called many times during the lifecycle of a pipeline so it should not generate any side effects. When the pipeline is being constructed, this is called in order to validate the pipeline and propagate schema. Any datasets registered at this time will be ignored. Config properties that contain macros will not have been evaluated yet. When the pipeline is deployed, this is called in order to validate the pipeline and create any datasets that are registered. Config properties that contain macros will not have been evaluated yet.

Parameters:

pipelineConfigurer - the configurer used to register required datasets and plugins

Throws:

ValidationException - if the pipeline stage is invalid