io.cdap.cdap.etl.api

Interface StageContext

All Superinterfaces:

LineageRecorder, MetadataReader, MetadataWriter, ServiceDiscoverer

All Known Subinterfaces:

ActionContext, AlertPublisherContext, BatchActionContext, BatchAggregatorContext, BatchContext, BatchJoinerContext, BatchJoinerRuntimeContext, BatchRuntimeContext, BatchSinkContext, BatchSourceContext, ConditionContext, StageSubmitterContext, TransformContext

@Beta
public interface StageContext
extends ServiceDiscoverer, MetadataReader, MetadataWriter, LineageRecorder

Context for a pipeline stage, providing access to information about the stage, metrics, and plugins.

Method Summary

Modifier and Type	Method and Description
Arguments	getArguments() Return the pipeline arguments for this run.
default FailureCollector	getFailureCollector() Returns a failure collector for the stage.
io.cdap.cdap.api.data.schema.Schema	getInputSchema() Return the input schema for the stage.
Map<String,io.cdap.cdap.api.data.schema.Schema>	getInputSchemas() Return the input schemas for the stage.
default List<String>	getInputStages() Return the input stage names.
long	getLogicalStartTime() Returns the logical start time of the batch.
StageMetrics	getMetrics() Get an instance of StageMetrics, used to collect metrics for this stage.
String	getNamespace() Gets the namespace of the pipeline.
Map<String,io.cdap.cdap.api.data.schema.Schema>	getOutputPortSchemas() Return the output port schemas for the stage.
io.cdap.cdap.api.data.schema.Schema	getOutputSchema() Return the output schema of the stage, as set by this stage when the pipeline was deployed.
String	getPipelineName() Gets the name of the pipeline.
PluginProperties	getPluginProperties() Gets the PluginProperties associated with the stage.
PluginProperties	getPluginProperties(String pluginId) Gets the PluginProperties associated with the given plugin id.
String	getStageName() Gets the unique stage name.
<T> Class<T>	loadPluginClass(String pluginId) Loads and returns a plugin class as specified by the given plugin id.
<T> T	newPluginInstance(String pluginId) Creates a new instance of a plugin.

Methods inherited from interface io.cdap.cdap.api.ServiceDiscoverer

getServiceURL, getServiceURL, getServiceURL, openConnection

Methods inherited from interface io.cdap.cdap.api.metadata.MetadataReader

getMetadata, getMetadata

Methods inherited from interface io.cdap.cdap.api.metadata.MetadataWriter

addProperties, addTags, addTags, removeMetadata, removeProperties, removeProperties, removeTags, removeTags

Methods inherited from interface io.cdap.cdap.etl.api.lineage.field.LineageRecorder

record

Method Detail

getStageName

String getStageName()

Gets the unique stage name.

Returns:

stage name

getNamespace

String getNamespace()

Gets the namespace of the pipeline.

Returns:

the pipeline namespace

getPipelineName

String getPipelineName()

Gets the name of the pipeline.

Returns:

pipeline name

getLogicalStartTime

long getLogicalStartTime()

Returns the logical start time of the batch. For batch pipelines, this is the time the pipeline was triggered. For realtime pipelines, this is the time for the current microbatch being processed.

Returns:

Logical time in milliseconds since epoch time (00:00:00 January 1, 1970 UTC).

getMetrics

StageMetrics getMetrics()

Get an instance of StageMetrics, used to collect metrics for this stage. Metrics emitted from one stage are independent from metrics emitted in another.

Returns:

StageMetrics for collecting metrics

getPluginProperties

PluginProperties getPluginProperties()

Gets the PluginProperties associated with the stage.

Returns:

the PluginProperties.

getPluginProperties

PluginProperties getPluginProperties(String pluginId)

Gets the PluginProperties associated with the given plugin id.

Parameters:

pluginId - the unique identifier provide when declaring plugin usage in the program.

Returns:

the PluginProperties.

Throws:

IllegalArgumentException - if pluginId is not found

UnsupportedOperationException - if the program does not support plugin

loadPluginClass

<T> Class<T> loadPluginClass(String pluginId)

Loads and returns a plugin class as specified by the given plugin id.

Type Parameters:

T - the class type of the plugin

Parameters:

pluginId - the unique identifier provide when declaring plugin usage in the program.

Returns:

the resulting plugin Class.

Throws:

IllegalArgumentException - if pluginId is not found

UnsupportedOperationException - if the program does not support plugin

newPluginInstance

<T> T newPluginInstance(String pluginId)
                 throws InstantiationException

Creates a new instance of a plugin. The instance returned will have the PluginConfig setup with PluginProperties provided at the time when the PluginConfigurer.usePlugin(String, String, String, PluginProperties) was called during the program configuration time.

Type Parameters:

T - the class type of the plugin

Parameters:

pluginId - the unique identifier provide when declaring plugin usage in the program.

Returns:

A new instance of the plugin being specified by the arguments

Throws:

InstantiationException - if failed create a new instance

IllegalArgumentException - if pluginId is not found

UnsupportedOperationException - if the program does not support plugin

getInputSchema

@Nullable
io.cdap.cdap.api.data.schema.Schema getInputSchema()

Return the input schema for the stage. A null input schema indicates that the previous stages did not set a concrete output schema when the pipeline was deployed. This can either mean the input schema is unknown, or it can mean the schema is not constant.

Returns:

the input schema for the stage

getInputSchemas

Map<String,io.cdap.cdap.api.data.schema.Schema> getInputSchemas()

Return the input schemas for the stage. The map key is the stage name and the map value is the schema from that stage. A null input schema indicates that the stage did not set a concrete output schema when the pipeline was deployed. This can either mean the input schema is unknown, or it can mean the schema is not constant.

Returns:

the map of input stage names to their schema

getInputStages

default List<String> getInputStages()

Return the input stage names. Stage name will be set even if the corresponding input schema is unknown or not constant.

Returns:

the list of input stage names

getOutputSchema

@Nullable
io.cdap.cdap.api.data.schema.Schema getOutputSchema()

Return the output schema of the stage, as set by this stage when the pipeline was deployed. If none was set, null will be returned. A null schema indicates that the schema is not known, or that the output schema is not constant.

Returns:

the output schema of the stage

getOutputPortSchemas

Map<String,io.cdap.cdap.api.data.schema.Schema> getOutputPortSchemas()

Return the output port schemas for the stage. The map key is the port name and the map value is the schema for that port. This map is the same map that was set at configure time by MultiOutputPipelineConfigurable. Stages that do not implement MultiOutputPipelineConfigurable should not use this method. If they do, an empty map will be returned.

Returns:

the map of output ports to their schema

getArguments

Arguments getArguments()

Return the pipeline arguments for this run.

Returns:

the pipeline arguments for this run

getFailureCollector

default FailureCollector getFailureCollector()

Returns a failure collector for the stage.

Returns:

a failure collector

Throws:

UnsupportedOperationException - if the implementation does not override this method