io.cdap.cdap.etl.api.batch

Class BatchSink<IN,KEY_OUT,VAL_OUT>

java.lang.Object

io.cdap.cdap.etl.api.batch.BatchConfigurable<BatchSinkContext>

io.cdap.cdap.etl.api.batch.BatchSink<IN,KEY_OUT,VAL_OUT>

Type Parameters:

IN - the type of input object to the sink

KEY_OUT - the type of key the sink outputs

VAL_OUT - the type of value the sink outputs

All Implemented Interfaces:

Destroyable, PipelineConfigurable, StageLifecycle<BatchRuntimeContext>, SubmitterLifecycle<BatchSinkContext>, Transformation<IN,KeyValue<KEY_OUT,VAL_OUT>>

@Beta
public abstract class BatchSink<IN,KEY_OUT,VAL_OUT>
extends BatchConfigurable<BatchSinkContext>
implements Transformation<IN,KeyValue<KEY_OUT,VAL_OUT>>, StageLifecycle<BatchRuntimeContext>

Batch Sink forms the last stage of a Batch ETL Pipeline. In addition to configuring the Batch run, it also transforms a single input object into a key value pair that the Batch run will output. By default, the input object is used as both the key and value. initialize(io.cdap.cdap.etl.api.batch.BatchRuntimeContext), transform(IN, io.cdap.cdap.etl.api.Emitter<io.cdap.cdap.api.dataset.lib.KeyValue<KEY_OUT, VAL_OUT>>) and destroy() methods are called inside the Batch Run while BatchConfigurable.prepareRun(T) and BatchConfigurable.onRunFinish(boolean, T) methods are called on the client side, which launches the Batch run, before the Batch run starts and after it finishes respectively.

Field Summary

Fields

Modifier and Type	Field and Description
static String	FORMAT_PLUGIN_TYPE
static String	PLUGIN_TYPE

Constructor Summary

Constructors

Constructor and Description
BatchSink()

Method Summary

Modifier and Type	Method and Description
void	destroy() Destroy the Batch Sink stage.
void	initialize(BatchRuntimeContext context) Initialize the Batch Sink stage.
void	transform(IN input, Emitter<KeyValue<KEY_OUT,VAL_OUT>> emitter) Transform the input received from previous stage to a KeyValue pair which can be consumed by the output, as set in BatchConfigurable.prepareRun(T).

Methods inherited from class io.cdap.cdap.etl.api.batch.BatchConfigurable

configurePipeline, onRunFinish, prepareRun

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PLUGIN_TYPE

public static final String PLUGIN_TYPE

See Also:

Constant Field Values

FORMAT_PLUGIN_TYPE

public static final String FORMAT_PLUGIN_TYPE

See Also:

Constant Field Values

Constructor Detail

BatchSink

public BatchSink()

Method Detail

initialize

public void initialize(BatchRuntimeContext context)
                throws Exception

Initialize the Batch Sink stage. Executed inside the Batch Run. This method is guaranteed to be invoked before any calls to transform(IN, io.cdap.cdap.etl.api.Emitter<io.cdap.cdap.api.dataset.lib.KeyValue<KEY_OUT, VAL_OUT>>) are made.

Specified by:

initialize in interface StageLifecycle<BatchRuntimeContext>

Parameters:

context - BatchRuntimeContext

Throws:

Exception - if there is any error during initialization

transform

public void transform(IN input,
                      Emitter<KeyValue<KEY_OUT,VAL_OUT>> emitter)
               throws Exception

Transform the input received from previous stage to a KeyValue pair which can be consumed by the output, as set in BatchConfigurable.prepareRun(T). By default, the input object is used as both key and value. This method is invoked inside the Batch run.

Specified by:

transform in interface Transformation<IN,KeyValue<KEY_OUT,VAL_OUT>>

Parameters:

input - the input to transform

emitter - Emitter to emit data to the next stage

Throws:

Exception - if there's an error during this method invocation

destroy

public void destroy()

Destroy the Batch Sink stage. Executed at the end of the Batch Run.

Specified by:

destroy in interface Destroyable