KEY_IN
- the type of input key from the Batch runVAL_IN
- the type of input value from the Batch runOUT
- the type of output for the source@Beta public abstract class BatchSource<KEY_IN,VAL_IN,OUT> extends BatchConfigurable<BatchSourceContext> implements Transformation<KeyValue<KEY_IN,VAL_IN>,OUT>, StageLifecycle<BatchRuntimeContext>
initialize(io.cdap.cdap.etl.api.batch.BatchRuntimeContext)
, transform(io.cdap.cdap.api.dataset.lib.KeyValue<KEY_IN, VAL_IN>, io.cdap.cdap.etl.api.Emitter<OUT>)
and destroy()
methods are called
inside the Batch Run while BatchConfigurable.prepareRun(T)
and BatchConfigurable.onRunFinish(boolean, T)
methods are called
on the client side, which launches the Batch run, before the Batch run starts and after it finishes respectively.Modifier and Type | Field and Description |
---|---|
static String |
FORMAT_PLUGIN_TYPE |
static String |
PLUGIN_TYPE |
Constructor and Description |
---|
BatchSource() |
Modifier and Type | Method and Description |
---|---|
void |
destroy()
Destroy the Batch Source stage.
|
void |
initialize(BatchRuntimeContext context)
Initialize the Batch Source stage.
|
void |
transform(KeyValue<KEY_IN,VAL_IN> input,
Emitter<OUT> emitter)
Transform the
KeyValue pair produced by the input, as set in BatchConfigurable.prepareRun(T) ,
to a single object and emit it to the next stage. |
configurePipeline, onRunFinish, prepareRun
public static final String PLUGIN_TYPE
public static final String FORMAT_PLUGIN_TYPE
public void initialize(BatchRuntimeContext context) throws Exception
transform(io.cdap.cdap.api.dataset.lib.KeyValue<KEY_IN, VAL_IN>, io.cdap.cdap.etl.api.Emitter<OUT>)
are made.initialize
in interface StageLifecycle<BatchRuntimeContext>
context
- BatchRuntimeContext
Exception
- if there is any error during initializationpublic void transform(KeyValue<KEY_IN,VAL_IN> input, Emitter<OUT> emitter) throws Exception
KeyValue
pair produced by the input, as set in BatchConfigurable.prepareRun(T)
,
to a single object and emit it to the next stage. By default it emits the value.
This method is invoked inside the Batch run.public void destroy()
destroy
in interface Destroyable
Copyright © 2023 Cask Data, Inc. Licensed under the Apache License, Version 2.0.