class KafkaTopic[T <: Serializable] extends FakeStage[T]
KafkaTopic stage which only embodies the topic name, to let a linked stage read from this topic.
- Alphabetic
- By Inheritance
- KafkaTopic
- FakeStage
- InputStage
- Stage
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Instance Constructors
- new KafkaTopic(topic: String)(implicit arg0: ClassTag[T], arg1: scala.reflect.api.JavaUniverse.TypeTag[T])
Value Members
-
def
:+[U <: Serializable, V <: Serializable](stage: Stage[U, V]): StageList
Create a list of stages by appending another stage.
Create a list of stages by appending another stage.
- stage
The other stage to add.
- returns
A new StageList with the stage added.
- Definition Classes
- Stage
-
def
getContext: Context
Get the Context of this stage.
Get the Context of this stage.
- Definition Classes
- Stage
-
def
getMainSource(groupId: String = null): DataStream[Nothing]
Returns the (main)buffer source of this stage.
Returns the (main)buffer source of this stage. The main source is the first parent of this stage. Other sources need to be joined in the Flink job.
- returns
The DataStream resulting from the buffer.
- Definition Classes
- Stage
-
def
getParents: Vector[Stage[Serializable, Serializable]]
Get all parents of this stage.
Get all parents of this stage.
- returns
The parents of this stage, can be empty.
- Definition Classes
- Stage
-
def
getSink(groupId: String = null): SinkFunction[T]
Returns the buffer sink of this stage.
Returns the buffer sink of this stage.
- returns
The SinkFunction resulting from the buffer.
- Definition Classes
- Stage
-
def
getSource[T <: Serializable](parentNode: Stage[Serializable, Serializable])(implicit arg0: ClassTag[T], arg1: scala.reflect.api.JavaUniverse.TypeTag[T]): DataStream[T]
Returns the buffer source of this stage.
Returns the buffer source of this stage.
- returns
The DataStream resulting from the buffer.
- Definition Classes
- Stage
-
def
hasMainSource: Boolean
Check if this stage is sourced from a org.codefeedr.buffer.Buffer.
Check if this stage is sourced from a org.codefeedr.buffer.Buffer.
- returns
True if this stage has a Buffer source.
- Definition Classes
- Stage
-
def
hasSink: Boolean
Check if this stage is sinked to a org.codefeedr.buffer.Buffer.
Check if this stage is sinked to a org.codefeedr.buffer.Buffer.
- returns
True if this stage has a Buffer sink.
- Definition Classes
- Stage
-
def
inList: StageList
Create a StageList with this stage.
-
var
inTypes: List[scala.reflect.api.JavaUniverse.Type]
Keep track of all incoming types.
Keep track of all incoming types. *
- Definition Classes
- Stage
-
def
main(context: Context): DataStream[T]
This will never be run.
-
val
outType: scala.reflect.api.JavaUniverse.Type
Keep track of the outgoing type.
Keep track of the outgoing type. *
- Definition Classes
- Stage
-
def
setUp(pipeline: Pipeline): Unit
Setups the pipeline object with a pipeline.
Setups the pipeline object with a pipeline.
- pipeline
The pipeline it belongs to.
- Definition Classes
- Stage
-
val
stageId: Option[String]
- Definition Classes
- Stage
-
def
transform(source: DataStream[Nothing]): DataStream[T]
Transforms the stage from its input type to its output type.
Transforms the stage from its input type to its output type. This requires using the Flink DataStream API.
In this case it is already implemented as a helper method. To use
main(): DataStream[Out]instead.- source
The input source with type In. In the case of an InputStage it is a NoType.
- returns
The transformed stream with type Out.
- Definition Classes
- InputStage → Stage