c

org.codefeedr.pipeline

PipelineBuilder

class PipelineBuilder extends Logging

Mutable class to build a Pipeline. A pipeline represents a set of stages interconnected with a org.codefeedr.buffer.Buffer. This pipeline gets translated into a DirectedAcyclicGraph.

This builder allows for setting the following properties: - Name of the pipeline. - Type of buffer (e.g. Kafka) and properties for this buffer. - Type of pipeline: sequential or DAG (nonsequential). - Properties for all the stages. - KeyManager for all the stages.

Linear Supertypes
Logging, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. PipelineBuilder
  2. Logging
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new PipelineBuilder()

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. def addParents[In <: Serializable, Mid1 <: Serializable, Mid2 <: Serializable, Out <: Serializable](stage: Stage[In, Mid1], parent: Stage[Mid2, Out]): PipelineBuilder

    Add a parent to a node.

    Add a parent to a node.

    In

    Incoming type from 'start' stage.

    Mid1

    Intermediate type which is output for 'start' stage.

    Mid2

    Intermediate type which is output for 'end' stage.

    Out

    Outgoing type for 'end' stage.

    stage

    The child stage.

    parent

    The parent stage.

    returns

    The builder instance.

  5. def addParents[In <: Serializable, Out <: Serializable](stage: Stage[In, Out], parents: StageList): PipelineBuilder

    Add multiple parents (stages) in given ordered list.

    Add multiple parents (stages) in given ordered list.

    In

    Incoming type of the stage.

    Out

    Outgoing type of the stage.

    stage

    Stage to add the parents to.

    parents

    Parents to connect to the child stage.

    returns

    The builder instance.

  6. def append[In <: Serializable](trans: (DataStream[In]) ⇒ Any)(implicit arg0: ClassTag[In], arg1: scala.reflect.api.JavaUniverse.TypeTag[In]): PipelineBuilder

    Append a stage created from an output function.

    Append a stage created from an output function.

    In

    Type of the DataStream.

    trans

    Output function.

    returns

    The builder instance.

  7. def append[In <: Serializable, Out <: Serializable](trans: (DataStream[In]) ⇒ DataStream[Out])(implicit arg0: ClassTag[In], arg1: scala.reflect.api.JavaUniverse.TypeTag[In], arg2: ClassTag[Out], arg3: scala.reflect.api.JavaUniverse.TypeTag[Out]): PipelineBuilder

    Append a stage created from a transform function.

    Append a stage created from a transform function.

    In

    Incoming type of the DataStream.

    Out

    Outgoing type of the DataStream.

    trans

    Transform function from one type to another.

    returns

    The builder instance.

  8. def append[IN <: Serializable, OUT <: Serializable](stage: Stage[IN, OUT]): PipelineBuilder

    Append a Stage in a sequential pipeline.

    Append a Stage in a sequential pipeline.

    IN

    Incoming type of the stage.

    OUT

    Outgoing type of the stage.

    stage

    The new stage to add.

    returns

    This builder instance.

  9. def appendSource[In <: Serializable](input: (StreamExecutionEnvironment) ⇒ DataStream[In])(implicit arg0: ClassTag[In], arg1: scala.reflect.api.JavaUniverse.TypeTag[In]): PipelineBuilder

    Append a stage created from an input function.

    Append a stage created from an input function.

    In

    Type of the DataStream.

    input

    Input function.

    returns

    The builder instance.

  10. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  11. var bufferProperties: Properties

    Properties of the buffer

    Properties of the buffer

    Attributes
    protected[org.codefeedr.pipeline]
  12. var bufferType: BufferType

    Type of buffer used in the pipeline

    Type of buffer used in the pipeline

    Attributes
    protected
  13. def build(): Pipeline

    Builds a pipeline from the builder configuration.

    Builds a pipeline from the builder configuration.

    returns

    A new Pipeline.

    Exceptions thrown

    EmptyPipelineException When no pipeline is defined (graph is empty).

  14. def clone(): AnyRef
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @native() @throws( ... )
  15. def disablePipelineVerification(): PipelineBuilder

    Disables the pipeline verification.

    Disables the pipeline verification. This is not recommended, it allows for nasty pipelines.

    returns

    The builder instance.

  16. def edge[In <: Serializable, Mid1 <: Serializable, Mid2 <: Serializable, Out <: Serializable](from: Stage[In, Mid1], to: Stage[Mid2, Out]): PipelineBuilder

    Creates an edge between two stages.

    Creates an edge between two stages. The 'to' must not already have a parent.

    If the graph is not configured yet (has no nodes), the graph is switched to a DAG automatically. If it was already configured as sequential, it will throw an IllegalStateException.

    In

    Incoming type from 'start' stage.

    Mid1

    Intermediate type which is output for 'start' stage.

    Mid2

    Intermediate type which is output for 'end' stage.

    Out

    Outgoing type for 'end' stage.

    from

    The stage to start from.

    to

    The stage to end at.

  17. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  18. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  19. def finalize(): Unit
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  20. def getBufferType: BufferType

    Get the type of the buffer.

    Get the type of the buffer.

    returns

    The buffer type.

  21. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  22. def getPipelineType: PipelineType

    Get the type of the pipeline.

    Get the type of the pipeline.

    returns

    Type of pipeline: Sequential or DAG.

  23. var graph: DirectedAcyclicGraph

    Graph of the pipeline

    Graph of the pipeline

    Attributes
    protected[org.codefeedr.pipeline]
  24. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  25. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  26. var keyManager: KeyManager

    Key manager

    Key manager

    Attributes
    protected
  27. val logger: Logger
    Attributes
    protected
    Definition Classes
    Logging
  28. var name: String

    The name of the pipeline, "CodeFeedr pipeline" by default.

    The name of the pipeline, "CodeFeedr pipeline" by default.

    Attributes
    protected
  29. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  30. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  31. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  32. var pipelineType: PipelineType

    Type of the pipeline graph

    Type of the pipeline graph

    Attributes
    protected
  33. var pipelineVerificationToggle: Boolean

    Pipeline verification toggle, defaults to true *

    Pipeline verification toggle, defaults to true *

    Attributes
    protected
  34. def setBufferProperty(key: String, value: String): PipelineBuilder

    Set a buffer property.

    Set a buffer property. A buffer property is generic for all buffers (like the serializer).

    key

    Key of the property.

    value

    Value of the property.

    returns

    This builder instance.

  35. def setBufferType(bufferType: BufferType): PipelineBuilder

    Set the type of the buffer.

    Set the type of the buffer.

    bufferType

    The new buffer type.

    returns

    The builder instance.

  36. def setKeyManager(km: KeyManager): PipelineBuilder

    Set a KeyManager.

    Set a KeyManager. A key manager handles API key management and can be accessed by every stage.

    km

    The key manager instance.

    returns

    This builder instance.

  37. def setPipelineName(name: String): PipelineBuilder

    Set name of the pipeline.

    Set name of the pipeline.

    name

    Name of the pipeline.

    returns

    This builder instance.

  38. def setPipelineType(pipelineType: PipelineType): PipelineBuilder

    Set the type of the pipeline.

    Set the type of the pipeline.

    pipelineType

    Type of the pipeline.

    returns

    This builder instance.

  39. def setSerializer(serializer: SerializerType): PipelineBuilder

    Sets the serializer type for the buffer.

    Sets the serializer type for the buffer.

    serializer

    The serializer type (which is basically a string).

    returns

    This builder instance.

  40. def setStageProperty(stage: Stage[_, _], key: String, value: String): PipelineBuilder

    Set a stage property.

    Set a stage property.

    key

    Key of the property.

    value

    Value of the property.

    returns

    This builder instance.

  41. def setStreamTimeCharacteristic(timeCharacteristic: TimeCharacteristic): PipelineBuilder

    Set TimeCharacteristic of the whole pipeline.

    Set TimeCharacteristic of the whole pipeline.

    timeCharacteristic

    The TimeCharacterisic.

    returns

    This builder instance.

  42. val stageProperties: HashMap[String, Properties]

    Stage properties

    Stage properties

    Attributes
    protected
  43. var streamTimeCharacteristic: TimeCharacteristic

    The StreamTimeCharacteristic.

    The StreamTimeCharacteristic. Default: TimeCharacteristic.EventTime

    Attributes
    protected
  44. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  45. def toString(): String
    Definition Classes
    AnyRef → Any
  46. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  47. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  48. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @throws( ... )

Inherited from Logging

Inherited from AnyRef

Inherited from Any

Ungrouped