Class

io.github.setl.workflow

Pipeline

Related Doc: package workflow

Permalink

class Pipeline extends Logging with HasRegistry[Stage] with HasDescription with Identifiable with HasBenchmark with HasDiagram

Pipeline is a complete data transformation workflow.

Annotations
@Evolving()
Linear Supertypes
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. Pipeline
  2. HasDiagram
  3. HasBenchmark
  4. Identifiable
  5. HasDescription
  6. HasRegistry
  7. Logging
  8. AnyRef
  9. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new Pipeline()

    Permalink

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. var _benchmark: Option[Boolean]

    Permalink
    Attributes
    protected
    Definition Classes
    HasBenchmark
  5. def addStage(stage: Stage): Pipeline.this.type

    Permalink

    Add a new stage to the pipeline

    Add a new stage to the pipeline

    stage

    stage object

  6. def addStage[T <: Factory[_]](constructorArgs: Array[Any] = Array.empty, persistence: Boolean = true)(implicit arg0: ClassTag[T]): Pipeline.this.type

    Permalink

    Add a new stage containing the given factory

    Add a new stage containing the given factory

    T

    Class of the Factory

    constructorArgs

    Arguments of the primary constructor of the factory

    persistence

    if set to true, then the write method of the factory will be called to persist the output.

    Annotations
    @throws( ... )
    Exceptions thrown

    IllegalArgumentException this will be thrown if arguments don't match

  7. def addStage(factory: Class[_ <: Factory[_]], constructorArgs: Any*): Pipeline.this.type

    Permalink

    Add a new stage containing the given factory

    Add a new stage containing the given factory

    factory

    Factory to be executed

    constructorArgs

    Arguments of the primary constructor of the factory

    Annotations
    @throws( ... )
    Exceptions thrown

    IllegalArgumentException this will be thrown if arguments don't match

  8. def addStage(factory: Factory[_]): Pipeline.this.type

    Permalink

    Add a new stage containing the given factory

    Add a new stage containing the given factory

    factory

    Factory to be executed

  9. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  10. def benchmark(boo: Boolean): Pipeline.this.type

    Permalink

    Set to true to enable the benchmarking

    Set to true to enable the benchmarking

    boo

    true to enable benchmarking

    returns

    this object

    Definition Classes
    HasBenchmark
  11. def benchmark: Option[Boolean]

    Permalink

    True if the benchmark will be measured, otherwise false

    True if the benchmark will be measured, otherwise false

    returns

    boolean

    Definition Classes
    HasBenchmark
  12. def clearRegistry(): Unit

    Permalink

    Clear the registry

    Clear the registry

    Attributes
    protected
    Definition Classes
    HasRegistry
  13. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  14. def deliverableDispatcher: DeliverableDispatcher

    Permalink

    Get the deliverable dispatcher of this pipeline

  15. def describe(): Pipeline.this.type

    Permalink

    Describe the pipeline

    Describe the pipeline

    Definition Classes
    PipelineHasDescription
  16. def diagramId: String

    Permalink

    Get the diagram ID

    Get the diagram ID

    Definition Classes
    PipelineHasDiagram
  17. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  18. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  19. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  20. def formatDiagramId(prettyName: String, deliveryId: String, suffix: String): String

    Permalink
    Attributes
    protected
    Definition Classes
    HasDiagram
  21. def getBenchmarkResult: Array[BenchmarkResult]

    Permalink

    Get the aggregated benchmark result.

    Get the aggregated benchmark result.

    returns

    an array of BenchmarkResult

    Definition Classes
    PipelineHasBenchmark
  22. def getCanonicalName: String

    Permalink
    Definition Classes
    Identifiable
  23. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  24. def getDeliverable(t: scala.reflect.api.JavaUniverse.Type): Array[Deliverable[_]]

    Permalink

    Get the Deliverable of the given runtime Type

    Get the Deliverable of the given runtime Type

    t

    runtime type of the Deliverable's payload

  25. def getLastOutput: Any

    Permalink

    Get the output of the last factory of the last stage

    Get the output of the last factory of the last stage

    returns

    an object. it has to be convert to the according type manually.

  26. def getOutput[A](cls: Class[_ <: Factory[_]]): A

    Permalink

    Get the output of a specific Factory

    Get the output of a specific Factory

    cls

    class of the Factory

  27. def getPrettyName: String

    Permalink
    Definition Classes
    HasDescription
  28. def getRegisteredItem(uuid: UUID): Option[Stage]

    Permalink

    For a given UUID, return the corresponding registered item

    For a given UUID, return the corresponding registered item

    uuid

    uuid

    Definition Classes
    HasRegistry
  29. def getRegistry: ListMap[UUID, Stage]

    Permalink

    Return the registry

    Return the registry

    Definition Classes
    HasRegistry
  30. def getRegistryLength: Long

    Permalink

    Return the number of items in the current registry

    Return the number of items in the current registry

    Definition Classes
    HasRegistry
  31. def getStage(id: Int): Option[Stage]

    Permalink
  32. def getTypeArgList(tpe: scala.reflect.api.JavaUniverse.Type): List[scala.reflect.api.JavaUniverse.Symbol]

    Permalink
    Attributes
    protected
    Definition Classes
    HasDiagram
  33. def getUUID: UUID

    Permalink
    Definition Classes
    Identifiable
  34. def hasRegisteredItem(uuid: UUID): Boolean

    Permalink

    Check if the UUID exists in the registry

    Check if the UUID exists in the registry

    uuid

    an UUID

    returns

    true if it already exists in the registry, false otherwise

    Definition Classes
    HasRegistry
  35. def hasRegisteredItem(item: Identifiable): Boolean

    Permalink

    Check if the Identifiable exists in the registry

    Check if the Identifiable exists in the registry

    item

    an object that inherit io.github.setl.internal.Identifiable

    returns

    true if it already exists in the registry, false otherwise

    Definition Classes
    HasRegistry
  36. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  37. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  38. def isRegistryEmpty: Boolean

    Permalink

    Return true if the registry is empty, false otherwise

    Return true if the registry is empty, false otherwise

    Definition Classes
    HasRegistry
  39. def lastRegisteredItem: Option[Stage]

    Permalink

    Return the last registered item

    Return the last registered item

    returns

    if the registry is empty, None will be returned

    Definition Classes
    HasRegistry
  40. def log: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  41. def logDebug(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  42. def logError(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  43. def logInfo(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  44. def logName: String

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  45. def logTrace(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  46. def logWarning(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  47. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  48. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  49. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  50. def optimization(boolean: Boolean): Pipeline.this.type

    Permalink

    Set to true to allow auto optimization of this pipeline.

    Set to true to allow auto optimization of this pipeline. The default SimplePipelineOptimizer will be used.

    boolean

    true to allow optimization, otherwise false

    returns

    this pipeline

  51. def optimization(optimizer: PipelineOptimizer): Pipeline.this.type

    Permalink

    Optimise execution with an optimizer

    Optimise execution with an optimizer

    optimizer

    an implementation of pipeline optimizer

    returns

    this pipeline

  52. def optimization: Boolean

    Permalink

    Boolean indicating if the pipeline will be optimized

  53. def pipelineInspector: PipelineInspector

    Permalink

    Get the inspector of this pipeline

  54. def registerNewItem(item: Stage): Unit

    Permalink

    Register a new io.github.setl.internal.Identifiable in registry

    Register a new io.github.setl.internal.Identifiable in registry

    item

    an object that inherit io.github.setl.internal.Identifiable

    returns

    true if the given item is registered, false otherwise

    Attributes
    protected
    Definition Classes
    HasRegistry
    Annotations
    @throws( ... )
  55. def registerNewItems(items: Iterable[Stage]): Unit

    Permalink

    Register multiple items

    Register multiple items

    items

    an io.github.setl.internal.Identifiable object

    Attributes
    protected
    Definition Classes
    HasRegistry
  56. def run(): Pipeline.this.type

    Permalink

    Execute the pipeline

  57. def setInput[T](payload: T, deliveryId: String)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Pipeline.this.type

    Permalink

    Set input of the pipeline

    Set input of the pipeline

    T

    type of payload

    payload

    object to be delivered

    deliveryId

    id of this delivery

  58. def setInput[T](payload: T)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Pipeline.this.type

    Permalink

    Set input of the pipeline

    Set input of the pipeline

    T

    type of payload

    payload

    object to be delivered

  59. def setInput[T](payload: T, consumer: Class[_ <: Factory[_]], deliveryId: String)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Pipeline.this.type

    Permalink

    Set input of the pipeline

    Set input of the pipeline

    T

    type of payload

    payload

    object to be delivered

    consumer

    consumer of this payload

    deliveryId

    id of this delivery

  60. def setInput[T](payload: T, consumer: Class[_ <: Factory[_]])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Pipeline.this.type

    Permalink

    Set input of the pipeline

    Set input of the pipeline

    T

    type of payload

    payload

    object to be delivered

    consumer

    consumer of this payload

  61. def setInput[T](payload: T, consumer: Class[_ <: Factory[_]], consumers: Class[_ <: Factory[_]]*)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Pipeline.this.type

    Permalink

    Set input of the pipeline

    Set input of the pipeline

    T

    type of payload

    payload

    object to be delivered

    consumer

    consumer of this payload

    consumers

    other consumers of the payload

  62. def setInput[T](payload: T, deliveryId: String, consumer: Class[_ <: Factory[_]], consumers: Class[_ <: Factory[_]]*)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Pipeline.this.type

    Permalink

    Set input of the pipeline

    Set input of the pipeline

    T

    type of payload

    payload

    object to be delivered

    deliveryId

    id of this delivery

    consumer

    consumer of this payload

    consumers

    other consumers of the payload

  63. def setInput[T](payload: T, consumer: Seq[Class[_ <: Factory[_]]], deliveryId: String)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Pipeline.this.type

    Permalink

    Set input of the pipeline

    Set input of the pipeline

    T

    type of payload

    payload

    object to be delivered

    consumer

    consumer of this payload

    deliveryId

    id of this delivery

  64. def setInput(deliverable: Deliverable[_]): Pipeline.this.type

    Permalink

    Put the Deliverable to the input pool of the pipeline

    Put the Deliverable to the input pool of the pipeline

    deliverable

    Deliverable object

  65. def showDiagram(): Unit

    Permalink

    Display the diagram

    Display the diagram

    Definition Classes
    PipelineHasDiagram
  66. def stages: List[Stage]

    Permalink

    Return all the stages of this pipeline

  67. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  68. def toDiagram: String

    Permalink

    Generate the diagram

    Generate the diagram

    Definition Classes
    PipelineHasDiagram
  69. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  70. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  71. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  72. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from HasDiagram

Inherited from HasBenchmark

Inherited from Identifiable

Inherited from HasDescription

Inherited from HasRegistry[Stage]

Inherited from Logging

Inherited from AnyRef

Inherited from Any

Ungrouped