Class

io.epiphanous.flinkrunner.flink

FlinkJob

Related Doc: package flink

Permalink

abstract class FlinkJob[IN <: FlinkEvent, OUT <: FlinkEvent] extends LazyLogging

An abstract flink job to transform on a stream of events from an algebraic data type (ADT).

IN

The type of input stream elements

OUT

The type of output stream elements

Linear Supertypes
LazyLogging, AnyRef, Any
Known Subclasses
DataControlJob, FilterByControlJob, IdentityJob
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. FlinkJob
  2. LazyLogging
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new FlinkJob()(implicit arg0: TypeInformation[IN], arg1: TypeInformation[OUT])

    Permalink

Abstract Value Members

  1. abstract def transform(in: DataStream[IN])(implicit config: FlinkConfig, env: SEE): DataStream[OUT]

    Permalink

    Primary method to transform the source data stream into the output data stream.

    Primary method to transform the source data stream into the output data stream. The output of this method is passed into sink(). This method must be overridden by subclasses.

    in

    input data stream created by source()

    config

    implicit flink job config

    returns

    output data stream

Concrete Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  5. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  6. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  7. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  8. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  9. def flow()(implicit config: FlinkConfig, env: SEE): DataStream[OUT]

    Permalink

    A pipeline for transforming a single stream.

    A pipeline for transforming a single stream. Passes the output of source() through transform() and the result of that into maybeSink(), which may pass it into sink() if we're not testing. Ultimately, returns the output data stream to facilitate testing.

    config

    implicit flink job config

    returns

    data output stream

  10. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  11. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  12. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  13. lazy val logger: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    LazyLogging

  14. def maybeAssignTimestampsAndWatermarks(in: DataStream[IN])(implicit config: FlinkConfig, env: SEE): Unit

    Permalink

  15. def maybeSink(out: DataStream[OUT])(implicit config: FlinkConfig, env: SEE): Unit

    Permalink

    The output stream will only be passed to sink() if FlinkConfig.mockEdges evaluates to false (ie, you're not testing).

    The output stream will only be passed to sink() if FlinkConfig.mockEdges evaluates to false (ie, you're not testing).

    out

    the output data stream to pass into sink()

    config

    implicit flink job config

  16. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  17. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  18. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  19. def run()(implicit config: FlinkConfig, env: SEE): Either[Iterator[OUT], Unit]

    Permalink

  20. def sink(out: DataStream[OUT])(implicit config: FlinkConfig, env: SEE): Unit

    Permalink

    Writes the transformed data stream to configured output sinks.

    Writes the transformed data stream to configured output sinks. *

    out

    a transformed stream from transform()

    config

    implicit flink job config

  21. def source()(implicit config: FlinkConfig, env: SEE): DataStream[IN]

    Permalink

    Returns source data stream to pass into transform().

    Returns source data stream to pass into transform(). This can be overridden by subclasses.

    returns

    input data stream

  22. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  23. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any

  24. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  25. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  26. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from LazyLogging

Inherited from AnyRef

Inherited from Any

Ungrouped