StreamSubTransformer

Value Members

1. final def !=(arg0: AnyRef): Boolean

   Definition Classes

   AnyRef

2. final def !=(arg0: Any): Boolean

   Definition Classes

   Any

3. final def ##(): Int

   Definition Classes

   AnyRef → Any

4. def -->(targets: ConnectionTarget[DataStream, SparkStreamingRuntime]*): Unit

   Definition Classes

   ConnectionSource

5. def -->(tgt: MultiInputStep[DataStream, SparkStreamingRuntime]): tgt.type

   Definition Classes

   ConnectionSource

6. def -->(tgt: ConnectionTarget[DataStream, SparkStreamingRuntime]): tgt.type

   Definition Classes

   ConnectionSource

7. final def ==(arg0: AnyRef): Boolean

   Definition Classes

   AnyRef

8. final def ==(arg0: Any): Boolean

   Definition Classes

   Any

9. def addStepListener(listener: StepListener[DataStream, SparkStreamingRuntime]): Unit

   Registers a new listener.

   Registers a new listener.

   Definition Classes

   Step

10. def addStreamDataListener(listener: StreamStepDataListener): Unit

    Registers a data listener.

    Registers a data listener.

    Definition Classes

    StreamStep

11. def allInputsRequired: Boolean

    Specifies if the step should throw an error if one of the inputs is not connected.

    Specifies if the step should throw an error if one of the inputs is not connected.

    Definition Classes

    Step

12. final def asInstanceOf[T0]: T0

    Definition Classes

    Any

13. val body: (ConnectionTarget[DataStream, SparkStreamingRuntime], ConnectionSource[DataStream, SparkStreamingRuntime])

14. def clone(): AnyRef

    Attributes

    protected[java.lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

15. def compute(index: Int)(implicit runtime: SparkStreamingRuntime): DataStream

    Triggers listener notification on stream events.

    Triggers listener notification on stream events.

    index

    the output value index.

    Attributes

    protected

    Definition Classes

    StreamStep → Step

16. def compute(arg: DataStream)(implicit runtime: SparkStreamingRuntime): DataStream

    Attributes

    protected

    Definition Classes

    SubTransformer → Transformer

17. lazy val connections: Set[Connection[DataStream, SparkStreamingRuntime]]

    Collects all connections given a set of steps.

    Collects all connections given a set of steps.

    Definition Classes

    SubFlow

18. def ctx(implicit runtime: SparkStreamingRuntime): SQLContext

    Returns the implicit SQLContext.

    Returns the implicit SQLContext.

    Attributes

    protected

    Definition Classes

    StreamStep

19. final def eq(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

20. final def evaluate(implicit runtime: SparkStreamingRuntime): IndexedSeq[DataStream]

    Evaluates all step's outputs and returns a list of results.

    Evaluates all step's outputs and returns a list of results.

    Definition Classes

    Step

    Annotations

    @throws( classOf[ExecutionException] )

21. def finalize(): Unit

    Attributes

    protected[java.lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( classOf[java.lang.Throwable] )

22. def from(src: ConnectionSource[DataStream, SparkStreamingRuntime]): StreamSubTransformer.this.type

    Definition Classes

    ConnectionTarget

23. final def getClass(): Class[_]

    Definition Classes

    AnyRef → Any

24. lazy val inPoints: List[ConnectionTarget[DataStream, SparkStreamingRuntime]]

    Definition Classes

    SubTransformer → SubFlow

25. val index: Int

    Definition Classes

    Transformer → SingleInputStep → ConnectionTarget → SingleOutputStep → ConnectionSource

26. def input(implicit runtime: SparkStreamingRuntime): DataStream

    Definition Classes

    SingleInputStep

27. val inputCount: Int

    The maximum number of input ports.

    The maximum number of input ports.

    Definition Classes

    SingleInputStep → Step

28. final def isInstanceOf[T0]: Boolean

    Definition Classes

    Any

29. final def ne(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

30. final def notify(): Unit

    Definition Classes

    AnyRef

31. final def notifyAll(): Unit

    Definition Classes

    AnyRef

32. lazy val outPoints: List[ConnectionSource[DataStream, SparkStreamingRuntime]]

    Definition Classes

    SubTransformer → SubFlow

33. def outbounds(index: Int): SingleOutputStep[DataStream, SparkStreamingRuntime]

    Definition Classes

    SingleOutputStep

34. final def output(implicit runtime: SparkStreamingRuntime): DataStream

    Shortcut for output(0).

    Shortcut for output(0). Computes a step output at index 0.

    Definition Classes

    Step

    Annotations

    @throws( classOf[ExecutionException] )

    Exceptions thrown

    ExecutionException

    in case of an error, or if the step is not connected.

35. final def output(index: Int)(implicit runtime: SparkStreamingRuntime): DataStream

    Computes a step output value at the specified index.

    Computes a step output value at the specified index.

    index

    the output value index.

    Definition Classes

    Step

    Annotations

    @throws( classOf[ExecutionException] )

    Exceptions thrown

    ExecutionException

    in case of an error, or if the step is not connected.

36. val outputCount: Int

    The number of output ports.

    The number of output ports.

    Definition Classes

    SingleOutputStep → Step

37. def outputToJson(implicit idGen: (Step[DataStream, SparkStreamingRuntime]) ⇒ String): JValue

    Outputs subflow into JSON.

    Outputs subflow into JSON.

    Definition Classes

    SubFlow

38. def outputToXml(implicit idGen: (Step[DataStream, SparkStreamingRuntime]) ⇒ String): Elem

    Outputs the subflow into XML.

    Outputs the subflow into XML.

    Attributes

    protected

    Definition Classes

    SubFlow

39. def register(implicit rt: SparkStreamingRuntime): Unit

    Registers the step with the runtime.

    Registers the step with the runtime. Subsequent restarts will keep binding the step to newly created contexts.

    Definition Classes

    StreamStep

40. def removeStepListener(listener: StepListener[DataStream, SparkStreamingRuntime]): Unit

    Unregisters a listener.

    Unregisters a listener.

    Definition Classes

    Step

41. def removeStreamDataListener(listener: StreamStepDataListener): Unit

    Unregisters a data listener.

    Unregisters a data listener.

    Definition Classes

    StreamStep

42. def resetCache(predecessors: Boolean, descendants: Boolean): Unit

    Reset the cache of the subflow and its constituents.

    Reset the cache of the subflow and its constituents.

    Definition Classes

    SubFlow → Step

43. def sc(implicit runtime: SparkStreamingRuntime): SparkContext

    Returns the implicit SparkContext.

    Returns the implicit SparkContext.

    Attributes

    protected

    Definition Classes

    StreamStep

44. def ssc(implicit runtime: SparkStreamingRuntime): StreamingContext

    Returns the implicit StreamingContext.

    Returns the implicit StreamingContext.

    Attributes

    protected

    Definition Classes

    StreamStep

45. val step: Transformer[DataStream, SparkStreamingRuntime]

    Definition Classes

    Transformer → SingleInputStep → ConnectionTarget → SingleOutputStep → ConnectionSource

46. lazy val steps: Set[Step[DataStream, SparkStreamingRuntime]]

    Collects all steps starting with the targets and going back through the predecessors.

    Collects all steps starting with the targets and going back through the predecessors.

    Definition Classes

    SubFlow

47. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes

    AnyRef

48. val tag: String

    Definition Classes

    StreamSubFlow

49. def to(targets: ConnectionTarget[DataStream, SparkStreamingRuntime]*): Unit

    Definition Classes

    ConnectionSource

50. def to(tgt: MultiInputStep[DataStream, SparkStreamingRuntime]): tgt.type

    Definition Classes

    ConnectionSource

51. def to(tgt: ConnectionTarget[DataStream, SparkStreamingRuntime]): tgt.type

    Definition Classes

    ConnectionSource

52. def toJson: JValue

    The default implementation uses default id generator.

    The default implementation uses default id generator.

    Definition Classes

    SubFlow → JsonExport

53. def toXml: Elem

    The default implementation uses default id generator.

    The default implementation uses default id generator.

    Definition Classes

    SubFlow → XmlExport

54. def unregister(implicit rt: SparkStreamingRuntime): Unit

    Unregisters the step with the runtime.

    Unregisters the step with the runtime. After the restart, the step will no longer be bound to the active context.

    Definition Classes

    StreamStep

55. def value(implicit runtime: SparkStreamingRuntime): DataStream

    Definition Classes

    SingleOutputStep → ConnectionSource

56. final def wait(): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

57. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

58. final def wait(arg0: Long): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

59. final def wrap[U](body: ⇒ U): U

    Wraps exceptions into ExecutionException instances.

    Wraps exceptions into ExecutionException instances.

    Attributes

    protected

    Definition Classes

    AbstractStep