StreamFlow

Value Members

1. final def !=(arg0: AnyRef): Boolean

   Definition Classes

   AnyRef

2. final def !=(arg0: Any): Boolean

   Definition Classes

   Any

3. final def ##(): Int

   Definition Classes

   AnyRef → Any

4. def -->(targets: ConnectionTarget[DataStream, SparkStreamingRuntime]*): Unit

   Definition Classes

   MultiOutputStep

5. def -->(tgt: MultiInputStep[DataStream, SparkStreamingRuntime]): tgt.type

   Definition Classes

   MultiOutputStep

6. def -->(tgt: ConnectionTarget[DataStream, SparkStreamingRuntime]): tgt.type

   Definition Classes

   MultiOutputStep

7. final def ==(arg0: AnyRef): Boolean

   Definition Classes

   AnyRef

8. final def ==(arg0: Any): Boolean

   Definition Classes

   Any

9. def addStepListener(listener: StepListener[DataStream, SparkStreamingRuntime]): Unit

   Registers a new listener.

   Registers a new listener.

   Definition Classes

   Step

10. def addStreamDataListener(listener: StreamStepDataListener): Unit

    Registers a data listener.

    Registers a data listener.

    Definition Classes

    StreamStep

11. def addStreamFlowListener(listener: StreamFlowListener): Unit

    Registers a flow listener.

12. val allInputsRequired: Boolean

    Specifies if the step should throw an error if one of the inputs is not connected.

    Specifies if the step should throw an error if one of the inputs is not connected.

    Definition Classes

    SubModule → Step

13. final def asInstanceOf[T0]: T0

    Definition Classes

    Any

14. def clone(): AnyRef

    Attributes

    protected[java.lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

15. def compute(index: Int)(implicit runtime: SparkStreamingRuntime): DataStream

    Triggers listener notification on stream events.

    Triggers listener notification on stream events.

    index

    the output value index.

    Attributes

    protected

    Definition Classes

    StreamStep → Step

16. def compute(args: IndexedSeq[DataStream], index: Int)(implicit runtime: SparkStreamingRuntime): DataStream

    Attributes

    protected

    Definition Classes

    SubModule → Module

17. lazy val connections: Set[Connection[DataStream, SparkStreamingRuntime]]

    Collects all connections given a set of steps.

    Collects all connections given a set of steps.

    Definition Classes

    SubFlow

18. def ctx(implicit runtime: SparkStreamingRuntime): SQLContext

    Returns the implicit SQLContext.

    Returns the implicit SQLContext.

    Attributes

    protected

    Definition Classes

    StreamStep

19. final def eq(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

20. final def evaluate(implicit runtime: SparkStreamingRuntime): IndexedSeq[DataStream]

    Evaluates all step's outputs and returns a list of results.

    Evaluates all step's outputs and returns a list of results.

    Definition Classes

    Step

    Annotations

    @throws( classOf[ExecutionException] )

21. def finalize(): Unit

    Attributes

    protected[java.lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( classOf[java.lang.Throwable] )

22. final def getClass(): Class[_]

    Definition Classes

    AnyRef → Any

23. val in: LazyArray[InPort]

    Definition Classes

    MultiInputStep

24. lazy val inPoints: Seq[ConnectionTarget[DataStream, SparkStreamingRuntime]]

    Definition Classes

    SubModule → SubFlow

25. def inputCount: Int

    The maximum number of input ports.

    The maximum number of input ports.

    Definition Classes

    SubModule → Step

26. def inputs(implicit runtime: SparkStreamingRuntime): IndexedSeq[DataStream]

    Definition Classes

    MultiInputStep

27. final def isInstanceOf[T0]: Boolean

    Definition Classes

    Any

28. final def ne(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

29. final def notify(): Unit

    Definition Classes

    AnyRef

30. final def notifyAll(): Unit

    Definition Classes

    AnyRef

31. val out: LazyArray[OutPort]

    Definition Classes

    MultiOutputStep

32. lazy val outPoints: Seq[ConnectionSource[DataStream, SparkStreamingRuntime]]

    Definition Classes

    SubModule → SubFlow

33. def outbounds(index: Int): OutPort

    Definition Classes

    MultiOutputStep

34. final def output(implicit runtime: SparkStreamingRuntime): DataStream

    Shortcut for output(0).

    Shortcut for output(0). Computes a step output at index 0.

    Definition Classes

    Step

    Annotations

    @throws( classOf[ExecutionException] )

    Exceptions thrown

    ExecutionException

    in case of an error, or if the step is not connected.

35. final def output(index: Int)(implicit runtime: SparkStreamingRuntime): DataStream

    Computes a step output value at the specified index.

    Computes a step output value at the specified index.

    index

    the output value index.

    Definition Classes

    Step

    Annotations

    @throws( classOf[ExecutionException] )

    Exceptions thrown

    ExecutionException

    in case of an error, or if the step is not connected.

36. def outputCount: Int

    The number of output ports.

    The number of output ports.

    Definition Classes

    SubModule → Step

37. def outputToJson(implicit idGen: (Step[DataStream, SparkStreamingRuntime]) ⇒ String): JValue

    Outputs subflow into JSON.

    Outputs subflow into JSON.

    Definition Classes

    SubFlow

38. def outputToXml(implicit idGen: (Step[DataStream, SparkStreamingRuntime]) ⇒ String): Elem

    Outputs the subflow into XML.

    Outputs the subflow into XML.

    Attributes

    protected

    Definition Classes

    SubFlow

39. def register(implicit rt: SparkStreamingRuntime): Unit

    Registers the step with the runtime.

    Registers the step with the runtime. Subsequent restarts will keep binding the step to newly created contexts.

    Definition Classes

    StreamStep

40. def removeStepListener(listener: StepListener[DataStream, SparkStreamingRuntime]): Unit

    Unregisters a listener.

    Unregisters a listener.

    Definition Classes

    Step

41. def removeStreamDataListener(listener: StreamStepDataListener): Unit

    Unregisters a data listener.

    Unregisters a data listener.

    Definition Classes

    StreamStep

42. def removeStreamFlowListener(listener: StreamFlowListener): Unit

    Unregisters a flow listener.

43. def resetCache(predecessors: Boolean, descendants: Boolean): Unit

    Reset the cache of the subflow and its constituents.

    Reset the cache of the subflow and its constituents.

    Definition Classes

    SubFlow → Step

44. def sc(implicit runtime: SparkStreamingRuntime): SparkContext

    Returns the implicit SparkContext.

    Returns the implicit SparkContext.

    Attributes

    protected

    Definition Classes

    StreamStep

45. def ssc(implicit runtime: SparkStreamingRuntime): StreamingContext

    Returns the implicit StreamingContext.

    Returns the implicit StreamingContext.

    Attributes

    protected

    Definition Classes

    StreamStep

46. lazy val steps: Set[Step[DataStream, SparkStreamingRuntime]]

    Collects all steps starting with the targets and going back through the predecessors.

    Collects all steps starting with the targets and going back through the predecessors.

    Definition Classes

    SubFlow

47. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes

    AnyRef

48. val tag: String

    Tag used for serialization.

    Tag used for serialization.

    Definition Classes

    StreamFlow → SubFlow

49. val targets: Iterable[ConnectionSource[DataStream, SparkStreamingRuntime]]

50. def to(targets: ConnectionTarget[DataStream, SparkStreamingRuntime]*): Unit

    Definition Classes

    MultiOutputStep

51. def to(tgt: MultiInputStep[DataStream, SparkStreamingRuntime]): tgt.type

    Definition Classes

    MultiOutputStep

52. def to(tgt: ConnectionTarget[DataStream, SparkStreamingRuntime]): tgt.type

    Definition Classes

    MultiOutputStep

53. def toJson: JValue

    The default implementation uses default id generator.

    The default implementation uses default id generator.

    Definition Classes

    SubFlow → JsonExport

54. def toXml: Elem

    The default implementation uses default id generator.

    The default implementation uses default id generator.

    Definition Classes

    SubFlow → XmlExport

55. def unregister(implicit rt: SparkStreamingRuntime): Unit

    Unregisters the step with the runtime.

    Unregisters the step with the runtime. After the restart, the step will no longer be bound to the active context.

    Definition Classes

    StreamStep

56. final def wait(): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

57. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

58. final def wait(arg0: Long): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

59. final def wrap[U](body: ⇒ U): U

    Wraps exceptions into ExecutionException instances.

    Wraps exceptions into ExecutionException instances.

    Attributes

    protected

    Definition Classes

    AbstractStep