RateStreamSource

Value Members

1. final def !=(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

2. final def ##(): Int

   

   Definition Classes

   AnyRef → Any

3. final def ==(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

4. final def asInstanceOf[T0]: T0

   

   Definition Classes

   Any

5. val clock: Clock

   

6. def clone(): AnyRef

   

   Attributes

   protected[java.lang]

   Definition Classes

   AnyRef

   Annotations

   @throws( ... )

7. def commit(end: Offset): Unit

   

   Informs the source that Spark has completed processing all data for offsets less than or equal to end and will only request offsets greater than end in the future.

   Informs the source that Spark has completed processing all data for offsets less than or equal to end and will only request offsets greater than end in the future.

   Definition Classes

   Source

8. final def eq(arg0: AnyRef): Boolean

   

   Definition Classes

   AnyRef

9. def equals(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

10. def finalize(): Unit

    

    Attributes

    protected[java.lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( classOf[java.lang.Throwable] )

11. def getBatch(start: Option[Offset], end: Offset): DataFrame

    

    Returns the data that is between the offsets (start, end].

    Returns the data that is between the offsets (start, end]. When start is None, then the batch should begin with the first record. This method must always return the same data for a particular start and end pair; even after the Source has been restarted on a different node.

    Higher layers will always call this method with a value of start greater than or equal to the last value passed to commit and a value of end less than or equal to the last value returned by getOffset

    It is possible for the Offset type to be a SerializedOffset when it was obtained from the log. Moreover, StreamExecution only compares the Offset JSON representation to determine if the two objects are equal. This could have ramifications when upgrading Offset JSON formats i.e., two equivalent Offset objects could differ between version. Consequently, StreamExecution may call this method with two such equivalent Offset objects. In which case, the Source should return an empty DataFrame

    Definition Classes

    RateStreamSource → Source

12. final def getClass(): Class[_]

    

    Definition Classes

    AnyRef → Any

13. def getOffset: Option[Offset]

    

    Returns the maximum available offset for this source.

    Returns the maximum available offset for this source. Returns None if this source has never received any data.

    Definition Classes

    RateStreamSource → Source

14. def hashCode(): Int

    

    Definition Classes

    AnyRef → Any

15. def initializeLogIfNecessary(isInterpreter: Boolean): Unit

    

    Attributes

    protected

    Definition Classes

    Logging

16. final def isInstanceOf[T0]: Boolean

    

    Definition Classes

    Any

17. def isTraceEnabled(): Boolean

    

    Attributes

    protected

    Definition Classes

    Logging

18. def log: Logger

    

    Attributes

    protected

    Definition Classes

    Logging

19. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    

    Attributes

    protected

    Definition Classes

    Logging

20. def logDebug(msg: ⇒ String): Unit

    

    Attributes

    protected

    Definition Classes

    Logging

21. def logError(msg: ⇒ String, throwable: Throwable): Unit

    

    Attributes

    protected

    Definition Classes

    Logging

22. def logError(msg: ⇒ String): Unit

    

    Attributes

    protected

    Definition Classes

    Logging

23. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    

    Attributes

    protected

    Definition Classes

    Logging

24. def logInfo(msg: ⇒ String): Unit

    

    Attributes

    protected

    Definition Classes

    Logging

25. def logName: String

    

    Attributes

    protected

    Definition Classes

    Logging

26. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    

    Attributes

    protected

    Definition Classes

    Logging

27. def logTrace(msg: ⇒ String): Unit

    

    Attributes

    protected

    Definition Classes

    Logging

28. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    

    Attributes

    protected

    Definition Classes

    Logging

29. def logWarning(msg: ⇒ String): Unit

    

    Attributes

    protected

    Definition Classes

    Logging

30. final def ne(arg0: AnyRef): Boolean

    

    Definition Classes

    AnyRef

31. final def notify(): Unit

    

    Definition Classes

    AnyRef

32. final def notifyAll(): Unit

    

    Definition Classes

    AnyRef

33. def schema: StructType

    

    Returns the schema of the data from this source

    Returns the schema of the data from this source

    Definition Classes

    RateStreamSource → Source

34. def stop(): Unit

    

    Stop this source and free any resources it has allocated.

    Stop this source and free any resources it has allocated.

    Definition Classes

    RateStreamSource → Source

35. final def synchronized[T0](arg0: ⇒ T0): T0

    

    Definition Classes

    AnyRef

36. def toString(): String

    

    Definition Classes

    RateStreamSource → AnyRef → Any

37. final def wait(): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

38. final def wait(arg0: Long, arg1: Int): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

39. final def wait(arg0: Long): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )