class ValueStateImplWithTTL[S] extends SingleKeyTTLStateImpl with ValueState[S]
Class that provides a concrete implementation for a single value state associated with state variables (with ttl expiration support) used in the streaming transformWithState operator.
- S
- data type of object that will be stored
- Alphabetic
- By Inheritance
- ValueStateImplWithTTL
- ValueState
- Serializable
- SingleKeyTTLStateImpl
- TTLState
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Instance Constructors
- new ValueStateImplWithTTL(store: StateStore, stateName: String, keyExprEnc: ExpressionEncoder[Any], valEncoder: Encoder[S], ttlConfig: TTLConfig, batchTimestampMs: Long)
- store
- reference to the StateStore instance to be used for storing state
- stateName
- name of logical state partition
- keyExprEnc
- Spark SQL encoder for key
- valEncoder
- Spark SQL encoder for value
- ttlConfig
- TTL configuration for values stored in this state
- batchTimestampMs
- current batch processing timestamp.
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clear(): Unit
Function to remove state for given key
Function to remove state for given key
- Definition Classes
- ValueStateImplWithTTL → ValueState
- def clearExpiredState(): Long
Clears any state which has ttl older than ttlExpirationMs.
Clears any state which has ttl older than ttlExpirationMs.
- returns
number of values cleaned up.
- Definition Classes
- SingleKeyTTLStateImpl → TTLState
- def clearIfExpired(groupingKey: Array[Byte]): Long
Clears the user state associated with this grouping key if it has expired.
Clears the user state associated with this grouping key if it has expired. This function is called by Spark to perform cleanup at the end of transformWithState processing.
Spark uses a secondary index to determine if the user state for this grouping key has expired. However, its possible that the user has updated the TTL and secondary index is out of date. Implementations must validate that the user State has actually expired before cleanup based on their own State data.
- groupingKey
grouping key for which cleanup should be performed.
- returns
true if the state was cleared, false otherwise.
- Definition Classes
- ValueStateImplWithTTL → SingleKeyTTLStateImpl
- def clearTTLState(): Unit
This function will be called when clear() on State Variables with ttl enabled is called.
This function will be called when clear() on State Variables with ttl enabled is called. This function should clear any associated ttlState, since we are clearing the user state.
- Definition Classes
- SingleKeyTTLStateImpl
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def exists(): Boolean
Function to check if state exists.
Function to check if state exists. Returns true if present and false otherwise
- Definition Classes
- ValueStateImplWithTTL → ValueState
- def get(): S
Function to return associated value with key if exists and null otherwise
Function to return associated value with key if exists and null otherwise
- Definition Classes
- ValueStateImplWithTTL → ValueState
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def getOption(): Option[S]
Function to return Option of value if exists and None otherwise
Function to return Option of value if exists and None otherwise
- Definition Classes
- ValueStateImplWithTTL → ValueState
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- def update(newState: S): Unit
Function to update and overwrite state associated with given key
Function to update and overwrite state associated with given key
- Definition Classes
- ValueStateImplWithTTL → ValueState
- def upsertTTLForStateKey(expirationMs: Long, groupingKey: Array[Byte]): Unit
- Definition Classes
- SingleKeyTTLStateImpl
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)