Packages

c

org.apache.spark.sql.execution.streaming.state

StreamingAggregationStateManagerImplV2

class StreamingAggregationStateManagerImplV2 extends StreamingAggregationStateManagerBaseImpl

The implementation of StreamingAggregationStateManager for state version 2. In state version 2, the schema of key and value in state are follow:

- key: Same as key expressions. - value: The diff between input row attributes and key expressions.

The schema of value is changed to optimize the memory/space usage in state, via removing duplicated columns in key-value pair. Hence key columns are excluded from the schema of value.

Linear Supertypes
StreamingAggregationStateManagerBaseImpl, StreamingAggregationStateManager, Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. StreamingAggregationStateManagerImplV2
  2. StreamingAggregationStateManagerBaseImpl
  3. StreamingAggregationStateManager
  4. Serializable
  5. Serializable
  6. AnyRef
  7. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new StreamingAggregationStateManagerImplV2(keyExpressions: Seq[Attribute], inputRowAttributes: Seq[Attribute])

    keyExpressions

    The attributes of keys.

    inputRowAttributes

    The attributes of input row.

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  6. def commit(store: StateStore): Long

    Commit all the updates that have been made to the target state store, and return the new version.

    Commit all the updates that have been made to the target state store, and return the new version.

    Definition Classes
    StreamingAggregationStateManagerBaseImpl → StreamingAggregationStateManager
  7. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  8. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  9. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  10. def get(store: ReadStateStore, key: UnsafeRow): UnsafeRow

    Get the current value of a non-null key from the target state store.

    Get the current value of a non-null key from the target state store.

    Definition Classes
    StreamingAggregationStateManagerImplV2 → StreamingAggregationStateManager
  11. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  12. def getKey(row: UnsafeRow): UnsafeRow

    Extract columns consisting key from input row, and return the new row for key columns.

    Extract columns consisting key from input row, and return the new row for key columns.

    Definition Classes
    StreamingAggregationStateManagerBaseImpl → StreamingAggregationStateManager
  13. def getStateValueSchema: StructType

    Calculate schema for the value of state.

    Calculate schema for the value of state. The schema is mainly passed to the StateStoreRDD.

    Definition Classes
    StreamingAggregationStateManagerImplV2 → StreamingAggregationStateManager
  14. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  15. val inputRowAttributes: Seq[Attribute]
    Attributes
    protected
    Definition Classes
    StreamingAggregationStateManagerBaseImpl
  16. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  17. def iterator(store: ReadStateStore): Iterator[UnsafeRowPair]

    Return an iterator containing all the key-value pairs in target state store.

    Return an iterator containing all the key-value pairs in target state store.

    Definition Classes
    StreamingAggregationStateManagerImplV2 → StreamingAggregationStateManager
  18. val keyExpressions: Seq[Attribute]
    Attributes
    protected
    Definition Classes
    StreamingAggregationStateManagerBaseImpl
  19. lazy val keyProjector: UnsafeProjection
    Attributes
    protected
    Definition Classes
    StreamingAggregationStateManagerBaseImpl
    Annotations
    @transient()
  20. def keys(store: ReadStateStore): Iterator[UnsafeRow]

    Return an iterator containing all the keys in target state store.

    Return an iterator containing all the keys in target state store.

    Definition Classes
    StreamingAggregationStateManagerBaseImpl → StreamingAggregationStateManager
  21. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  22. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  23. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  24. def put(store: StateStore, row: UnsafeRow): Unit

    Put a new value for a non-null key to the target state store.

    Put a new value for a non-null key to the target state store. Note that key will be extracted from the input row, and the key would be same as the result of getKey(inputRow).

    Definition Classes
    StreamingAggregationStateManagerImplV2 → StreamingAggregationStateManager
  25. def remove(store: StateStore, key: UnsafeRow): Unit

    Remove a single non-null key from the target state store.

    Remove a single non-null key from the target state store.

    Definition Classes
    StreamingAggregationStateManagerBaseImpl → StreamingAggregationStateManager
  26. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  27. def toString(): String
    Definition Classes
    AnyRef → Any
  28. def values(store: ReadStateStore): Iterator[UnsafeRow]

    Return an iterator containing all the values in target state store.

    Return an iterator containing all the values in target state store.

    Definition Classes
    StreamingAggregationStateManagerImplV2 → StreamingAggregationStateManager
  29. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  30. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  31. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from StreamingAggregationStateManagerBaseImpl

Inherited from StreamingAggregationStateManager

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped