Class

org.apache.spark.sql.execution.columnar.encoding

DictionaryDecoderBase

Related Doc: package encoding

Permalink

abstract class DictionaryDecoderBase extends ColumnDecoder with DictionaryEncoding

Linear Supertypes
Known Subclasses
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. DictionaryDecoderBase
  2. DictionaryEncoding
  3. ColumnDecoder
  4. ColumnEncoding
  5. AnyRef
  6. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new DictionaryDecoderBase()

    Permalink

Abstract Value Members

  1. abstract def hasNulls: Boolean

    Permalink
    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    ColumnDecoder
  2. abstract def initializeNulls(columnBytes: AnyRef, cursor: Long, field: StructField): Long

    Permalink
    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    ColumnDecoder
  3. abstract def isNull(columnBytes: AnyRef, ordinal: Int): Boolean

    Permalink

    Sequential null check for a value which should be invoked for all values of ordinal from 0 until maximum in order.

    Sequential null check for a value which should be invoked for all values of ordinal from 0 until maximum in order.

    Definition Classes
    ColumnDecoder

Concrete Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. def absoluteArray(columnBytes: AnyRef, position: Int): Long

    Permalink
    Definition Classes
    ColumnDecoder
  5. def absoluteBinary(columnBytes: AnyRef, position: Int): Long

    Permalink
    Definition Classes
    ColumnDecoder
  6. def absoluteBoolean(columnBytes: AnyRef, position: Int): Long

    Permalink

    Random access to the encoded data.

    Random access to the encoded data.

    Definition Classes
    ColumnDecoder
  7. def absoluteByte(columnBytes: AnyRef, position: Int): Long

    Permalink
    Definition Classes
    ColumnDecoder
  8. def absoluteDate(columnBytes: AnyRef, position: Int): Long

    Permalink
    Definition Classes
    ColumnDecoder
  9. def absoluteDecimal(columnBytes: AnyRef, position: Int): Long

    Permalink
    Definition Classes
    ColumnDecoder
  10. def absoluteDouble(columnBytes: AnyRef, position: Int): Long

    Permalink
    Definition Classes
    ColumnDecoder
  11. def absoluteFloat(columnBytes: AnyRef, position: Int): Long

    Permalink
    Definition Classes
    ColumnDecoder
  12. def absoluteInt(columnBytes: AnyRef, position: Int): Long

    Permalink
    Definition Classes
    DictionaryDecoderBaseColumnDecoder
  13. def absoluteInterval(columnBytes: AnyRef, position: Int): Long

    Permalink
    Definition Classes
    ColumnDecoder
  14. def absoluteLong(columnBytes: AnyRef, position: Int): Long

    Permalink
    Definition Classes
    DictionaryDecoderBaseColumnDecoder
  15. def absoluteLongDecimal(columnBytes: AnyRef, position: Int): Long

    Permalink
    Definition Classes
    ColumnDecoder
  16. def absoluteMap(columnBytes: AnyRef, position: Int): Long

    Permalink
    Definition Classes
    ColumnDecoder
  17. def absoluteShort(columnBytes: AnyRef, position: Int): Long

    Permalink
    Definition Classes
    ColumnDecoder
  18. def absoluteStruct(columnBytes: AnyRef, position: Int): Long

    Permalink
    Definition Classes
    ColumnDecoder
  19. def absoluteTimestamp(columnBytes: AnyRef, position: Int): Long

    Permalink
    Definition Classes
    ColumnDecoder
  20. def absoluteUTF8String(columnBytes: AnyRef, position: Int): Long

    Permalink
    Definition Classes
    DictionaryDecoderBaseColumnDecoder
  21. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  22. final var baseCursor: Long

    Permalink
    Attributes
    protected
    Definition Classes
    ColumnDecoder
  23. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  24. final var currentCursor: Long

    Permalink

    Normally not used by decoder but supplied by caller to the methods but can be used if required but needs to be set by caller explicitly.

    Normally not used by decoder but supplied by caller to the methods but can be used if required but needs to be set by caller explicitly.

    Definition Classes
    ColumnDecoder
  25. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  26. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  27. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  28. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  29. final def getStringDictionary: StringDictionary

    Permalink
    Definition Classes
    DictionaryDecoderBaseColumnDecoder
  30. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  31. final def initialize(columnBytes: AnyRef, cursor: Long, field: StructField): Long

    Permalink

    Delta encoder/decoder depend on initialize being final and invoking initializeCursor and initializeNulls as below.

    Delta encoder/decoder depend on initialize being final and invoking initializeCursor and initializeNulls as below.

    Definition Classes
    ColumnDecoder
  32. final def initialize(buffer: ByteBuffer, field: StructField): Long

    Permalink
    Definition Classes
    ColumnDecoder
  33. def initializeCursor(columnBytes: AnyRef, cursor: Long, field: StructField): Long

    Permalink

    Initialization will fill in the dictionaries as written by the DictionaryEncoder.

    Initialization will fill in the dictionaries as written by the DictionaryEncoder. For string maps it reads in the value array written using ByteBufferHashMap by the encoder expecting the size of UTF8 encoded string followed by the string contents. Long and integer dictionaries are still using the old ObjectHashSet which needs to be moved to ByteBufferHashMap once DictionaryEncoder adds support for long/integer dictionary encoding.

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    DictionaryDecoderBaseColumnDecoder
  34. final var intDictionary: Array[Int]

    Permalink
    Attributes
    protected[this]
  35. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  36. def isNullAt(columnBytes: AnyRef, position: Int): Boolean

    Permalink

    Absolute ordinal null check for random access.

    Absolute ordinal null check for random access.

    Definition Classes
    ColumnDecoder
  37. final var longDictionary: Array[Long]

    Permalink
    Attributes
    protected[this]
  38. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  39. def nextArray(columnBytes: AnyRef, cursor: Long): Long

    Permalink
    Definition Classes
    ColumnDecoder
  40. def nextBinary(columnBytes: AnyRef, cursor: Long): Long

    Permalink
    Definition Classes
    ColumnDecoder
  41. def nextBoolean(columnBytes: AnyRef, cursor: Long): Long

    Permalink
    Definition Classes
    ColumnDecoder
  42. def nextByte(columnBytes: AnyRef, cursor: Long): Long

    Permalink
    Definition Classes
    ColumnDecoder
  43. def nextDecimal(columnBytes: AnyRef, cursor: Long): Long

    Permalink
    Definition Classes
    ColumnDecoder
  44. def nextDouble(columnBytes: AnyRef, cursor: Long): Long

    Permalink
    Definition Classes
    ColumnDecoder
  45. def nextFloat(columnBytes: AnyRef, cursor: Long): Long

    Permalink
    Definition Classes
    ColumnDecoder
  46. def nextInt(columnBytes: AnyRef, cursor: Long): Long

    Permalink
    Definition Classes
    DictionaryDecoderBaseColumnDecoder
  47. def nextInterval(columnBytes: AnyRef, cursor: Long): Long

    Permalink
    Definition Classes
    ColumnDecoder
  48. def nextLong(columnBytes: AnyRef, cursor: Long): Long

    Permalink
    Definition Classes
    DictionaryDecoderBaseColumnDecoder
  49. def nextLongDecimal(columnBytes: AnyRef, cursor: Long): Long

    Permalink
    Definition Classes
    ColumnDecoder
  50. def nextMap(columnBytes: AnyRef, cursor: Long): Long

    Permalink
    Definition Classes
    ColumnDecoder
  51. def nextShort(columnBytes: AnyRef, cursor: Long): Long

    Permalink
    Definition Classes
    ColumnDecoder
  52. def nextStruct(columnBytes: AnyRef, cursor: Long): Long

    Permalink
    Definition Classes
    ColumnDecoder
  53. def nextUTF8String(columnBytes: AnyRef, cursor: Long): Long

    Permalink
    Definition Classes
    DictionaryDecoderBaseColumnDecoder
  54. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  55. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  56. def numNullsUntilPosition(columnBytes: AnyRef, position: Int): Int

    Permalink

    Get the number of null values till given 0-based position (exclusive) for random access.

    Get the number of null values till given 0-based position (exclusive) for random access.

    Attributes
    protected
    Definition Classes
    ColumnDecoder
  57. def readArray(columnBytes: AnyRef, cursor: Long): ArrayData

    Permalink
    Definition Classes
    ColumnDecoder
  58. def readBinary(columnBytes: AnyRef, cursor: Long): Array[Byte]

    Permalink
    Definition Classes
    ColumnDecoder
  59. def readBoolean(columnBytes: AnyRef, cursor: Long): Boolean

    Permalink
    Definition Classes
    ColumnDecoder
  60. def readByte(columnBytes: AnyRef, cursor: Long): Byte

    Permalink
    Definition Classes
    ColumnDecoder
  61. def readDate(columnBytes: AnyRef, cursor: Long): Int

    Permalink
    Definition Classes
    ColumnDecoder
  62. def readDecimal(columnBytes: AnyRef, precision: Int, scale: Int, cursor: Long): Decimal

    Permalink
    Definition Classes
    ColumnDecoder
  63. def readDictionaryIndex(columnBytes: AnyRef, cursor: Long): Int

    Permalink
    Definition Classes
    DictionaryDecoderBaseColumnDecoder
  64. def readDouble(columnBytes: AnyRef, cursor: Long): Double

    Permalink
    Definition Classes
    ColumnDecoder
  65. def readFloat(columnBytes: AnyRef, cursor: Long): Float

    Permalink
    Definition Classes
    ColumnDecoder
  66. def readInt(columnBytes: AnyRef, cursor: Long): Int

    Permalink
    Definition Classes
    DictionaryDecoderBaseColumnDecoder
  67. def readInterval(columnBytes: AnyRef, cursor: Long): CalendarInterval

    Permalink
    Definition Classes
    ColumnDecoder
  68. def readLong(columnBytes: AnyRef, cursor: Long): Long

    Permalink
    Definition Classes
    DictionaryDecoderBaseColumnDecoder
  69. def readLongDecimal(columnBytes: AnyRef, precision: Int, scale: Int, cursor: Long): Decimal

    Permalink
    Definition Classes
    ColumnDecoder
  70. def readMap(columnBytes: AnyRef, cursor: Long): MapData

    Permalink
    Definition Classes
    ColumnDecoder
  71. def readShort(columnBytes: AnyRef, cursor: Long): Short

    Permalink
    Definition Classes
    ColumnDecoder
  72. def readStruct(columnBytes: AnyRef, numFields: Int, cursor: Long): InternalRow

    Permalink
    Definition Classes
    ColumnDecoder
  73. def readTimestamp(columnBytes: AnyRef, cursor: Long): Long

    Permalink
    Definition Classes
    ColumnDecoder
  74. def readUTF8String(columnBytes: AnyRef, cursor: Long): UTF8String

    Permalink
    Definition Classes
    DictionaryDecoderBaseColumnDecoder
  75. final var stringDictionary: StringDictionary

    Permalink
    Attributes
    protected[this]
  76. final def supports(dataType: DataType): Boolean

    Permalink
    Definition Classes
    DictionaryEncodingColumnEncoding
  77. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  78. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  79. def typeId: Int

    Permalink
    Definition Classes
    DictionaryDecoderBaseColumnEncoding
  80. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  81. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  82. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from DictionaryEncoding

Inherited from ColumnDecoder

Inherited from ColumnEncoding

Inherited from AnyRef

Inherited from Any

Ungrouped