org.apache.spark.sql.execution.columnar.impl
ColumnFormatValue
Related Doc:
package impl
Callers of this method should have a corresponding release method for eager release to work else off-heap object may keep around occupying system RAM until the next GC cycle.
Callers of this method should have a corresponding release method for eager release to work else off-heap object may keep around occupying system RAM until the next GC cycle. Callers may decide whether to keep the release method in a finally block to ensure its invocation, or do it only in normal paths because JVM reference collector will eventually clean it in any case.
Calls to this specific class are guaranteed to always return buffers which have position as zero so callers can make simplifying assumptions about the same.
Value object in the column store simply encapsulates binary data as a ByteBuffer. This can be either a direct buffer or a heap buffer depending on the system off-heap configuration. The reason for a separate type is to easily store data off-heap without any major changes to engine otherwise as well as efficiently serialize/deserialize them directly to Oplog/socket.
This class extends SerializedDiskBuffer to avoid a copy when reading/writing from Oplog. Consequently it writes the serialization header itself (typeID + classID + size) into stream as would be written by DataSerializer.writeObject. This helps it avoid additional byte writes when transferring data to the channels.