org.apache.hadoop.hbase.io.encoding
Interface DataBlockEncoder

All Known Implementing Classes:

CopyKeyDataBlockEncoder, DiffKeyDeltaEncoder, FastDiffDeltaEncoder, PrefixKeyDeltaEncoder

@InterfaceAudience.Private
public interface DataBlockEncoder

Encoding of KeyValue. It aims to be fast and efficient using assumptions:

• the KeyValues are stored sorted by key
• we know the structure of KeyValue
• the values are always iterated forward from beginning of block
• knowledge of Key Value format

It is designed to work fast enough to be feasible as in memory compression. After encoding, it also optionally compresses the encoded data if a compression algorithm is specified in HFileBlockEncodingContext argument of encodeKeyValues(ByteBuffer, boolean, HFileBlockEncodingContext).

Nested Class Summary

static interface

DataBlockEncoder.EncodedSeeker An interface which enable to seek while underlying data is encoded.

Method Summary

DataBlockEncoder.EncodedSeeker	createSeeker(org.apache.hadoop.io.RawComparator<byte[]> comparator, boolean includesMemstoreTS) Create a HFileBlock seeker which find KeyValues within a block.
ByteBuffer	decodeKeyValues(DataInputStream source, boolean includesMemstoreTS) Decode.
ByteBuffer	decodeKeyValues(DataInputStream source, int allocateHeaderLength, int skipLastBytes, boolean includesMemstoreTS) Uncompress.
void	encodeKeyValues(ByteBuffer in, boolean includesMemstoreTS, HFileBlockEncodingContext encodingContext) Encodes KeyValues.
ByteBuffer	getFirstKeyInBlock(ByteBuffer block) Return first key in block.
HFileBlockDecodingContext	newDataBlockDecodingContext(Compression.Algorithm compressionAlgorithm) Creates an encoder specific decoding context, which will prepare the data before actual decoding
HFileBlockEncodingContext	newDataBlockEncodingContext(Compression.Algorithm compressionAlgorithm, DataBlockEncoding encoding, byte[] headerBytes) Creates a encoder specific encoding context

Method Detail

encodeKeyValues

void encodeKeyValues(ByteBuffer in,
                     boolean includesMemstoreTS,
                     HFileBlockEncodingContext encodingContext)
                     throws IOException

Encodes KeyValues. It will first encode key value pairs, and then optionally do the compression for the encoded data.

Parameters:

in - Source of KeyValue for compression.

includesMemstoreTS - true if including memstore timestamp after every key-value pair

encodingContext - the encoding context which will contain encoded uncompressed bytes as well as compressed encoded bytes if compression is enabled, and also it will reuse resources across multiple calls.

Throws:

IOException - If there is an error writing to output stream.

decodeKeyValues

ByteBuffer decodeKeyValues(DataInputStream source,
                           boolean includesMemstoreTS)
                           throws IOException

Decode.

Parameters:

source - Compressed stream of KeyValues.

includesMemstoreTS - true if including memstore timestamp after every key-value pair

Returns:

Uncompressed block of KeyValues.

Throws:

IOException - If there is an error in source.

decodeKeyValues

ByteBuffer decodeKeyValues(DataInputStream source,
                           int allocateHeaderLength,
                           int skipLastBytes,
                           boolean includesMemstoreTS)
                           throws IOException

Uncompress.

Parameters:

source - encoded stream of KeyValues.

allocateHeaderLength - allocate this many bytes for the header.

skipLastBytes - Do not copy n last bytes.

includesMemstoreTS - true if including memstore timestamp after every key-value pair

Returns:

Uncompressed block of KeyValues.

Throws:

IOException - If there is an error in source.

getFirstKeyInBlock

ByteBuffer getFirstKeyInBlock(ByteBuffer block)

Return first key in block. Useful for indexing. Typically does not make a deep copy but returns a buffer wrapping a segment of the actual block's byte array. This is because the first key in block is usually stored unencoded.

Parameters:

block - encoded block we want index, the position will not change

Returns:

First key in block.

createSeeker

DataBlockEncoder.EncodedSeeker createSeeker(org.apache.hadoop.io.RawComparator<byte[]> comparator,
                                            boolean includesMemstoreTS)

Create a HFileBlock seeker which find KeyValues within a block.

Parameters:

comparator - what kind of comparison should be used

includesMemstoreTS - true if including memstore timestamp after every key-value pair

Returns:

A newly created seeker.

newDataBlockEncodingContext

HFileBlockEncodingContext newDataBlockEncodingContext(Compression.Algorithm compressionAlgorithm,
                                                      DataBlockEncoding encoding,
                                                      byte[] headerBytes)

Creates a encoder specific encoding context

Parameters:

compressionAlgorithm - compression algorithm used if the final data needs to be compressed

encoding - encoding strategy used

headerBytes - header bytes to be written, put a dummy header here if the header is unknown

Returns:

a newly created encoding context

newDataBlockDecodingContext

HFileBlockDecodingContext newDataBlockDecodingContext(Compression.Algorithm compressionAlgorithm)

Creates an encoder specific decoding context, which will prepare the data before actual decoding

Parameters:

compressionAlgorithm - compression algorithm used if the data needs to be decompressed

Returns:

a newly created decoding context