org.apache.hadoop.hbase.codec.prefixtree.encode
Class PrefixTreeEncoder

java.lang.Object
  org.apache.hadoop.hbase.codec.prefixtree.encode.PrefixTreeEncoder

All Implemented Interfaces:

CellOutputStream

@InterfaceAudience.Private
public class PrefixTreeEncoder
extends Object
implements CellOutputStream

This is the primary class for converting a CellOutputStream into an encoded byte[]. As Cells are added they are completely copied into the various encoding structures. This is important because usually the cells being fed in during compactions will be transient.

Usage:
1) constructor
4) append cells in sorted order: write(Cell cell)
5) flush()

Field Summary

protected PrefixTreeBlockMeta	blockMeta
protected CellTypeEncoder	cellTypeEncoder
protected ByteRangeSet	familyDeduplicator
protected ByteRange	familyRange
protected Tokenizer	familyTokenizer
protected ColumnSectionWriter	familyWriter
protected boolean	includeMvccVersion
protected static org.apache.commons.logging.Log	LOG static
protected int	maxTagLength
protected int	maxValueLength
static boolean	MULITPLE_FAMILIES_POSSIBLE
protected LongEncoder	mvccVersionEncoder
protected long[]	mvccVersions
protected long	numResets fields
protected OutputStream	outputStream
protected ByteRangeSet	qualifierDeduplicator
protected ByteRange	qualifierRange
protected Tokenizer	qualifierTokenizer
protected ColumnSectionWriter	qualifierWriter
protected ByteRange	rowRange
protected Tokenizer	rowTokenizer
protected RowSectionWriter	rowWriter
protected byte[]	tags
protected ByteRangeSet	tagsDeduplicator
protected int[]	tagsOffsets
protected ByteRange	tagsRange
protected Tokenizer	tagsTokenizer
protected ColumnSectionWriter	tagsWriter
protected LongEncoder	timestampEncoder
protected long[]	timestamps
protected int	totalBytes
protected int	totalCells
protected int	totalTagBytes
protected int	totalUnencodedBytes
protected int	totalValueBytes
protected byte[]	typeBytes
protected int[]	valueOffsets
protected byte[]	values

Constructor Summary

PrefixTreeEncoder(OutputStream outputStream, boolean includeMvccVersion)
construct

Method Summary

protected void	compile() Now that all the cells have been added, do the work to reduce them to a series of byte[] fragments that are ready to be written to the output stream.
protected void	compileFamilies()
protected void	compileMvccVersions()
protected void	compileQualifiers()
protected void	compileRows()
protected void	compileTags()
protected void	compileTimestamps()
protected void	compileTypes() The following "compile" methods do any intermediate work necessary to transform the cell fragments collected during the writing phase into structures that are ready to write to the outputStream.
protected void	ensurePerCellCapacities() Check that the arrays used to hold cell fragments are large enough for the cell that is being added.
void	flush() Expensive method.
PrefixTreeBlockMeta	getBlockMeta() get/set
ByteRangeSet	getFamilySorter()
ColumnSectionWriter	getFamilyWriter()
LongEncoder	getMvccVersionEncoder()
long[]	getMvccVersions()
ByteRangeSet	getQualifierSorter()
ColumnSectionWriter	getQualifierWriter()
Tokenizer	getRowTokenizer()
RowSectionWriter	getRowWriter()
ByteRangeSet	getTagSorter()
ColumnSectionWriter	getTagWriter()
LongEncoder	getTimestampEncoder()
long[]	getTimestamps()
int	getTotalBytes()
byte[]	getTypeBytes()
ByteRange	getValueByteRange()
int	getValueLength(int index)
long	getValueOffset(int index) convenience getters
protected void	initializeTagHelpers()
void	reset(OutputStream outputStream, boolean includeMvccVersion)
void	write(Cell cell)
void	writeWithRepeatRow(Cell cell) Add a Cell to the output stream but repeat the previous row.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

protected static final org.apache.commons.logging.Log LOG

static

MULITPLE_FAMILIES_POSSIBLE

public static final boolean MULITPLE_FAMILIES_POSSIBLE

See Also:

Constant Field Values

numResets

protected long numResets

fields

outputStream

protected OutputStream outputStream

includeMvccVersion

protected boolean includeMvccVersion

rowRange

protected ByteRange rowRange

familyRange

protected ByteRange familyRange

qualifierRange

protected ByteRange qualifierRange

tagsRange

protected ByteRange tagsRange

timestamps

protected long[] timestamps

mvccVersions

protected long[] mvccVersions

typeBytes

protected byte[] typeBytes

valueOffsets

protected int[] valueOffsets

tagsOffsets

protected int[] tagsOffsets

values

protected byte[] values

tags

protected byte[] tags

blockMeta

protected PrefixTreeBlockMeta blockMeta

timestampEncoder

protected LongEncoder timestampEncoder

mvccVersionEncoder

protected LongEncoder mvccVersionEncoder

cellTypeEncoder

protected CellTypeEncoder cellTypeEncoder

familyDeduplicator

protected ByteRangeSet familyDeduplicator

qualifierDeduplicator

protected ByteRangeSet qualifierDeduplicator

tagsDeduplicator

protected ByteRangeSet tagsDeduplicator

rowTokenizer

protected Tokenizer rowTokenizer

familyTokenizer

protected Tokenizer familyTokenizer

qualifierTokenizer

protected Tokenizer qualifierTokenizer

tagsTokenizer

protected Tokenizer tagsTokenizer

rowWriter

protected RowSectionWriter rowWriter

familyWriter

protected ColumnSectionWriter familyWriter

qualifierWriter

protected ColumnSectionWriter qualifierWriter

tagsWriter

protected ColumnSectionWriter tagsWriter

totalCells

protected int totalCells

totalUnencodedBytes

protected int totalUnencodedBytes

totalValueBytes

protected int totalValueBytes

totalTagBytes

protected int totalTagBytes

maxValueLength

protected int maxValueLength

maxTagLength

protected int maxTagLength

totalBytes

protected int totalBytes

Constructor Detail

PrefixTreeEncoder

public PrefixTreeEncoder(OutputStream outputStream,
                         boolean includeMvccVersion)

construct

Method Detail

reset

public void reset(OutputStream outputStream,
                  boolean includeMvccVersion)

initializeTagHelpers

protected void initializeTagHelpers()

ensurePerCellCapacities

protected void ensurePerCellCapacities()

Check that the arrays used to hold cell fragments are large enough for the cell that is being added. Since the PrefixTreeEncoder is cached between uses, these arrays may grow during the first few block encodings but should stabilize quickly.

writeWithRepeatRow

public void writeWithRepeatRow(Cell cell)

Add a Cell to the output stream but repeat the previous row.

write

public void write(Cell cell)

Specified by:

write in interface CellOutputStream

flush

public void flush()
           throws IOException

Expensive method. The second half of the encoding work happens here. Take all the separate accumulated data structures and turn them into a single stream of bytes which is written to the outputStream.

Specified by:

flush in interface CellOutputStream

Throws:

IOException

compile

protected void compile()

Now that all the cells have been added, do the work to reduce them to a series of byte[] fragments that are ready to be written to the output stream.

compileTypes

protected void compileTypes()

The following "compile" methods do any intermediate work necessary to transform the cell fragments collected during the writing phase into structures that are ready to write to the outputStream.

The family and qualifier treatment is almost identical, as is timestamp and mvccVersion.

compileMvccVersions

protected void compileMvccVersions()

compileTimestamps

protected void compileTimestamps()

compileQualifiers

protected void compileQualifiers()

compileFamilies

protected void compileFamilies()

compileTags

protected void compileTags()

compileRows

protected void compileRows()

getValueOffset

public long getValueOffset(int index)

convenience getters

getValueLength

public int getValueLength(int index)

getBlockMeta

public PrefixTreeBlockMeta getBlockMeta()

get/set

getRowTokenizer

public Tokenizer getRowTokenizer()

getTimestampEncoder

public LongEncoder getTimestampEncoder()

getTotalBytes

public int getTotalBytes()

getTimestamps

public long[] getTimestamps()

getMvccVersions

public long[] getMvccVersions()

getTypeBytes

public byte[] getTypeBytes()

getMvccVersionEncoder

public LongEncoder getMvccVersionEncoder()

getFamilySorter

public ByteRangeSet getFamilySorter()

getQualifierSorter

public ByteRangeSet getQualifierSorter()

getTagSorter

public ByteRangeSet getTagSorter()

getFamilyWriter

public ColumnSectionWriter getFamilyWriter()

getQualifierWriter

public ColumnSectionWriter getQualifierWriter()

getTagWriter

public ColumnSectionWriter getTagWriter()

getRowWriter

public RowSectionWriter getRowWriter()

getValueByteRange

public ByteRange getValueByteRange()