Package convex.core.data

Class BlobTree

java.lang.Object

convex.core.data.AObject

convex.core.data.ACell

convex.core.data.ACountable<CVMLong>

convex.core.data.ABlobLike<CVMLong>

convex.core.data.ABlob

convex.core.data.BlobTree

All Implemented Interfaces:

IValidated, IWriteable, Comparable<ABlobLike<?>>

public class BlobTree
extends ABlob

Implementation of a large Blob data structure consisting of 2 or more chunks. Intention is to enable relatively large binary content to be handled without too many tree levels, and without too many references in a single tree node We choose a branching factor of 16 as a reasonable tradeoff. Level 1 can hold up to 64k Level 2 can hold up to 1mb Level 3 can hold up to 16mb Level 4 can hold up to 256mb ... Level 15 (max) should be big enough for the moment One smart reference is maintained for each child node at each level

Field Summary

Fields

Modifier and Type	Field	Description
static final int	BIT_SHIFT_PER_LEVEL
static final int	FANOUT
static final int	MAX_ENCODING_SIZE	Maximum byte length of an encoded BlobTree node.

Fields inherited from class convex.core.data.ABlob

count

Fields inherited from class convex.core.data.ACell

cachedRef, memorySize

Fields inherited from class convex.core.data.AObject

encoding

Method Summary

Modifier and Type	Method	Description
ABlob	append(ABlob d)	Appends another blob to this BlobTree.
boolean	appendHex(BlobBuilder bb, long length)	Append hex string up to the given length in hex digits (a multiple of two)
byte	byteAtUnchecked(long i)	Gets the byte at the specified position in this data object, possibly skipping bounds checking.
static long	calcChunks(long length)	Computes the number of chunks (4096 bytes or less) for the canonical BlobTree of the given length
static int	childCount(long length)	Gets the number of children for a BlobTree of given total length.
static long	childSize(long length)	Gets the size of a BlobTree child for a blob of given total length.
long	commonHexPrefixLength(ABlob b)
static BlobTree	create(ABlob blob)	Create a BlobTree from an arbitrary Blob.
static BlobTree	createWithChildren(ABlob[] children)	Create a BlobTree with the given children.
int	encodeRaw(byte[] bs, int pos)	Writes this Blob's encoding to a byte array, excluding the tag byte
boolean	equals(ABlob a)	Determines if this Blob is equal to another Blob.
boolean	equals(BlobTree b)
boolean	equalsBytes(byte[] bytes, long byteOffset)	Tests if the byte contents of this instance are equal to a subset of a byte array
boolean	equalsBytes(ABlob b)	Tests if this Blob has exactly the same bytes as another Blob
int	estimatedEncodingSize()	Estimate the encoded data size for this Cell.
ByteBuffer	getByteBuffer()	Gets a byte buffer containing this Blob's raw data.
int	getBytes(byte[] dest, int pos)	Copies the bytes from this instance to a given destination array
Blob	getChunk(long chunkIndex)	Gets a chunk of this Blob, as a canonical flat Blob up to the maximum Blob chunk size.
<R extends ACell> Ref<R>	getRef(int i)	Gets a numbered child Ref from within this Cell.
int	getRefCount()	Gets the number of Refs contained within this Cell.
long	hexMatch(ABlobLike<?> b)	Computes the length of the longest common hex prefix between two blobs
long	hexMatch(ABlobLike<?> b, long start, long length)	Returns the number of matching hex digits in the given hex range of another Blob.
boolean	isCanonical()	Returns true if this Cell is in a canonical representation.
boolean	isFullyPacked()	Returns true if this is a fully packed set of chunks
long	longValue()	Converts this BlobLike to the corresponding long value.
int	read(long offset, long count, ByteBuffer dest)	Gets bytes from this Blob into a ByteBuffer
static BlobTree	read(long count, Blob src, int pos)	Reads an encoded BlobTree from a Blob.
ABlob	slice(long start, long end)	Gets a contiguous slice of this Blob, as a new Blob.
ABlob	toCanonical()	Converts this Cell to a canonical version.
Blob	toFlatBlob()	Converts the contents of this value to a flat array-backed Blob instance.
protected void	updateDigest(MessageDigest digest)
BlobTree	updateRefs(IRefFunction func)	Updates all Refs in this object using the given function.
void	validate()	Validates the complete structure of this object.
void	validateCell()	Validates the local structure and invariants of this cell.

Methods inherited from class convex.core.data.ABlob

byteAt, compareTo, computeHash, count, empty, encode, equals, get, getContentHash, getElementRef, getTag, getType, hashCode, isChunkPacked, isCVMValue, print, read, replaceSlice, shortAt, size, slice, toBlob, toByteBuffer

Methods inherited from class convex.core.data.ABlobLike

getBytes, getHexDigit, hexEquals, hexLength, isDataValue, toHexString, toHexString

Methods inherited from class convex.core.data.ACountable

isEmpty

Methods inherited from class convex.core.data.ACell

attachMemorySize, attachRef, cachedEncoding, cachedHash, calcMemorySize, createEncoding, createRef, equals, genericEquals, getCanonical, getChildRefs, getEncoding, getEncodingLength, getHash, getMemorySize, getMemorySize, getRef, isEmbedded, toCVMString, toString

Methods inherited from class convex.core.data.AObject

attachEncoding, print, print

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Details

BIT_SHIFT_PER_LEVEL

public static final int BIT_SHIFT_PER_LEVEL

See Also:

• Constant Field Values

FANOUT

public static final int FANOUT

See Also:

• Constant Field Values

MAX_ENCODING_SIZE

public static final int MAX_ENCODING_SIZE

Maximum byte length of an encoded BlobTree node. Note: - Last child might be embedded, others cannot

See Also:

• Constant Field Values

Method Details

create

public static BlobTree create(ABlob blob)

Create a BlobTree from an arbitrary Blob. Must be of sufficient size to convert to BlobTree

Parameters:

blob - Source of BlobTree data

Returns:

New BlobTree instance

createWithChildren

public static BlobTree createWithChildren(ABlob[] children)

Create a BlobTree with the given children. SECURITY: Does not validate children in any way

Parameters:

children - Child blobs for this BlobTree node

Returns:

New BlobTree instance

calcChunks

public static long calcChunks(long length)

Computes the number of chunks (4096 bytes or less) for the canonical BlobTree of the given length

Parameters:

length - The length of the Blob in bytes

Returns:

Number of chunks needed for a given byte length.

isCanonical

public boolean isCanonical()

Description copied from class: ACell

Returns true if this Cell is in a canonical representation. Non-canonical objects may be used on a temporary internal basis, they should be converted to canonical representations for general purpose use.

Specified by:

isCanonical in class ACell

Returns:

true if the object is in canonical format, false otherwise

getBytes

public int getBytes(byte[] dest,
 int pos)

Description copied from class: ABlobLike

Copies the bytes from this instance to a given destination array

Specified by:

getBytes in class ABlobLike<CVMLong>

Parameters:

dest - Destination array

pos - Offset into destination array

Returns:

End position in destination array after writing

slice

public ABlob slice(long start,
 long end)

Description copied from class: ABlob

Gets a contiguous slice of this Blob, as a new Blob. Shares underlying backing data where possible

Specified by:

slice in class ABlob

Parameters:

start - Start position for the created slice (inclusive)

end - End of the slice (exclusive)

Returns:

A blob of the specified length, representing a slice of this blob, or null if the slice is invalid

toFlatBlob

public Blob toFlatBlob()

Description copied from class: ABlobLike

Converts the contents of this value to a flat array-backed Blob instance. Warning: might be O(n) in size of Blob, may not be canonical etc.

Specified by:

toFlatBlob in class ABlob

Returns:

A Blob instance containing the same data as this value. Might be `this` if already a flat Blob.

updateDigest

protected void updateDigest(MessageDigest digest)

Specified by:

updateDigest in class ABlob

byteAtUnchecked

public byte byteAtUnchecked(long i)

Description copied from class: ABlobLike

Gets the byte at the specified position in this data object, possibly skipping bounds checking. Only safe if index is known to be in bounds, otherwise result is undefined.

Overrides:

byteAtUnchecked in class ABlobLike<CVMLong>

Parameters:

i - Index of the byte to get

Returns:

The byte at the specified position

equals

public boolean equals(ABlob a)

Description copied from class: ABlob

Determines if this Blob is equal to another Blob. Blobs are defined to be equal if they have the same encoded representation, i.e. if and only if all of the following are true: - Blobs are of the same length - All byte values are equal

Specified by:

equals in class ABlob

Parameters:

a - Blob to compare with

Returns:

true if Blobs are equal, false otherwise

equals

public boolean equals(BlobTree b)

equalsBytes

public boolean equalsBytes(byte[] bytes,
 long byteOffset)

Description copied from class: ABlob

Tests if the byte contents of this instance are equal to a subset of a byte array

Specified by:

equalsBytes in class ABlob

Parameters:

bytes - Byte array to compare with

byteOffset - Offset into byte array from which to start comparison

Returns:

true if exactly equal, false otherwise

equalsBytes

public boolean equalsBytes(ABlob b)

Description copied from class: ABlob

Tests if this Blob has exactly the same bytes as another Blob

Specified by:

equalsBytes in class ABlob

Parameters:

b - Blob to compare with

Returns:

True if byte content is exactly equal, false otherwise

encodeRaw

public int encodeRaw(byte[] bs,
 int pos)

Description copied from class: ABlob

Writes this Blob's encoding to a byte array, excluding the tag byte

Specified by:

encodeRaw in class ABlob

Parameters:

bs - A byte array to which to write the encoding

pos - The offset into the byte array

Returns:

New position after writing

read

public static BlobTree read(long count,
 Blob src,
 int pos)
                     throws BadFormatException

Reads an encoded BlobTree from a Blob. Assumes there will be encoded children.

Parameters:

count - Length to read

src - Source data, assumed to include tag and count at start

pos - Position to read from, assumed to be tag byte

Returns:

BlobTree instance.

Throws:

BadFormatException - If BlobTree encoding is invalid

estimatedEncodingSize

public int estimatedEncodingSize()

Description copied from interface: IWriteable

Estimate the encoded data size for this Cell. Used for quickly sizing buffers. Implementations should try to return a size that is highly likely to contain the entire object when encoded, including the tag byte. Should not traverse soft Refs, i.e. must be usable on arbitrary partial data structures

Returns:

The estimated size for the binary representation of this object.

append

public ABlob append(ABlob d)

Appends another blob to this BlobTree. Potentially O(n) but can be faster. We are careful to slice from (0...n) on the appended array, to minimise reconstruction of BlobTrees

Specified by:

append in class ABlob

Parameters:

d - Blob to append

Returns:

A new Blob, containing the additional data appended to this blob.

isFullyPacked

public boolean isFullyPacked()

Returns true if this is a fully packed set of chunks

Specified by:

isFullyPacked in class ABlob

Returns:

True if fully packed, false otherwise

getChunk

public Blob getChunk(long chunkIndex)

Description copied from class: ABlob

Gets a chunk of this Blob, as a canonical flat Blob up to the maximum Blob chunk size. Returns empty Blob if and only if referencing the end of a Blob with fully packed chunks

Specified by:

getChunk in class ABlob

Parameters:

chunkIndex - Index of chunk

Returns:

A Blob containing the specified chunk data.

validate

public void validate()
              throws InvalidDataException

Description copied from interface: IValidated

Validates the complete structure of this object. It is necessary to ensure all child Refs are validated, so the general contract for validate is:

1. Call super.validate() - which will indirectly call validateCell()
2. Call validate() on any contained cells in this class

Specified by:

validate in interface IValidated

Overrides:

validate in class ABlob

Throws:

InvalidDataException - If the data Value is invalid in any way

getByteBuffer

public ByteBuffer getByteBuffer()

Description copied from class: ABlob

Gets a byte buffer containing this Blob's raw data. Will have remaining bytes equal to this Blob's size.

Specified by:

getByteBuffer in class ABlob

Returns:

A ByteBuffer containing the Blob's data.

appendHex

public boolean appendHex(BlobBuilder bb,
 long length)

Description copied from class: ABlobLike

Append hex string up to the given length in hex digits (a multiple of two)

Overrides:

appendHex in class ABlobLike<CVMLong>

Parameters:

bb - BlobBuilder instance to append to

length - Length in Hex digits to append

Returns:

true if Blob fully appended, false if more more hex digits remain

validateCell

public void validateCell()
                  throws InvalidDataException

Description copied from class: ACell

Validates the local structure and invariants of this cell. Called by validate() super implementation. Should validate directly contained data, but should not validate all other structure of this cell. In particular, should not traverse potentially missing child Refs.

Overrides:

validateCell in class ABlob

Throws:

InvalidDataException - If the Cell is invalid

hexMatch

public long hexMatch(ABlobLike<?> b)

Description copied from class: ABlobLike

Computes the length of the longest common hex prefix between two blobs

Overrides:

hexMatch in class ABlobLike<CVMLong>

Parameters:

b - Blob to compare with

Returns:

The length of the longest common prefix in hex digits

commonHexPrefixLength

public long commonHexPrefixLength(ABlob b)

hexMatch

public long hexMatch(ABlobLike<?> b,
 long start,
 long length)

Description copied from class: ABlobLike

Returns the number of matching hex digits in the given hex range of another Blob. Assumes range is valid for both blobs. Returns length if this Blob is exactly equal to the specified hex range.

Specified by:

hexMatch in class ABlobLike<CVMLong>

Parameters:

b - Blob to compare with

start - Start position (in hex digits)

length - Length to compare (in hex digits)

Returns:

The number of matching hex characters

longValue

public long longValue()

Description copied from class: ABlobLike

Converts this BlobLike to the corresponding long value. Assumes big-endian format, as if the entire blob is interpreted as an unsigned big integer. Higher bytes outside the Long range will be ignored, i.e. the lowest 64 bits are taken

Specified by:

longValue in class ABlobLike<CVMLong>

Returns:

long value of this blob

getRefCount

public int getRefCount()

Description copied from class: ACell

Gets the number of Refs contained within this Cell. This number is final / immutable for any given instance and is defined by the Cell encoding rules. Contained Refs may be either external or embedded.

Overrides:

getRefCount in class ACell

Returns:

The number of Refs in this Cell

getRef

public <R extends ACell> Ref<R> getRef(int i)

Description copied from class: ACell

Gets a numbered child Ref from within this Cell. WARNING: May need to convert to a canonical instance

Overrides:

getRef in class ACell

Type Parameters:

R - Type of referenced Cell

Parameters:

i - Index of ref to get

Returns:

The Ref at the specified index

updateRefs

public BlobTree updateRefs(IRefFunction func)

Description copied from class: ACell

Updates all Refs in this object using the given function. The function *must not* change the hash value of Refs, in order to ensure structural integrity of modified data structures. The implementation *should* re-attach any original encoding in order to prevent re-encoding or surplus hashing This is a building block for a very sneaky trick that enables use to do a lot of efficient operations on large trees of smart references. Must return the same object if no Refs are altered.

Overrides:

updateRefs in class ACell

Parameters:

func - Ref update function

Returns:

Cell with updated Refs

childSize

public static long childSize(long length)

Gets the size of a BlobTree child for a blob of given total length.

Parameters:

length - Length of Blob

Returns:

Size of child, or 1 if not a BlobTree

childCount

public static int childCount(long length)

Gets the number of children for a BlobTree of given total length.

Parameters:

length - Length of Blob

Returns:

Number of Child blobs

toCanonical

public ABlob toCanonical()

Description copied from class: ACell

Converts this Cell to a canonical version. Must return this Cell if already canonical, may be O(n) in size of value otherwise. Callers should usually use getCanonical(), which caches canonical instances once created

Specified by:

toCanonical in class ABlob

Returns:

Canonical version of Cell

read

public int read(long offset,
 long count,
 ByteBuffer dest)

Description copied from class: ABlob

Gets bytes from this Blob into a ByteBuffer

Specified by:

read in class ABlob

Parameters:

offset - Offset into this Blob to read from

count - Number of bytes to read. Must be in bounds

dest - Destination byte buffer

Returns:

Number of bytes read