Package convex.core.data

Class BlobTree

java.lang.Object

convex.core.data.AObject

convex.core.data.ACell

convex.core.data.ACountable<CVMByte>

convex.core.data.ABlob

convex.core.data.BlobTree

All Implemented Interfaces:

IValidated, IWriteable, Comparable<ABlob>

public class BlobTree
extends ABlob

Implementation of a large Blob data structure consisting of 2 or more chunks. Intention is to enable relatively large binary content to be handled without too many tree levels, and without too many references in a single tree node We choose a branching factor of 16 as a reasonable tradeoff. Level 1 can hold up to 64k Level 2 can hold up to 1mb Level 3 can hold up to 16mb Level 4 can hold up to 256mb ... Level 15 (max) should be big enough for the moment One smart reference is maintained for each child node at each level

Field Summary

Fields

Modifier and Type	Field	Description
static final int	BIT_SHIFT_PER_LEVEL
static final int	FANOUT
static final int	MAX_ENCODING_SIZE	Maximum byte length of an encoded BlobTree node.

Fields inherited from class convex.core.data.ABlob

contentHash

Fields inherited from class convex.core.data.ACell

cachedRef, EMPTY_ARRAY

Fields inherited from class convex.core.data.AObject

encoding

Method Summary

Modifier and Type	Method	Description
ABlob	append(ABlob d)	Appends another blob to this BlobTree.
boolean	appendHex(BlobBuilder bb, long length)	Append hex string up to the given length in hex digits (a multiple of two)
static long	calcChunks(long length)	Computes the number of chunks (4096 bytes or less) for the canonical BlobTree of the given length
static int	childCount(long length)	Gets the number of children for a BlobTree of given total length.
static long	childSize(long length)	Gets the size of a BlobTree child for a blob of given total length.
long	commonHexPrefixLength(ABlob b)	Computes the length of the longest common hex prefix between two blobs
long	count()	Gets the length of this Blob
static BlobTree	create(ABlob blob)	Create a BlobTree from an arbitrary Blob.
static BlobTree	createWithChildren(ABlob[] children)	Create a BlobTree with the given children.
int	encode(byte[] bs, int pos)	Writes this Cell's encoding to a byte array, including a tag byte which will be written first.
int	encodeRaw(byte[] bs, int pos)	Writes this Blob's encoding to a byte array, excluding the tag byte
int	encodeRawData(byte[] bs, int pos)	Encodes the raw data of this Blob.
boolean	equals(ABlob a)	Determines if this Blob is equal to another Blob.
boolean	equals(BlobTree b)
boolean	equalsBytes(byte[] bytes, int byteOffset)	Tests if this Blob is equal to a subset of a byte array
boolean	equalsBytes(ABlob b)	Tests if this Blob has exactly the same bytes as another Blob
int	estimatedEncodingSize()	Estimate the encoded data size for this Cell.
ByteBuffer	getByteBuffer()	Gets a byte buffer containing this Blob's raw data.
void	getBytes(byte[] dest, int destOffset)	Copies the bytes from this blob to a given destination
Blob	getChunk(long chunkIndex)	Gets a chunk of this Blob, as a canonical Blob up to the maximum chunk size.
<R extends ACell> Ref<R>	getRef(int i)	Gets a numbered child Ref from within this Cell.
int	getRefCount()	Gets the number of Refs contained within this Cell.
byte	getTag()	Gets the tag byte for this cell.
byte	getUnchecked(long i)	Gets the byte at the specified position in this data object, without bounds checking.
long	hexMatchLength(ABlob b, long start, long length)	Returns the number of matching hex digits in the given hex range of another Blob.
boolean	isCanonical()	Returns true if this Cell is in a canonical representation for message writing.
boolean	isChunkPacked()	Returns true if this is a fully packed set of chunks
boolean	isFullyPacked()	Returns true if this is a fully packed set of chunks
long	longValue()	Gets the long value of this Blob if the length is exactly 8 bytes, otherwise throws an Exception
static BlobTree	read(Blob src, long count)
static BlobTree	read(ByteBuffer bb, long count)	Reads a BlobTree from a bytebuffer.
ABlob	slice(long start, long end)	Gets a contiguous slice of this Blob, as a new Blob.
ABlob	toCanonical()	Converts this Cell to its canonical version.
Blob	toFlatBlob()	Converts this object to a flat array-backed Blob instance.
long	toLong()	Converts this Blob to the corresponding long value.
protected void	updateDigest(MessageDigest digest)
BlobTree	updateRefs(IRefFunction func)	Updates all Refs in this object using the given function.
void	validate()	Validates the complete structure of this object.
void	validateCell()	Validates the local structure and invariants of this cell.
int	writeToBuffer(byte[] bs, int pos)	Writes the raw byte contents of this blob to a byte array
ByteBuffer	writeToBuffer(ByteBuffer bb)	Writes the raw byte contents of this Blob to a ByteBuffer.

Methods inherited from class convex.core.data.ABlob

byteAt, compareTo, computeHash, empty, equals, get, getBytes, getContentHash, getElementRef, getHexDigit, getType, hashCode, hexEquals, hexEquals, hexLength, isCVMValue, isRegularBlob, print, slice, toByteBuffer, toHexString, toHexString

Methods inherited from class convex.core.data.ACountable

isEmpty, size

Methods inherited from class convex.core.data.ACell

announce, announce, attachMemorySize, attachRef, cachedEncoding, cachedHash, calcMemorySize, createAnnounced, createEncoding, createPersisted, createPersisted, createRef, equals, getCanonical, getChildRefs, getEncoding, getEncodingLength, getHash, getMemorySize, getRef, isCompletelyEncoded, isEmbedded, mark, mark, toCVMString, toString, write

Methods inherited from class convex.core.data.AObject

attachEncoding, print, print

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Details

BIT_SHIFT_PER_LEVEL

public static final int BIT_SHIFT_PER_LEVEL

See Also:

• Constant Field Values

FANOUT

public static final int FANOUT

See Also:

• Constant Field Values

MAX_ENCODING_SIZE

public static final int MAX_ENCODING_SIZE

Maximum byte length of an encoded BlobTree node. Note: - Last child might be embedded, others cannot - With max 16 children , not possible to have biggest VLC length

See Also:

• Constant Field Values

Method Details

create

public static BlobTree create(ABlob blob)

Create a BlobTree from an arbitrary Blob. Must be of sufficient size to convert to BlobTree

Parameters:

blob - Source of BlobTree data

Returns:

New BlobTree instance

createWithChildren

public static BlobTree createWithChildren(ABlob[] children)

Create a BlobTree with the given children. SECURITY: Does not validate children in any way

Parameters:

children - Child blobs for this BlobTree node

Returns:

New BlobTree instance

calcChunks

public static long calcChunks(long length)

Computes the number of chunks (4096 bytes or less) for the canonical BlobTree of the given length

Parameters:

length - The length of the Blob in bytes

Returns:

Number of chunks needed for a given byte length.

isCanonical

public boolean isCanonical()

Description copied from class: ACell

Returns true if this Cell is in a canonical representation for message writing. Non-canonical objects may be used on a temporary internal basis, they must always be converted to canonical representations for external use (e.g. Encoding).

Specified by:

isCanonical in class ACell

Returns:

true if the object is in canonical format, false otherwise

getBytes

public void getBytes(byte[] dest,
 int destOffset)

Description copied from class: ABlob

Copies the bytes from this blob to a given destination

Specified by:

getBytes in class ABlob

Parameters:

dest - Destination array

destOffset - Offset into destination array

count

public long count()

Description copied from class: ABlob

Gets the length of this Blob

Specified by:

count in class ABlob

Returns:

The length in bytes of this data object

slice

public ABlob slice(long start,
 long end)

Description copied from class: ABlob

Gets a contiguous slice of this Blob, as a new Blob. Shares underlying backing data where possible

Specified by:

slice in class ABlob

Parameters:

start - Start position for the created slice (inclusive)

end - End of the slice (exclusive)

Returns:

A blob of the specified length, representing a slice of this blob.

toFlatBlob

public Blob toFlatBlob()

Description copied from class: ABlob

Converts this object to a flat array-backed Blob instance. Warning: might be O(n) in size of Blob, may not be canonical etc.

Specified by:

toFlatBlob in class ABlob

Returns:

A Blob instance containing the same data as this Blob.

updateDigest

protected void updateDigest(MessageDigest digest)

Specified by:

updateDigest in class ABlob

getUnchecked

public byte getUnchecked(long i)

Description copied from class: ABlob

Gets the byte at the specified position in this data object, without bounds checking. Only safe if index is known to be in bounds, otherwise result is undefined.

Specified by:

getUnchecked in class ABlob

Parameters:

i - Index of the byte to get

Returns:

The byte at the specified position

equals

public boolean equals(ABlob a)

Description copied from class: ABlob

Determines if this Blob is equal to another Blob. Blobs are defined to be equal if they have the same on-chain representation, i.e. if and only if all of the following are true: - Blob is of the same general type - Blobs are of the same length - All byte values are equal

Specified by:

equals in class ABlob

Parameters:

a - Blob to compare with

Returns:

true if Blobs are equal, false otherwise

equals

public boolean equals(BlobTree b)

equalsBytes

public boolean equalsBytes(byte[] bytes,
 int byteOffset)

Description copied from class: ABlob

Tests if this Blob is equal to a subset of a byte array

Specified by:

equalsBytes in class ABlob

Parameters:

bytes - Byte array to compare with

byteOffset - Offset into byte array

Returns:

true if exactly equal, false otherwise

equalsBytes

public boolean equalsBytes(ABlob b)

Description copied from class: ABlob

Tests if this Blob has exactly the same bytes as another Blob

Specified by:

equalsBytes in class ABlob

Parameters:

b - Blob to compare with

Returns:

True if byte content is exactly equal, false otherwise

encode

public int encode(byte[] bs,
 int pos)

Description copied from class: ACell

Writes this Cell's encoding to a byte array, including a tag byte which will be written first. Cell must be canonical, or else an error may occur.

Specified by:

encode in interface IWriteable

Specified by:

encode in class ACell

Parameters:

bs - A byte array to which to write the encoding

pos - The offset into the byte array

Returns:

New position after writing

encodeRaw

public int encodeRaw(byte[] bs,
 int pos)

Description copied from class: ABlob

Writes this Blob's encoding to a byte array, excluding the tag byte

Specified by:

encodeRaw in class ABlob

Parameters:

bs - A byte array to which to write the encoding

pos - The offset into the byte array

Returns:

New position after writing

encodeRawData

public int encodeRawData(byte[] bs,
 int pos)

Description copied from class: ABlob

Encodes the raw data of this Blob. Assumes buffer has enough space for (length) bytes.

Overrides:

encodeRawData in class ABlob

Parameters:

bs - Byte array to write to

pos - Position to write at

Returns:

Updates position

writeToBuffer

public ByteBuffer writeToBuffer(ByteBuffer bb)

Description copied from class: ABlob

Writes the raw byte contents of this Blob to a ByteBuffer. May be big!

Specified by:

writeToBuffer in class ABlob

Parameters:

bb - ByteBuffer to write to

Returns:

The passed ByteBuffer, after writing byte content

writeToBuffer

public int writeToBuffer(byte[] bs,
 int pos)

Description copied from class: ABlob

Writes the raw byte contents of this blob to a byte array

Specified by:

writeToBuffer in class ABlob

Parameters:

bs - Byte array to write to

pos - Starting position in byte array to write to

Returns:

The position in the array after writing

read

public static BlobTree read(ByteBuffer bb,
 long count)
                     throws BadFormatException

Reads a BlobTree from a bytebuffer. Assumes that tag byte and count are already read

Parameters:

bb - ByteBuffer

count - Count of bytes in BlobTree being read

Returns:

Decoded BlobTree

Throws:

BadFormatException - if the encoding was invalid

read

public static BlobTree read(Blob src,
 long count)
                     throws BadFormatException

Throws:

BadFormatException

estimatedEncodingSize

public int estimatedEncodingSize()

Description copied from interface: IWriteable

Estimate the encoded data size for this Cell. Used for quickly sizing buffers. Implementations should try to return a size that is likely to contain the entire object when represented in binary format, including the tag byte.

Returns:

The estimated size for the binary representation of this object.

append

public ABlob append(ABlob d)

Appends another blob to this BlobTree. Potentially O(n) but can be faster. We are careful to slice from (0...n) on the appended array, to minimise reconstruction of BlobTrees

Specified by:

append in class ABlob

Parameters:

d - Blob to append

Returns:

A new Blob, containing the additional data appended to this blob.

isFullyPacked

public boolean isFullyPacked()

Returns true if this is a fully packed set of chunks

Returns:

True if fully packed, false otherwise

isChunkPacked

public boolean isChunkPacked()

Returns true if this is a fully packed set of chunks

Returns:

True if fully packed, false otherwise

getChunk

public Blob getChunk(long chunkIndex)

Description copied from class: ABlob

Gets a chunk of this Blob, as a canonical Blob up to the maximum chunk size. Returns empty Blob if and only if referencing the end of a Blob with fully packed chunks

Specified by:

getChunk in class ABlob

Parameters:

chunkIndex - Index of chunk

Returns:

A Blob containing the specified chunk data.

validate

public void validate()
              throws InvalidDataException

Description copied from interface: IValidated

Validates the complete structure of this object. It is necessary to ensure all child Refs are validated, so the general contract for validate is:

1. Call super.validate() - which will indirectly call validateCell()
2. Call validate() on any contained cells in this class

Specified by:

validate in interface IValidated

Overrides:

validate in class ABlob

Throws:

InvalidDataException - If the data Value is invalid in any way

getByteBuffer

public ByteBuffer getByteBuffer()

Description copied from class: ABlob

Gets a byte buffer containing this Blob's raw data. Will have remaining bytes equal to this Blob's size.

Specified by:

getByteBuffer in class ABlob

Returns:

A ByteBuffer containing the Blob's data.

appendHex

public boolean appendHex(BlobBuilder bb,
 long length)

Description copied from class: ABlob

Append hex string up to the given length in hex digits (a multiple of two)

Specified by:

appendHex in class ABlob

Parameters:

bb - BlobBuilder instance to append to

length - Length in Hex digits to append

Returns:

true if Blob fully appended, false if more more hex digits remain

validateCell

public void validateCell()
                  throws InvalidDataException

Description copied from class: ACell

Validates the local structure and invariants of this cell. Called by validate() super implementation. Should validate directly contained data, but should not validate all other structure of this cell. In particular, should not traverse potentially missing child Refs.

Overrides:

validateCell in class ABlob

Throws:

InvalidDataException - If the Cell is invalid

commonHexPrefixLength

public long commonHexPrefixLength(ABlob b)

Description copied from class: ABlob

Computes the length of the longest common hex prefix between two blobs

Specified by:

commonHexPrefixLength in class ABlob

Parameters:

b - Blob to compare with

Returns:

The length of the longest common prefix in hex digits

hexMatchLength

public long hexMatchLength(ABlob b,
 long start,
 long length)

Description copied from class: ABlob

Returns the number of matching hex digits in the given hex range of another Blob. Assumes range is valid for both blobs. Returns length if this Blob is exactly equal to the specified hex range.

Specified by:

hexMatchLength in class ABlob

Parameters:

b - Blob to compare with

start - Start position (in hex digits)

length - Length to compare (in hex digits)

Returns:

The number of matching hex characters

longValue

public long longValue()

Description copied from class: ABlob

Gets the long value of this Blob if the length is exactly 8 bytes, otherwise throws an Exception

Specified by:

longValue in class ABlob

Returns:

The long value represented by the Blob

toLong

public long toLong()

Description copied from class: ABlob

Converts this Blob to the corresponding long value. Assumes big-endian format, as if the entire blob is interpreted as a big integer. Higher bytes outside the Long range will be ignored.

Specified by:

toLong in class ABlob

Returns:

long value of this blob

getRefCount

public int getRefCount()

Description copied from class: ACell

Gets the number of Refs contained within this Cell. This number is final / immutable for any given instance and is defined by the Cell encoding rules. Contained Refs may be either external or embedded.

Specified by:

getRefCount in class ACell

Returns:

The number of Refs in this Cell

getRef

public <R extends ACell> Ref<R> getRef(int i)

Description copied from class: ACell

Gets a numbered child Ref from within this Cell.

Overrides:

getRef in class ACell

Type Parameters:

R - Type of referenced Cell

Parameters:

i - Index of ref to get

Returns:

The Ref at the specified index

updateRefs

public BlobTree updateRefs(IRefFunction func)

Description copied from class: ACell

Updates all Refs in this object using the given function. The function *must not* change the hash value of Refs, in order to ensure structural integrity of modified data structures. This is a building block for a very sneaky trick that enables use to do a lot of efficient operations on large trees of smart references. Must return the same object if no Refs are altered.

Overrides:

updateRefs in class ACell

Parameters:

func - Ref update function

Returns:

Cell with updated Refs

getTag

public byte getTag()

Description copied from class: ACell

Gets the tag byte for this cell. The tag byte is always written as the first byte of the Cell's Encoding

Specified by:

getTag in class ACell

Returns:

Tag byte for this Cell

toCanonical

public ABlob toCanonical()

Description copied from class: ACell

Converts this Cell to its canonical version. Returns this Cell if already canonical, may be O(n) in size of value otherwise.

Specified by:

toCanonical in class ABlob

Returns:

Canonical version of Cell

childSize

public static long childSize(long length)

Gets the size of a BlobTree child for a blob of given total length.

Parameters:

length - Length of Blob

Returns:

Size of child, or 1 if not a BlobTree

childCount

public static int childCount(long length)

Gets the number of children for a BlobTree of given total length.

Parameters:

length - Length of Blob

Returns:

Number of Child blobs