Package convex.core.data

Class Format

java.lang.Object

convex.core.data.Format

public class Format
extends Object

Static utility class for message format encoding "Standards are always out of date. That's what makes them standards." - Alan Bennett

Field Summary

Fields

Modifier and Type	Field	Description
static final long	FULL_EMBEDDED_MEMORY_SIZE	Memory size of a fully embedded value (zero)
static final int	LIMIT_ENCODING_LENGTH	8191 byte system-wide limit on the legal length of a data object encoding.
static final int	MAX_EMBEDDED_LENGTH	Maximum size in bytes of an embedded value, including tag
static final long	MAX_MESSAGE_LENGTH	Maximum allowed encoded message length in bytes
static final int	MAX_REF_LENGTH	Maximum length in bytes of a Ref encoding (may be an embedded data object)
static final int	MAX_VLC_COUNT_LENGTH	Maximum length for a VLC encoded Count
static final int	MAX_VLC_LONG_LENGTH	Maximum length for a VLC encoded Long
static final int	NULL_ENCODING_LENGTH	Encoded length of a null value

Constructor Summary

Constructors

Constructor	Description
Format()

Method Summary

Modifier and Type	Method	Description
static ACell[]	decodeCells(Blob data)	Decodes an array of Cells packed in a Blob.
static void	decodeCells(HashMap<Hash,ACell> acc, Blob data)	Decode encoded non-embedded Cells into an accumulator HashMap
static <T extends ACell> T	decodeMultiCell(Blob data)	Reads a cell from a Blob of data, allowing for non-embedded children following the first cell
static Blob	encodeCells(List<ACell> cells)	Encodes a flat list of cells in order specified in multi-cell format
static Blob	encodedBlob(ACell o)	Gets the encoded Blob for an object in canonical message format
static Blob	encodeDelta(List<ACell> cells)	Encode a list of cells as a delta message.
static String	encodedString(ACell cell)	Gets a hex String representing an object's encoding
static String	encodedString(Object o)	Gets a hex String representing an object's encoding.
static Blob	encodeMultiCell(ACell a, boolean everything)	Encode a Cell completely in multi-cell message format.
static int	estimateEncodingSize(ACell cell)	Estimate the encoding size of a Cell value.
static int	getEncodingLength(ACell value)
static int	getLongLength(long value)	Gets the length of a Long in bytes (minimum bytes needs to express value
static int	getVLCCountLength(long x)	Gets the length in bytes of VLC count encoding for the given long value
static int	getVLCLength(long x)	Gets the length in bytes of VLC encoding for the given long value
static boolean	isCanonical(ACell o)	Returns true if the object is a canonical data object.
static boolean	isEmbedded(ACell cell)	Determines if an object should be embedded directly in the encoding rather than referenced with a Ref / hash.
static int	peekMessageLength(ByteBuffer bb)	Peeks for a VLC encoded message length at the start of a ByteBuffer, which must contain at least 1 byte, maximum 2.
static <T extends ACell> T	read(Blob blob)	Decodes a single Value from a Blob.
static <T extends ACell> T	read(Blob blob, int offset)	Decodes a single Value from a Blob, starting at a given offset Assumes the presence of a tag.
static <T extends ACell> T	read(String hexString)	Helper method to read a value encoded as a hex string
static long	readLong(Blob blob, int offset, int length)	Reads a long value represented by the specified bytes in a Blob
static <T extends ACell> Ref<T>	readRef(Blob b, int pos)	Reads a Ref or embedded Cell value from a Blob Converts Embedded Cells to Direct Refs automatically.
static AString	readUTF8String(Blob blob, int pos, int len)	Reads UTF-8 String data from a Blob.
static AString	readUTF8String(ByteBuffer bb, int len)	Reads a UTF-8 String from a ByteBuffer.
static long	readVLCCount(byte[] data, int pos)	Reads a VLC encoded count (non-negative integer) as a long from the given location in a byte array.
static long	readVLCCount(AArrayBlob blob, int pos)
static long	readVLCLong(byte[] data, int pos)	Reads a VLC encoded long as a long from the given location in a byte array.
static long	readVLCLong(AArrayBlob blob, int pos)
protected static boolean	vlcContinuesFrom(byte octet)	Checks if VLC continues from given byte, i.e.
static long	vlcSignExtend(byte b)	Sign extend 7th bit (sign in a VLC byte) of a byte to all bits in a long i.e.
static int	write(byte[] bs, int pos, ACell cell)	Writes a cell encoding to a byte array, preceded by the appropriate tag
static int	writeHexDigits(byte[] bs, int pos, ABlob src, long start, long length)	Writes hex digits from digit position start, total length.
static ByteBuffer	writeLength(ByteBuffer bb, int i)
static ByteBuffer	writeMessageLength(ByteBuffer bb, int len)	Writes a message length as a VLC encoded long
static int	writeRawUTF8String(byte[] bs, int pos, String s)	Writes a raw string without tag to the byte array.
static int	writeVLCCount(byte[] bs, int pos, long x)	Puts a variable length count into the specified byte array (with no tag) Format: MSB of each byte 0=last octet, 1=more octets Following MSB, 7 bits of integer representation for each octet
static int	writeVLCLong(byte[] bs, int pos, long x)	Puts a variable length integer into the specified byte array (with no tag) Format: MSB of each byte 0=last octet, 1=more octets Following MSB, 7 bits of integer representation for each octet Second highest bit of first byte is interpreted as the sign
static ByteBuffer	writeVLCLong(ByteBuffer bb, long x)	Puts a VLC encoded long into the specified bytebuffer (with no tag) Format: MSB of each byte 0=last octet, 1=more octets Following MSB, 7 bits of integer representation for each octet Second highest bit of first byte is interpreted as the sign

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LIMIT_ENCODING_LENGTH

public static final int LIMIT_ENCODING_LENGTH

8191 byte system-wide limit on the legal length of a data object encoding. Technical reasons for this choice:

• This is the max length that can be VLC encoded in a 2 byte message header. This simplifies message encoding and decoding.
• It is big enough to include a 4096-byte Blob

See Also:

• Constant Field Values

MAX_VLC_LONG_LENGTH

public static final int MAX_VLC_LONG_LENGTH

Maximum length for a VLC encoded Long

See Also:

• Constant Field Values

MAX_VLC_COUNT_LENGTH

public static final int MAX_VLC_COUNT_LENGTH

Maximum length for a VLC encoded Count

See Also:

• Constant Field Values

MAX_EMBEDDED_LENGTH

public static final int MAX_EMBEDDED_LENGTH

Maximum size in bytes of an embedded value, including tag

See Also:

• Constant Field Values

NULL_ENCODING_LENGTH

public static final int NULL_ENCODING_LENGTH

Encoded length of a null value

See Also:

• Constant Field Values

MAX_REF_LENGTH

public static final int MAX_REF_LENGTH

Maximum length in bytes of a Ref encoding (may be an embedded data object)

MAX_MESSAGE_LENGTH

public static final long MAX_MESSAGE_LENGTH

Maximum allowed encoded message length in bytes

See Also:

• Constant Field Values

FULL_EMBEDDED_MEMORY_SIZE

public static final long FULL_EMBEDDED_MEMORY_SIZE

Memory size of a fully embedded value (zero)

See Also:

• Constant Field Values

Constructor Details

Format

public Format()

Method Details

getVLCLength

public static int getVLCLength(long x)

Gets the length in bytes of VLC encoding for the given long value

Parameters:

x - Long value to encode

Returns:

Length of VLC encoding

getVLCCountLength

public static int getVLCCountLength(long x)

Gets the length in bytes of VLC count encoding for the given long value

Parameters:

x - Long value to encode

Returns:

Length of VLC encoding

writeVLCLong

public static ByteBuffer writeVLCLong(ByteBuffer bb,
 long x)

Puts a VLC encoded long into the specified bytebuffer (with no tag) Format:

• MSB of each byte 0=last octet, 1=more octets
• Following MSB, 7 bits of integer representation for each octet
• Second highest bit of first byte is interpreted as the sign

Parameters:

bb - ByteBuffer to write to

x - Value to VLC encode

Returns:

Updated ByteBuffer

writeVLCLong

public static int writeVLCLong(byte[] bs,
 int pos,
 long x)

Puts a variable length integer into the specified byte array (with no tag) Format:

• MSB of each byte 0=last octet, 1=more octets
• Following MSB, 7 bits of integer representation for each octet
• Second highest bit of first byte is interpreted as the sign

Parameters:

bs - Byte array to write to

pos - Initial position in byte array

x - Long value to write

Returns:

end position in byte array after writing VLC long

writeVLCCount

public static int writeVLCCount(byte[] bs,
 int pos,
 long x)

Puts a variable length count into the specified byte array (with no tag) Format:

• MSB of each byte 0=last octet, 1=more octets
• Following MSB, 7 bits of integer representation for each octet

Parameters:

bs - Byte array to write to

pos - Initial position in byte array

x - Long value to write

Returns:

end position in byte array after writing VLC long

vlcSignExtend

public static long vlcSignExtend(byte b)

Sign extend 7th bit (sign in a VLC byte) of a byte to all bits in a long i.e. sign extend excluding the continuation bit: where VLC Byte = csxxxxxx

Parameters:

b - Byte to extend

Returns:

The sign-extended byte as a long

vlcContinuesFrom

protected static boolean vlcContinuesFrom(byte octet)

Checks if VLC continues from given byte, i.e. if high bit is set

Parameters:

octet -

Returns:

True if VLC coding continues, false otherwise

readVLCLong

public static long readVLCLong(AArrayBlob blob,
 int pos)
                        throws BadFormatException

Throws:

BadFormatException

readVLCLong

public static long readVLCLong(byte[] data,
 int pos)
                        throws BadFormatException

Reads a VLC encoded long as a long from the given location in a byte array. Assumes no tag

Parameters:

data - Byte array

pos - Position from which to read in byte array

Returns:

long value from byte array

Throws:

BadFormatException - If format is invalid, or reading beyond end of array

readVLCCount

public static long readVLCCount(byte[] data,
 int pos)
                         throws BadFormatException

Reads a VLC encoded count (non-negative integer) as a long from the given location in a byte array. Assumes no tag

Parameters:

data - Byte array

pos - Position from which to read in byte array

Returns:

long value from byte array

Throws:

BadFormatException - If format is invalid, or reading beyond end of array

readVLCCount

public static long readVLCCount(AArrayBlob blob,
 int pos)
                         throws BadFormatException

Throws:

BadFormatException

peekMessageLength

public static int peekMessageLength(ByteBuffer bb)
                             throws BadFormatException

Peeks for a VLC encoded message length at the start of a ByteBuffer, which must contain at least 1 byte, maximum 2. Does not move the buffer position.

Parameters:

bb - ByteBuffer containing a message length

Returns:

The message length, or negative if insufficient bytes

Throws:

BadFormatException - If the ByteBuffer does not start with a valid message length

writeMessageLength

public static ByteBuffer writeMessageLength(ByteBuffer bb,
 int len)

Writes a message length as a VLC encoded long

Parameters:

bb - ByteBuffer with capacity available for writing

len - Length of message to write

Returns:

The ByteBuffer after writing the message length

write

public static int write(byte[] bs,
 int pos,
 ACell cell)

Writes a cell encoding to a byte array, preceded by the appropriate tag

Parameters:

bs - Byte array to write to

pos - Starting position to write in byte array

cell - Cell to write (may be null)

Returns:

Position in byte array after writing the specified object

writeRawUTF8String

public static int writeRawUTF8String(byte[] bs,
 int pos,
 String s)

Writes a raw string without tag to the byte array. Includes length in bytes of UTF-8 representation

Parameters:

bs - Byte array

pos - Starting position to write in byte array

s - String to write

Returns:

Position in byte array after writing

readUTF8String

public static AString readUTF8String(ByteBuffer bb,
 int len)
                              throws BadFormatException

Reads a UTF-8 String from a ByteBuffer. Assumes any tag has already been read

Parameters:

bb - ByteBuffer to read from

len - Number of UTF-8 bytes to read

Returns:

String from ByteBuffer

Throws:

BadFormatException - If encoding is invalid

readUTF8String

public static AString readUTF8String(Blob blob,
 int pos,
 int len)
                              throws BadFormatException

Reads UTF-8 String data from a Blob. Assumes any tag has already been read

Parameters:

blob - Blob data to read from

pos - Position of first UTF-8 byte

len - Number of UTF-8 bytes to read

Returns:

String from ByteBuffer

Throws:

BadFormatException - If there are insufficient bytes in Blob to read

writeLength

public static ByteBuffer writeLength(ByteBuffer bb,
 int i)

readRef

public static <T extends ACell> Ref<T> readRef(Blob b,
 int pos)
                                        throws BadFormatException

Reads a Ref or embedded Cell value from a Blob Converts Embedded Cells to Direct Refs automatically.

Type Parameters:

T - Type of referenced value

Parameters:

b - Blob containing a ref to read

pos - Position to read Ref from (should point to tag)

Returns:

Ref as read from ByteBuffer

Throws:

BadFormatException - If the data is badly formatted, or a non-embedded value is found where it should be a Ref.

read

public static <T extends ACell> T read(Blob blob)
                                throws BadFormatException

Decodes a single Value from a Blob. Assumes the presence of a tag. throws an exception if the Blob contents are not fully consumed

Parameters:

blob - Blob representing the Encoding of the Value

Returns:

Value read from the blob of encoded data

Throws:

BadFormatException - In case of encoding error

read

public static <T extends ACell> T read(Blob blob,
 int offset)
                                throws BadFormatException

Decodes a single Value from a Blob, starting at a given offset Assumes the presence of a tag. throws an exception if the Blob contents are not fully consumed

Parameters:

blob - Blob representing the Encoding of the Value

offset - Offset of tag byte in blob

Returns:

Value read from the blob of encoded data

Throws:

BadFormatException - In case of encoding error

read

public static <T extends ACell> T read(String hexString)
                                throws BadFormatException

Helper method to read a value encoded as a hex string

Type Parameters:

T - Type of value to read

Parameters:

hexString - A valid hex String

Returns:

Value read

Throws:

BadFormatException - If encoding is invalid

isCanonical

public static boolean isCanonical(ACell o)

Returns true if the object is a canonical data object. Canonical data objects can be used as first class decentralised data objects.

Parameters:

o - Value to test

Returns:

true if object is canonical, false otherwise.

isEmbedded

public static boolean isEmbedded(ACell cell)

Determines if an object should be embedded directly in the encoding rather than referenced with a Ref / hash. Defined to be true for most small objects.

Parameters:

cell - Value to test

Returns:

true if object is embedded, false otherwise

encodedBlob

public static Blob encodedBlob(ACell o)

Gets the encoded Blob for an object in canonical message format

Parameters:

o - The object to encode

Returns:

Encoded data as a blob

writeHexDigits

public static int writeHexDigits(byte[] bs,
 int pos,
 ABlob src,
 long start,
 long length)

Writes hex digits from digit position start, total length. Fills final hex digit with 0 if length is odd.

Parameters:

bs - Byte array

pos - Position to write into byte array

src - Source Blob for hex digits

start - Start position in source blob (hex digit number from beginning)

length - Number of hex digits to write

Returns:

position after writing

encodedString

public static String encodedString(ACell cell)

Gets a hex String representing an object's encoding

Parameters:

cell - Any cell

Returns:

Hex String

encodedString

public static String encodedString(Object o)

Gets a hex String representing an object's encoding. Used in testing only.

Parameters:

o - Any object, will be cast to appropriate CVM type

Returns:

Hex String

estimateEncodingSize

public static int estimateEncodingSize(ACell cell)

Estimate the encoding size of a Cell value. Useful for pre-sizing buffers.

Parameters:

cell - Cell to estimate encoding size for

Returns:

Estimated encoding size. May not be precise.

decodeCells

public static ACell[] decodeCells(Blob data)
                           throws BadFormatException

Decodes an array of Cells packed in a Blob. Cells may be partial.

Parameters:

data - Data containing cell encodings in multi-cell format

Returns:

Array of decoded cells

Throws:

BadFormatException - In event of any encoding error detected

decodeMultiCell

public static <T extends ACell> T decodeMultiCell(Blob data)
                                           throws BadFormatException

Reads a cell from a Blob of data, allowing for non-embedded children following the first cell

Parameters:

data - Data to decode

Returns:

Cell instance

Throws:

BadFormatException - If encoding format is invalid

decodeCells

public static void decodeCells(HashMap<Hash,ACell> acc,
 Blob data)
                        throws BadFormatException

Decode encoded non-embedded Cells into an accumulator HashMap

Parameters:

acc - Accumulator for Cells, keyed by Hash

data - Encoding to read

Throws:

BadFormatException - In case of bad format, including any embedded values

encodeMultiCell

public static Blob encodeMultiCell(ACell a,
 boolean everything)

Encode a Cell completely in multi-cell message format. Format places top level cell first, following cells in arbitrary order.

Parameters:

a - Cell to Encode

everything - If true, traverse the entire Cell tree

Returns:

Blob encoding

encodeCells

public static Blob encodeCells(List<ACell> cells)

Encodes a flat list of cells in order specified in multi-cell format

Parameters:

cells - Cells to Encode

Returns:

Blob containing multi-cell encoding

encodeDelta

public static Blob encodeDelta(List<ACell> cells)

Encode a list of cells as a delta message. Encodes list in reverse order

Parameters:

cells - Cells to encode

Returns:

Encoded multi-cell blob containing the given cells

getEncodingLength

public static int getEncodingLength(ACell value)

readLong

public static long readLong(Blob blob,
 int offset,
 int length)
                     throws BadFormatException

Reads a long value represented by the specified bytes in a Blob

Parameters:

blob - Blob instance

offset - Offset into blob

length - Length in bytes to read

Returns:

Long value

Throws:

BadFormatException - If the Long format is not canonical (i.e. starts with 0x00)

getLongLength

public static int getLongLength(long value)

Gets the length of a Long in bytes (minimum bytes needs to express value

Parameters:

value - Long value to analyse

Returns:

Number of bytes needed to express Long