Package convex.core.data

Class Ref<T extends ACell>

java.lang.Object

convex.core.data.AObject

convex.core.data.Ref<T>

Type Parameters:

T - Type of stored value

All Implemented Interfaces:

IValidated, IWriteable, Comparable<Ref<T>>

Direct Known Subclasses:

RefDirect, RefSoft

public abstract class Ref<T extends ACell>
extends AObject
implements Comparable<Ref<T>>, IWriteable, IValidated

Class representing a smart reference to a decentralised data value. "The greatest trick the Devil ever pulled was convincing the world he didn’t exist." - The Usual Suspects A Ref itself is not a Cell, but may be contained within a Cell, in which case the Cell class must implement IRefContainer in order to persist and update contained Refs correctly Refs include a status that indicates the level of validation proven. It is important not to rely on the value of a Ref until it has a sufficient status - e.g. a minimum status of PERSISTED is required to be able to guarantee walking an entire nested data structure. Guarantees: - O(1) access to the Hash value, cached on first access - O(1) access to the referenced object (though may required hitting storage if not cached) - Indirectly referenced values may be collected by the garbage collector, with the assumption that they can be retrieved from storage if required

Field Summary

Fields

Modifier and Type	Field	Description
static final int	ANNOUNCED	Ref status indicating the Ref has been shared by this peer in an announced Belief.
static final int	BAD_MASK	Mask bit for bad data, especially signatures proved invalid
static final int	EMBEDDING_MASK	Mask for embedding status
static final RefDirect<CVMBool>	FALSE_VALUE
protected int	flags	Flag values including Status of this Ref.
protected Hash	hash	Hash of the serialised representation of the value Computed and stored upon demand.
static final int	INDIRECT_ENCODING_LENGTH	Length of an external Reference encoding.
static final int	INTERNAL_FLAGS	Flags for internal constant values
static final int	INVALID	Ref status indicating that the Ref refers to data that has been proven to be invalid
static final int	KNOWN_EMBEDDED_MASK	Mask bit for a proven embedded value
static final int	MARKED	Ref status indicating the value is marked in the store for GC copying.
static final int	MAX_STATUS	Maximum Ref status
static final int	NON_EMBEDDED_MASK	Mask bit for a proven non-embedded value
static final RefDirect<?>	NULL_VALUE	Ref for null value.
static final int	PERSISTED	Ref status indicating the Ref has been deeply persisted in long term storage.
static final int	STATUS_MASK	Mask for Ref flag bits representing the Status
static final int	STORED	Ref status indicating the Ref has been shallowly persisted in long term storage.
static final RefDirect<CVMBool>	TRUE_VALUE
static final int	UNKNOWN	Ref status indicating the status of this Ref is unknown.
static final int	VALIDATED	Ref status indicating the Ref has been both persisted and validated as genuine valid CVM data.
static final int	VERIFICATION_MASK	Mask bit for bad data, especially signatures proved invalid
static final int	VERIFIED_MASK	Mask bit for verified data, especially signatures

Fields inherited from class convex.core.data.AObject

encoding

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	Ref(Hash hash, int flags)

Method Summary

Modifier and Type	Method	Description
ASet<ACell>	addAllToSet(ASet<ACell> store)	Adds the value of this Ref and all non-embedded child values to a given set.
final Hash	cachedHash()	Gets the Hash of this ref's value, or null if not yet computed
int	compareTo(Ref<T> a)
static <T extends ACell> Ref<T>[]	createArray(T[] values)
protected Blob	createEncoding()	Creates a Blob object representing this object.
final int	encode(byte[] bs, int pos)	Writes the ref to a byte array.
abstract Ref<T>	ensureCanonical()	Ensures this Ref is canonical
abstract boolean	equals(Ref<T> a)	Checks if two Ref Values are equal.
boolean	equals(Object o)
void	findMissing(HashSet<Hash> missingSet, long limit)	Finds all instances of missing data in this Ref, and adds them to the missing set
int	flagsWithStatus(int newStatus)	Gets flags with an updated status
static <T extends ACell> RefSoft<T>	forHash(Hash hash)	Creates a RefSoft using a specific Hash.
static <T extends ACell> Ref<T>	get(T value)	Returns a direct Ref wrapping the given value.
final long	getEncodingLength()	Gets the encoding length for writing this Ref.
int	getFlags()	Gets the flags for this Ref
abstract Hash	getHash()	Gets the Hash of this ref's value.
long	getMemorySize()	Gets the indirect memory size for this Ref
int	getStatus()	Gets the status of this Ref
abstract T	getValue()	Gets the value from this Ref.
int	hashCode()
abstract boolean	isDirect()	Return true if this Ref is a direct reference, i.e.
final boolean	isEmbedded()	Check if the Ref's value is embedded.
boolean	isMarked()	Return true if this Ref's status indicates it has definitely been marked within storage May return false negatives, e.g.
abstract boolean	isMissing()	Checks if this Ref refers to missing data, i.e.
boolean	isPersisted()	Return true if this Ref's status indicates it has definitely been persisted to storage.
Ref<T>	markEmbedded(boolean isEmbedded)
static int	mergeFlags(int a, int b)	Merges flags in an idempotent way.
static final <T extends ACell> Ref<T>	nil()
<R extends ACell> Ref<R>	persist()	Persists this Ref in the current store if not embedded and not already persisted.
<R extends ACell> Ref<R>	persist(Consumer<Ref<ACell>> noveltyHandler)	Persists this Ref in the current store if not embedded and not already persisted.
<R extends ACell> Ref<R>	persistShallow()	Persists a Ref shallowly in the current store.
<R extends ACell> Ref<R>	persistShallow(Consumer<Ref<ACell>> noveltyHandler)	Persists a Ref shallowly in the current store.
boolean	print(BlobBuilder bb, long limit)	Prints this Object to a readable String Representation.
static <T extends ACell> Ref<T>	readRaw(Blob b, int pos)	Reads a ref from the given Blob position.
Ref<T>	setFlags(int newFlags)	Sets the Flags for this Ref.
abstract RefDirect<T>	toDirect()	Converts this Ref to a RefDirect
abstract RefSoft<T>	toSoft(AStore store)	Converts this Ref to a RefSoft.
String	toString()
static <T extends ACell> T	update(T o, IRefFunction func)	Updates Refs in an arbitrary Cell
static <T extends ACell> Ref<T>[]	updateRefs(Ref<T>[] refs, IRefFunction func)	Updates an array of Refs with the given function.
static <R extends ACell> R	updateRefs(R a, IRefFunction func)
void	validate()	Validates the complete structure of this object.
abstract Ref<T>	withFlags(int newFlags)	Create a new Ref of the same type with updated flags
Ref<T>	withMinimumStatus(int newStatus)	Updates the Ref has the given status, at minimum Assumes any necessary changes to storage will be made separately.
abstract Ref<T>	withValue(T newValue)	Updates the value stored within this Ref.

Methods inherited from class convex.core.data.AObject

attachEncoding, getEncoding, print, print

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface convex.core.data.IWriteable

estimatedEncodingSize

Field Details

UNKNOWN

public static final int UNKNOWN

Ref status indicating the status of this Ref is unknown. This is the default for new Refs

See Also:

• Constant Field Values

STORED

public static final int STORED

Ref status indicating the Ref has been shallowly persisted in long term storage. The Ref can be made soft, and retrieved from storage if needed. No guarantee about the existence / status of any child objects.

See Also:

• Constant Field Values

PERSISTED

public static final int PERSISTED

Ref status indicating the Ref has been deeply persisted in long term storage. The Ref and its children can be assumed to be accessible for the life of the storage subsystem execution.

See Also:

• Constant Field Values

VALIDATED

public static final int VALIDATED

Ref status indicating the Ref has been both persisted and validated as genuine valid CVM data.

See Also:

• Constant Field Values

ANNOUNCED

public static final int ANNOUNCED

Ref status indicating the Ref has been shared by this peer in an announced Belief. This means that the Peer has a commitment to maintain this data

See Also:

• Constant Field Values

MARKED

public static final int MARKED

Ref status indicating the value is marked in the store for GC copying. Marked values are retained until next GC cycle

See Also:

• Constant Field Values

MAX_STATUS

public static final int MAX_STATUS

Maximum Ref status

See Also:

• Constant Field Values

STATUS_MASK

public static final int STATUS_MASK

Mask for Ref flag bits representing the Status

See Also:

• Constant Field Values

KNOWN_EMBEDDED_MASK

public static final int KNOWN_EMBEDDED_MASK

Mask bit for a proven embedded value

See Also:

• Constant Field Values

NON_EMBEDDED_MASK

public static final int NON_EMBEDDED_MASK

Mask bit for a proven non-embedded value

See Also:

• Constant Field Values

EMBEDDING_MASK

public static final int EMBEDDING_MASK

Mask for embedding status

See Also:

• Constant Field Values

VERIFIED_MASK

public static final int VERIFIED_MASK

Mask bit for verified data, especially signatures

See Also:

• Constant Field Values

BAD_MASK

public static final int BAD_MASK

Mask bit for bad data, especially signatures proved invalid

See Also:

• Constant Field Values

VERIFICATION_MASK

public static final int VERIFICATION_MASK

Mask bit for bad data, especially signatures proved invalid

See Also:

• Constant Field Values

INTERNAL_FLAGS

public static final int INTERNAL_FLAGS

Flags for internal constant values

See Also:

• Constant Field Values

INVALID

public static final int INVALID

Ref status indicating that the Ref refers to data that has been proven to be invalid

See Also:

• Constant Field Values

NULL_VALUE

public static final RefDirect<?> NULL_VALUE

Ref for null value. Important because we can't persist this, since null collides with the result of an empty soft reference.

TRUE_VALUE

public static final RefDirect<CVMBool> TRUE_VALUE

FALSE_VALUE

public static final RefDirect<CVMBool> FALSE_VALUE

INDIRECT_ENCODING_LENGTH

public static final int INDIRECT_ENCODING_LENGTH

Length of an external Reference encoding. Will be a tag byte plus the Hash length

See Also:

• Constant Field Values

hash

protected Hash hash

Hash of the serialised representation of the value Computed and stored upon demand.

flags

protected int flags

Flag values including Status of this Ref. See public Ref status constants. May be incremented atomically in the event of validation, proven storage.

Constructor Details

Ref

protected Ref(Hash hash,
 int flags)

Method Details

getStatus

public int getStatus()

Gets the status of this Ref

Returns:

UNKNOWN, PERSISTED, VERIFIED, ACCOUNCED or INVALID Ref status constants

flagsWithStatus

public int flagsWithStatus(int newStatus)

Gets flags with an updated status

Parameters:

newStatus - New status to apply to flags

Returns:

Updated flags (does not change this Ref)

getFlags

public int getFlags()

Gets the flags for this Ref

Returns:

flag int value

withMinimumStatus

public Ref<T> withMinimumStatus(int newStatus)

Updates the Ref has the given status, at minimum Assumes any necessary changes to storage will be made separately. SECURITY: Dangerous if misused since may invalidate storage assumptions

Parameters:

newStatus - New status to apply to Ref

Returns:

Updated Ref (may be same Ref is status unchanged)

withFlags

public abstract Ref<T> withFlags(int newFlags)

Create a new Ref of the same type with updated flags

Parameters:

newFlags - New flags to set

Returns:

Updated Ref

getValue

public abstract T getValue()

Gets the value from this Ref. Important notes: - May throw a MissingDataException if the data does not exist in available storage - Will return null if and only if the Ref refers to the null value

Returns:

The value contained in this Ref

hashCode

public int hashCode()

Overrides:

hashCode in class Object

print

public boolean print(BlobBuilder bb,
 long limit)

Description copied from class: AObject

Prints this Object to a readable String Representation. SECURITY: Must halt and return false in O(1) time when limit of printing is exceeded otherwise DoS attacks may be possible.

Specified by:

print in class AObject

Parameters:

bb - BlobBuilder to append to. May be partially written if print limit exceeded

limit - Limit of printing in string bytes

Returns:

True if fully printed within limit, false otherwise

toString

public String toString()

Overrides:

toString in class Object

equals

public boolean equals(Object o)

Overrides:

equals in class Object

equals

public abstract boolean equals(Ref<T> a)

Checks if two Ref Values are equal. Equality is defined as referring to the same data, i.e. have an identical hash.

Parameters:

a - The Ref to compare with

Returns:

true if Refs have the same value, false otherwise

compareTo

public int compareTo(Ref<T> a)

Specified by:

compareTo in interface Comparable<T extends ACell>

getHash

public abstract Hash getHash()

Gets the Hash of this ref's value.

Returns:

Hash of the value

cachedHash

public final Hash cachedHash()

Gets the Hash of this ref's value, or null if not yet computed

Returns:

Hash of the value, or null if not yet computed

get

public static <T extends ACell> Ref<T> get(T value)

Returns a direct Ref wrapping the given value. Does not perform any Ref lookup in stores etc.

Parameters:

value - Value to wrap in the Ref

Returns:

New Ref wrapping the given value.

forHash

public static <T extends ACell> RefSoft<T> forHash(Hash hash)

Creates a RefSoft using a specific Hash. Fetches the actual value lazily from the current thread's store on demand. Internal soft reference may be initially empty: This Ref might not have available data in the store, in which case calls to getValue() may result in a MissingDataException WARNING: Does not mark as either embedded or non-embedded, as this might be a top level entry in the store. isEmbedded() will query the store to determine status.

Parameters:

hash - The hash value for this Ref to refer to

Returns:

Ref for the specific hash.

markEmbedded

public Ref<T> markEmbedded(boolean isEmbedded)

setFlags

public Ref<T> setFlags(int newFlags)

Sets the Flags for this Ref. WARNING: caller must have performed any necessary validation

Parameters:

newFlags - Flags to set

Returns:

Updated Ref

readRaw

public static <T extends ACell> Ref<T> readRaw(Blob b,
 int pos)
                                        throws BadFormatException

Reads a ref from the given Blob position. Assumes no tag. Marks as non-embedded, since only non-embedded cells should be encoded this way

Parameters:

b - Blob containing the data to read at the current position

pos - position in Blob to read

Returns:

Ref read from ByteBuffer

Throws:

BadFormatException - If there are insufficient bytes to read a full Ref

validate

public void validate()
              throws InvalidDataException

Description copied from interface: IValidated

Validates the complete structure of this object. It is necessary to ensure all child Refs are validated, so the general contract for validate is:

1. Call super.validate() - which will indirectly call validateCell()
2. Call validate() on any contained cells in this class

Specified by:

validate in interface IValidated

Throws:

InvalidDataException - If the data Value is invalid in any way

isDirect

public abstract boolean isDirect()

Return true if this Ref is a direct reference, i.e. the value is pinned in memory and cannot be garbage collected

Returns:

true if this Ref is direct, false otherwise

isPersisted

public boolean isPersisted()

Return true if this Ref's status indicates it has definitely been persisted to storage. May return false negatives, e.g. the object could be in the store but this Ref instance still has a status of "UNKNOWN".

Returns:

true if this Ref has a status of PERSISTED or above, false otherwise

isMarked

public boolean isMarked()

Return true if this Ref's status indicates it has definitely been marked within storage May return false negatives, e.g. the object could be marked in the store but this Ref instance still has a status of "UNKNOWN".

Returns:

true if this Ref has a status of MARKED or above, false otherwise

persist

public <R extends ACell> Ref<R> persist(Consumer<Ref<ACell>> noveltyHandler)

Persists this Ref in the current store if not embedded and not already persisted. This may convert the Ref from a direct reference to a soft reference. If the persisted Ref represents novelty, will trigger the specified novelty handler

Parameters:

noveltyHandler - Novelty handler to call (may be null)

Returns:

the persisted Ref

Throws:

MissingDataException - If the Ref's value does not exist or has been garbage collected before being persisted

persist

public <R extends ACell> Ref<R> persist()

Persists this Ref in the current store if not embedded and not already persisted. Resulting status will be PERSISTED or higher. This may convert the Ref from a direct reference to a soft reference.

Returns:

the persisted Ref

Throws:

MissingDataException - if the Ref cannot be fully persisted.

updateRefs

public static <T extends ACell> Ref<T>[] updateRefs(Ref<T>[] refs,
 IRefFunction func)

Updates an array of Refs with the given function. Returns the original array unchanged if no refs were changed, otherwise returns a new array.

Parameters:

refs - Array of Refs to update

func - Ref update function

Returns:

Array of updated Refs

createArray

public static <T extends ACell> Ref<T>[] createArray(T[] values)

addAllToSet

public ASet<ACell> addAllToSet(ASet<ACell> store)

Adds the value of this Ref and all non-embedded child values to a given set. Logically, provides the guarantee that the set will contain all cells needed to recreate the complete value of this Ref.

Parameters:

store - Store to add to

Returns:

Set containing this Ref and all direct or indirect child refs

isEmbedded

public final boolean isEmbedded()

Check if the Ref's value is embedded. If false, the value must be a branch.

Returns:

true if embedded, false otherwise

toDirect

public abstract RefDirect<T> toDirect()

Converts this Ref to a RefDirect

Returns:

Direct Ref

toSoft

public abstract RefSoft<T> toSoft(AStore store)

Converts this Ref to a RefSoft. Does not perform any persistence: doing this may make MissingDataExceptions happen.

Parameters:

store - store to set

Returns:

Direct Ref

persistShallow

public <R extends ACell> Ref<R> persistShallow()

Persists a Ref shallowly in the current store. Status will be updated to STORED or higher.

Returns:

Ref with status of STORED or above

persistShallow

public <R extends ACell> Ref<R> persistShallow(Consumer<Ref<ACell>> noveltyHandler)

Persists a Ref shallowly in the current store. Status will be updated STORED or higher. Novelty handler will be called exactly once if and only if the ref was not previously stored

Parameters:

noveltyHandler - Novelty handler to call (may be null)

Returns:

Ref with status of STORED or above

withValue

public abstract Ref<T> withValue(T newValue)

Updates the value stored within this Ref. New value must be equal in value to the old value (identical hash), but may have updated internal refs etc.

Parameters:

newValue - New value

Returns:

Updated Ref. May be identical if value unchanged

encode

public final int encode(byte[] bs,
 int pos)

Writes the ref to a byte array. Embeds embedded values as necessary.

Specified by:

encode in interface IWriteable

Parameters:

bs - Byte array to encode to

pos - position at which to write the value

Returns:

Updated position

createEncoding

protected Blob createEncoding()

Description copied from class: AObject

Creates a Blob object representing this object. Should be called only after the cached encoding has been checked.

Specified by:

createEncoding in class AObject

Returns:

Blob Encoding of Object

getEncodingLength

public final long getEncodingLength()

Gets the encoding length for writing this Ref. Will be equal to the encoding length of the Ref's value if embedded, otherwise INDIRECT_ENCODING_LENGTH

Returns:

Exact length of encoding

getMemorySize

public long getMemorySize()

Gets the indirect memory size for this Ref

Returns:

0 for fully embedded values with no child refs, memory size of referred value otherwise

findMissing

public void findMissing(HashSet<Hash> missingSet,
 long limit)

Finds all instances of missing data in this Ref, and adds them to the missing set

Parameters:

missingSet - Set to add missing instances to

limit - Maximum number of missing branches to identity

isMissing

public abstract boolean isMissing()

Checks if this Ref refers to missing data, i.e. a Cell that does not exist in the current store. May cause a read to the store.

Returns:

true if this specific Ref has missing data, false otherwise.

mergeFlags

public static int mergeFlags(int a,
 int b)

Merges flags in an idempotent way. Assume flags are valid

Parameters:

a - First set of flags

b - Second set of flags

Returns:

Merged flags

ensureCanonical

public abstract Ref<T> ensureCanonical()

Ensures this Ref is canonical

Returns:

this Ref if already canonical, potentially a new Ref with canonical value otherwise

update

public static <T extends ACell> T update(T o,
 IRefFunction func)

Updates Refs in an arbitrary Cell

Type Parameters:

T - Type of Cell

Parameters:

o - Cell to update

func - Ref update function

Returns:

Updated Cell (will be the same cell if Refs unchanged)

updateRefs

public static <R extends ACell> R updateRefs(R a,
 IRefFunction func)

nil

public static final <T extends ACell> Ref<T> nil()