Package convex.core.data

Class Ref<T extends ACell>

java.lang.Object
convex.core.data.AObject
convex.core.data.Ref<T>
Type Parameters:
T - Type of stored value
All Implemented Interfaces:
IValidated, IWriteable, Comparable<Ref<T>>
Direct Known Subclasses:
RefDirect, RefSoft
public abstract class Ref<T extends ACell> extends AObject implements Comparable<Ref<T>>, IWriteable, IValidated
Class representing a smart reference to a decentralised data value. "The greatest trick the Devil ever pulled was convincing the world he didn’t exist." - The Usual Suspects A Ref itself is not a Cell, but may be contained within a Cell, in which case the Cell class must implement IRefContainer in order to persist and update contained Refs correctly Refs include a status that indicates the level of validation proven. It is important not to rely on the value of a Ref until it has a sufficient status - e.g. a minimum status of PERSISTED is required to be able to guarantee walking an entire nested data structure. Guarantees: - O(1) access to the Hash value, cached on first access - O(1) access to the referenced object (though may required hitting storage if not cached) - Indirectly referenced values may be collected by the garbage collector, with the assumption that they can be retrieved from storage if required

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final int
    ANNOUNCED
    Ref status indicating the Ref has been shared by this peer in an announced Belief.
    static final int
    BAD_MASK
    Mask bit for bad data, especially signatures proved invalid
    static final int
    EMBEDDING_MASK
    Mask for embedding status
    static final RefDirect<CVMBool>
    FALSE_VALUE
     
    protected int
    flags
    Flag values including Status of this Ref.
    protected Hash
    hash
    Hash of the serialised representation of the value Computed and stored upon demand.
    static final int
    INDIRECT_ENCODING_LENGTH
    Length of an external Reference encoding.
    static final int
    INTERNAL
    Ref status indicating the Ref is an internal embedded value that can be encoded and used independently of any given store state
    static final int
    INTERNAL_FLAGS
    Flags for internal constant values
    static final int
    INVALID
    Ref status indicating that the Ref refers to data that has been proven to be invalid
    static final int
    KNOWN_EMBEDDED_MASK
    Mask bit for a proven embedded value
    static final int
    MARKED
    Ref status indicating the value is marked in the store for GC copying.
    static final int
    MAX_STATUS
    Maximum Ref status
    static final int
    NON_EMBEDDED_MASK
    Mask bit for a proven non-embedded value
    static final RefDirect<?>
    NULL_VALUE
    Ref for null value.
    static final int
    PERSISTED
    Ref status indicating the Ref has been deeply persisted in long term storage.
    static final int
    STATUS_MASK
    Mask for Ref flag bits representing the Status
    static final int
    STORED
    Ref status indicating the Ref has been shallowly persisted in long term storage.
    static final RefDirect<CVMBool>
    TRUE_VALUE
     
    static final int
    UNKNOWN
    Ref status indicating the status of this Ref is unknown.
    static final int
    VALIDATED
    Ref status indicating the Ref has been both persisted and validated as genuine valid CVM data.
    static final int
    VERIFICATION_MASK
    Mask bit for bad data, especially signatures proved invalid
    static final int
    VERIFIED_MASK
    Mask bit for verified data, especially signatures

    Fields inherited from class convex.core.data.AObject

    encoding

  • Constructor Summary

    Constructors
    Modifier
    Constructor
    Description
    protected
    Ref(Hash hash, int flags)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    static Set<Ref<?>>
    accumulateRefSet(Object a)
    Accumulates the set of all unique Refs in the given object.
    ASet<ACell>
    addAllToSet(ASet<ACell> store)
    Adds the value of this Ref and all non-embedded child values to a given set.
    final Hash
    cachedHash()
    Gets the Hash of this ref's value, or null if not yet computed
    int
    compareTo(Ref<T> a)
     
    static <T extends ACell>
    Ref<T>[]
    createArray(T[] values)
     
    protected Blob
    createEncoding()
    Creates a Blob object representing this object.
    final int
    encode(byte[] bs, int pos)
    Writes the ref to a byte array.
    abstract Ref<T>
    ensureCanonical()
    Ensures this Ref is canonical
    abstract boolean
    equals(Ref<T> a)
    Checks if two Ref Values are equal.
    boolean
    equals(Object o)
     
    void
    findMissing(HashSet<Hash> missingSet)
    Finds all instances of missing data in this Ref, and adds them to the missing set
    int
    flagsWithStatus(int newStatus)
    Gets flags with an updated status
    static <T extends ACell>
    RefSoft<T>
    forHash(Hash hash)
    Creates a RefSoft using a specific Hash.
    static <T extends ACell>
    Ref<T>
    get(Object value)
     
    static <T extends ACell>
    Ref<T>
    get(T value)
    Returns a direct Ref wrapping the given value.
    final long
    getEncodingLength()
    Gets the encoding length for writing this Ref.
    int
    getFlags()
    Gets the flags for this Ref
    abstract Hash
    getHash()
    Gets the Hash of this ref's value.
    long
    getMemorySize()
    Gets the indirect memory size for this Ref
    int
    getStatus()
    Gets the status of this Ref
    abstract T
    getValue()
    Gets the value from this Ref.
    int
    hashCode()
     
    abstract boolean
    isDirect()
    Return true if this Ref is a direct reference, i.e.
    final boolean
    isEmbedded()
    Check if the Ref's value is embedded.
    boolean
    isMarked()
    Return true if this Ref's status indicates it has definitely been marked within storage May return false negatives, e.g.
    abstract boolean
    isMissing()
    Checks if this Ref refers to missing data, i.e.
    boolean
    isPersisted()
    Return true if this Ref's status indicates it has definitely been persisted to storage.
    Ref<T>
    markEmbedded(boolean isEmbedded)
     
    static int
    mergeFlags(int a, int b)
    Merges flags in an idempotent way.
    <R extends ACell>
    Ref<R>
    persist()
    Persists this Ref in the current store if not embedded and not already persisted.
    <R extends ACell>
    Ref<R>
    persist(Consumer<Ref<ACell>> noveltyHandler)
    Persists this Ref in the current store if not embedded and not already persisted.
    <R extends ACell>
    Ref<R>
    persistShallow()
    Persists a Ref shallowly in the current store.
    <R extends ACell>
    Ref<R>
    persistShallow(Consumer<Ref<ACell>> noveltyHandler)
    Persists a Ref shallowly in the current store.
    boolean
    print(BlobBuilder bb, long limit)
    Prints this Object to a readable String Representation.
    static <T extends ACell>
    Ref<T>
    readRaw(ByteBuffer data)
    Reads a ref from the given ByteBuffer.
    Ref<T>
    setFlags(int newFlags)
    Sets the Flags for this Ref.
    Ref<T>
    toDirect()
    Converts this Ref to a RefDirect
    String
    toString()
     
    static <T extends ACell>
    Ref<T>[]
    updateRefs(Ref<T>[] refs, IRefFunction func)
    Updates an array of Refs with the given function.
    void
    validate()
    Validates the complete structure of this object.
    abstract Ref<T>
    withFlags(int newFlags)
    Create a new Ref of the same type with updated flags
    Ref<T>
    withMinimumStatus(int newStatus)
    Ensures the Ref has the given status, at minimum Assumes any necessary changes to storage will be made separately.
    abstract Ref<T>
    withValue(T newValue)
    Updates the value stored within this Ref.
    final ByteBuffer
    write(ByteBuffer bb)
    Writes this object to a ByteBuffer including an appropriate message tag

    Methods inherited from class convex.core.data.AObject

    attachEncoding, getEncoding, print, print

    Methods inherited from class java.lang.Object

    clone, finalize, getClass, notify, notifyAll, wait, wait, wait

    Methods inherited from interface convex.core.data.IWriteable

    estimatedEncodingSize

  • Field Details

    • UNKNOWN

      public static final int UNKNOWN
      Ref status indicating the status of this Ref is unknown. This is the default for new Refs
      See Also:

    • STORED

      public static final int STORED
      Ref status indicating the Ref has been shallowly persisted in long term storage. The Ref can be made soft, and retrieved from storage if needed. No guarantee about the existence / status of any child objects.
      See Also:

    • PERSISTED

      public static final int PERSISTED
      Ref status indicating the Ref has been deeply persisted in long term storage. The Ref and its children can be assumed to be accessible for the life of the storage subsystem execution.
      See Also:

    • VALIDATED

      public static final int VALIDATED
      Ref status indicating the Ref has been both persisted and validated as genuine valid CVM data.
      See Also:

    • ANNOUNCED

      public static final int ANNOUNCED
      Ref status indicating the Ref has been shared by this peer in an announced Belief. This means that the Peer has a commitment to maintain this data
      See Also:

    • INTERNAL

      public static final int INTERNAL
      Ref status indicating the Ref is an internal embedded value that can be encoded and used independently of any given store state
      See Also:

    • MARKED

      public static final int MARKED
      Ref status indicating the value is marked in the store for GC copying. Marked values are retained until next GC cycle
      See Also:

    • MAX_STATUS

      public static final int MAX_STATUS
      Maximum Ref status
      See Also:

    • STATUS_MASK

      public static final int STATUS_MASK
      Mask for Ref flag bits representing the Status
      See Also:

    • KNOWN_EMBEDDED_MASK

      public static final int KNOWN_EMBEDDED_MASK
      Mask bit for a proven embedded value
      See Also:

    • NON_EMBEDDED_MASK

      public static final int NON_EMBEDDED_MASK
      Mask bit for a proven non-embedded value
      See Also:

    • EMBEDDING_MASK

      public static final int EMBEDDING_MASK
      Mask for embedding status
      See Also:

    • VERIFIED_MASK

      public static final int VERIFIED_MASK
      Mask bit for verified data, especially signatures
      See Also:

    • BAD_MASK

      public static final int BAD_MASK
      Mask bit for bad data, especially signatures proved invalid
      See Also:

    • VERIFICATION_MASK

      public static final int VERIFICATION_MASK
      Mask bit for bad data, especially signatures proved invalid
      See Also:

    • INTERNAL_FLAGS

      public static final int INTERNAL_FLAGS
      Flags for internal constant values
      See Also:

    • INVALID

      public static final int INVALID
      Ref status indicating that the Ref refers to data that has been proven to be invalid
      See Also:

    • NULL_VALUE

      public static final RefDirect<?> NULL_VALUE
      Ref for null value. Important because we can't persist this, since null collides with the result of an empty soft reference.

    • TRUE_VALUE

      public static final RefDirect<CVMBool> TRUE_VALUE

    • FALSE_VALUE

      public static final RefDirect<CVMBool> FALSE_VALUE

    • INDIRECT_ENCODING_LENGTH

      public static final int INDIRECT_ENCODING_LENGTH
      Length of an external Reference encoding. Will be a tag byte plus the Hash length
      See Also:

    • hash

      protected Hash hash
      Hash of the serialised representation of the value Computed and stored upon demand.

    • flags

      protected int flags
      Flag values including Status of this Ref. See public Ref status constants. May be incremented atomically in the event of validation, proven storage.

  • Constructor Details

    • Ref

      protected Ref(Hash hash, int flags)

  • Method Details

    • getStatus

      public int getStatus()
      Gets the status of this Ref
      Returns:
      UNKNOWN, PERSISTED, VERIFIED, ACCOUNCED or INVALID Ref status constants

    • flagsWithStatus

      public int flagsWithStatus(int newStatus)
      Gets flags with an updated status
      Parameters:
      newStatus - New status to apply to flags
      Returns:
      Updated flags (does not change this Ref)

    • getFlags

      public int getFlags()
      Gets the flags for this Ref
      Returns:
      flag int value

    • withMinimumStatus

      public Ref<T> withMinimumStatus(int newStatus)
      Ensures the Ref has the given status, at minimum Assumes any necessary changes to storage will be made separately. SECURITY: Dangerous if misused since may invalidate storage assumptions
      Parameters:
      newStatus - New status to apply to Ref
      Returns:
      Updated Ref

    • withFlags

      public abstract Ref<T> withFlags(int newFlags)
      Create a new Ref of the same type with updated flags
      Parameters:
      newFlags - New flags to set
      Returns:
      Updated Ref

    • getValue

      public abstract T getValue()
      Gets the value from this Ref. Important notes: - May throw a MissingDataException if the data does not exist in available storage - Will return null if and only if the Ref refers to the null value
      Returns:
      The value contained in this Ref

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • print

      public boolean print(BlobBuilder bb, long limit)
      Description copied from class: AObject
      Prints this Object to a readable String Representation. SECURITY: Must halt and return false in O(1) time if limit of printing is exceeded otherwise DoS attacks may be possible.
      Specified by:
      print in class AObject
      Parameters:
      bb - BlobBuilder to append to
      limit - Limit of printing in string bytes
      Returns:
      True if fully printed within limit, false otherwise

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class Object

    • equals

      public abstract boolean equals(Ref<T> a)
      Checks if two Ref Values are equal. Equality is defined as referring to the same data, i.e. have an identical hash.
      Parameters:
      a - The Ref to compare with
      Returns:
      true if Refs have the same value, false otherwise

    • compareTo

      public int compareTo(Ref<T> a)
      Specified by:
      compareTo in interface Comparable<T extends ACell>

    • getHash

      public abstract Hash getHash()
      Gets the Hash of this ref's value.
      Returns:
      Hash of the value

    • cachedHash

      public final Hash cachedHash()
      Gets the Hash of this ref's value, or null if not yet computed
      Returns:
      Hash of the value, or null if not yet computed

    • get

      public static <T extends ACell> Ref<T> get(T value)
      Returns a direct Ref wrapping the given value. Does not perform any Ref lookup in stores etc.
      Parameters:
      value - Value to wrap in the Ref
      Returns:
      New Ref wrapping the given value.

    • get

      public static <T extends ACell> Ref<T> get(Object value)

    • forHash

      public static <T extends ACell> RefSoft<T> forHash(Hash hash)
      Creates a RefSoft using a specific Hash. Fetches the actual value lazily from the store on demand. Internal soft reference may be initially empty: This Ref might not have available data in the store, in which case calls to getValue() may result in a MissingDataException WARNING: Does not mark as either embedded or non-embedded, as this might be a top level entry in the store. isEmbedded() will query the store to determine status.
      Parameters:
      hash - The hash value for this Ref to refer to
      Returns:
      Ref for the specific hash.

    • markEmbedded

      public Ref<T> markEmbedded(boolean isEmbedded)

    • setFlags

      public Ref<T> setFlags(int newFlags)
      Sets the Flags for this Ref. WARNING: caller must have performed any necessary validation
      Parameters:
      newFlags - Flags to set
      Returns:
      Updated Ref

    • readRaw

      public static <T extends ACell> Ref<T> readRaw(ByteBuffer data)
      Reads a ref from the given ByteBuffer. Assumes no tag. Marks as non-embedded
      Parameters:
      data - ByteBuffer containing the data to read at the current position
      Returns:
      Ref read from ByteBuffer

    • validate

      public void validate() throws InvalidDataException
      Description copied from interface: IValidated
      Validates the complete structure of this object. It is necessary to ensure all child Refs are validated, so the general contract for validate is:
      1. Call super.validate() - which will indirectly call validateCell()
      2. Call validate() on any contained cells in this class
      Specified by:
      validate in interface IValidated
      Throws:
      InvalidDataException - If the data Value is invalid in any way

    • isDirect

      public abstract boolean isDirect()
      Return true if this Ref is a direct reference, i.e. the value is pinned in memory and cannot be garbage collected
      Returns:
      true if this Ref is direct, false otherwise

    • isPersisted

      public boolean isPersisted()
      Return true if this Ref's status indicates it has definitely been persisted to storage. May return false negatives, e.g. the object could be in the store but this Ref instance still has a status of "UNKNOWN".
      Returns:
      true if this Ref has a status of PERSISTED or above, false otherwise

    • isMarked

      public boolean isMarked()
      Return true if this Ref's status indicates it has definitely been marked within storage May return false negatives, e.g. the object could be marked in the store but this Ref instance still has a status of "UNKNOWN".
      Returns:
      true if this Ref has a status of MARKED or above, false otherwise

    • persist

      public <R extends ACell> Ref<R> persist(Consumer<Ref<ACell>> noveltyHandler)
      Persists this Ref in the current store if not embedded and not already persisted. This may convert the Ref from a direct reference to a soft reference. If the persisted Ref represents novelty, will trigger the specified novelty handler
      Parameters:
      noveltyHandler - Novelty handler to call (may be null)
      Returns:
      the persisted Ref
      Throws:
      MissingDataException - If the Ref's value does not exist or has been garbage collected before being persisted

    • persist

      public <R extends ACell> Ref<R> persist()
      Persists this Ref in the current store if not embedded and not already persisted. Resulting status will be PERSISTED or higher. This may convert the Ref from a direct reference to a soft reference.
      Returns:
      the persisted Ref
      Throws:
      MissingDataException - if the Ref cannot be fully persisted.

    • accumulateRefSet

      public static Set<Ref<?>> accumulateRefSet(Object a)
      Accumulates the set of all unique Refs in the given object. Might stack overflow if nesting is too deep - not for use in on-chain code.
      Parameters:
      a - Ref or Cell
      Returns:
      Set containing all unique refs (accoumulated recursively) within the given object

    • updateRefs

      public static <T extends ACell> Ref<T>[] updateRefs(Ref<T>[] refs, IRefFunction func)
      Updates an array of Refs with the given function. Returns the original array unchanged if no refs were changed, otherwise returns a new array.
      Parameters:
      refs - Array of Refs to update
      func - Ref update function
      Returns:
      Array of updated Refs

    • createArray

      public static <T extends ACell> Ref<T>[] createArray(T[] values)

    • addAllToSet

      public ASet<ACell> addAllToSet(ASet<ACell> store)
      Adds the value of this Ref and all non-embedded child values to a given set. Logically, provides the guarantee that the set will contain all cells needed to recreate the complete value of this Ref.
      Parameters:
      store - Store to add to
      Returns:
      Set containing this Ref and all direct or indirect child refs

    • isEmbedded

      public final boolean isEmbedded()
      Check if the Ref's value is embedded. If false, the value must be an ACell instance.
      Returns:
      true if embedded, false otherwise

    • toDirect

      public Ref<T> toDirect()
      Converts this Ref to a RefDirect
      Returns:
      Direct Ref

    • persistShallow

      public <R extends ACell> Ref<R> persistShallow()
      Persists a Ref shallowly in the current store. Status will be updated to STORED or higher.
      Returns:
      Ref with status of STORED or above

    • persistShallow

      public <R extends ACell> Ref<R> persistShallow(Consumer<Ref<ACell>> noveltyHandler)
      Persists a Ref shallowly in the current store. Status will be updated STORED or higher. Novelty handler will be called exactly once if and only if the ref was not previously stored
      Parameters:
      noveltyHandler - Novelty handler to call (may be null)
      Returns:
      Ref with status of STORED or above

    • withValue

      public abstract Ref<T> withValue(T newValue)
      Updates the value stored within this Ref. New value must be equal in value to the old value (identical hash), but may have updated internal refs etc.
      Parameters:
      newValue - New value
      Returns:
      Updated Ref

    • encode

      public final int encode(byte[] bs, int pos)
      Writes the ref to a byte array. Embeds embedded values as necessary.
      Specified by:
      encode in interface IWriteable
      Parameters:
      bs - Byte array to encode to
      pos - position at which to write the value
      Returns:
      Updated position

    • write

      public final ByteBuffer write(ByteBuffer bb)
      Description copied from interface: IWriteable
      Writes this object to a ByteBuffer including an appropriate message tag
      Specified by:
      write in interface IWriteable
      Parameters:
      bb - ByteBuffer to write to
      Returns:
      The updated ByteBuffer

    • createEncoding

      protected Blob createEncoding()
      Description copied from class: AObject
      Creates a Blob object representing this object. Should be called only after the cached encoding has been checked.
      Specified by:
      createEncoding in class AObject
      Returns:
      Blob Encoding of Object

    • getEncodingLength

      public final long getEncodingLength()
      Gets the encoding length for writing this Ref. Will be equal to the encoding length of the Ref's value if embedded, otherwise INDIRECT_ENCODING_LENGTH
      Returns:
      Exact length of encoding

    • getMemorySize

      public long getMemorySize()
      Gets the indirect memory size for this Ref
      Returns:
      0 for fully embedded values with no child refs, memory size of referred value otherwise

    • findMissing

      public void findMissing(HashSet<Hash> missingSet)
      Finds all instances of missing data in this Ref, and adds them to the missing set
      Parameters:
      missingSet - Set to add missing instances to

    • isMissing

      public abstract boolean isMissing()
      Checks if this Ref refers to missing data, i.e. a Cell that does not exist in the currect store.
      Returns:
      true if this specific Ref has missing data, false otherwise.

    • mergeFlags

      public static int mergeFlags(int a, int b)
      Merges flags in an idempotent way. Assume flags are valid
      Parameters:
      a - First set of flags
      b - Second set of flags
      Returns:
      Merged flags

    • ensureCanonical

      public abstract Ref<T> ensureCanonical()
      Ensures this Ref is canonical
      Returns:
      this Ref if already canonical, potentially a new Ref with canonical value otherwise