Class ByteString

  • All Implemented Interfaces:
    java.io.Serializable, java.lang.Iterable<java.lang.Byte>

    public abstract class ByteString
    extends java.lang.Object
    implements java.lang.Iterable<java.lang.Byte>, java.io.Serializable
    Immutable sequence of bytes. Provides conversions to and from byte[], String, ByteBuffer, InputStream, OutputStream. Also provides a conversion to CodedInputStream.

    Like String, the contents of a ByteString can never be observed to change, not even in the presence of a data race or incorrect API usage in the client code.

    Substring is supported by sharing the reference to the immutable underlying bytes. Concatenation is likewise supported without copying (long strings) by building a tree of pieces in RopeByteString.

    Author:
    [email protected] Bob Lee, [email protected] Kenton Varda, [email protected] Carl Haverl, [email protected] Martin Buchholz
    See Also:
    Serialized Form
    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description
      static interface  ByteString.ByteIterator
      This interface extends Iterator<Byte>, so that we can return an unboxed byte.
      static class  ByteString.Output
      Outputs to a ByteString instance.
    • Field Summary

      Fields 
      Modifier and Type Field Description
      static ByteString EMPTY
      Empty ByteString.
    • Method Summary

      All Methods Static Methods Instance Methods Abstract Methods Concrete Methods Deprecated Methods 
      Modifier and Type Method Description
      abstract java.nio.ByteBuffer asReadOnlyByteBuffer()
      Constructs a read-only java.nio.ByteBuffer whose content is equal to the contents of this byte string.
      abstract java.util.List<java.nio.ByteBuffer> asReadOnlyByteBufferList()
      Constructs a list of read-only java.nio.ByteBuffer objects such that the concatenation of their contents is equal to the contents of this byte string.
      abstract byte byteAt​(int index)
      Gets the byte at the given index.
      ByteString concat​(ByteString other)
      Concatenate the given ByteString to this one.
      static ByteString copyFrom​(byte[] bytes)
      Copies the given bytes into a ByteString.
      static ByteString copyFrom​(byte[] bytes, int offset, int size)
      Copies the given bytes into a ByteString.
      static ByteString copyFrom​(java.lang.Iterable<ByteString> byteStrings)
      Concatenates all byte strings in the iterable and returns the result.
      static ByteString copyFrom​(java.lang.String text, java.lang.String charsetName)
      Encodes text into a sequence of bytes using the named charset and returns the result as a ByteString.
      static ByteString copyFrom​(java.lang.String text, java.nio.charset.Charset charset)
      Encodes text into a sequence of bytes using the named charset and returns the result as a ByteString.
      static ByteString copyFrom​(java.nio.ByteBuffer bytes)
      Copies the remaining bytes from a java.nio.ByteBuffer into a ByteString.
      static ByteString copyFrom​(java.nio.ByteBuffer bytes, int size)
      Copies the next size bytes from a java.nio.ByteBuffer into a ByteString.
      static ByteString copyFromUtf8​(java.lang.String text)
      Encodes text into a sequence of UTF-8 bytes and returns the result as a ByteString.
      void copyTo​(byte[] target, int offset)
      Copies bytes into a buffer at the given offset.
      void copyTo​(byte[] target, int sourceOffset, int targetOffset, int numberToCopy)
      Deprecated.
      Instead, call byteString.substring(sourceOffset, sourceOffset + numberToCopy).copyTo(target, targetOffset)
      abstract void copyTo​(java.nio.ByteBuffer target)
      Copies bytes into a ByteBuffer.
      protected abstract void copyToInternal​(byte[] target, int sourceOffset, int targetOffset, int numberToCopy)
      Internal (package private) implementation of copyTo(byte[],int,int,int).
      boolean endsWith​(ByteString suffix)
      Tests if this bytestring ends with the specified suffix.
      abstract boolean equals​(java.lang.Object o)  
      protected abstract int getTreeDepth()
      Return the depth of the tree representing this ByteString, if any, whose root is this node.
      int hashCode()
      Compute the hashCode using the traditional algorithm from ByteString.
      protected abstract boolean isBalanced()
      Return true if this ByteString is literal (a leaf node) or a flat-enough tree in the sense of RopeByteString.
      boolean isEmpty()
      Returns true if the size is 0, false otherwise.
      abstract boolean isValidUtf8()
      Tells whether this ByteString represents a well-formed UTF-8 byte sequence, such that the original bytes can be converted to a String object and then round tripped back to bytes without loss.
      ByteString.ByteIterator iterator()
      Return a ByteString.ByteIterator over the bytes in the ByteString.
      abstract CodedInputStream newCodedInput()
      Creates a CodedInputStream which can be used to read the bytes.
      abstract java.io.InputStream newInput()
      Creates an InputStream which can be used to read the bytes.
      static ByteString.Output newOutput()
      Creates a new ByteString.Output.
      static ByteString.Output newOutput​(int initialCapacity)
      Creates a new ByteString.Output with the given initial capacity.
      protected abstract int partialHash​(int h, int offset, int length)
      Compute the hash across the value bytes starting with the given hash, and return the result.
      protected abstract int partialIsValidUtf8​(int state, int offset, int length)
      Tells whether the given byte sequence is a well-formed, malformed, or incomplete UTF-8 byte sequence.
      protected int peekCachedHashCode()
      Return the cached hash code if available.
      static ByteString readFrom​(java.io.InputStream streamToDrain)
      Completely reads the given stream's bytes into a ByteString, blocking if necessary until all bytes are read through to the end of the stream.
      static ByteString readFrom​(java.io.InputStream streamToDrain, int chunkSize)
      Completely reads the given stream's bytes into a ByteString, blocking if necessary until all bytes are read through to the end of the stream.
      static ByteString readFrom​(java.io.InputStream streamToDrain, int minChunkSize, int maxChunkSize)  
      abstract int size()
      Gets the number of bytes.
      boolean startsWith​(ByteString prefix)
      Tests if this bytestring starts with the specified prefix.
      ByteString substring​(int beginIndex)
      Return the substring from beginIndex, inclusive, to the end of the string.
      abstract ByteString substring​(int beginIndex, int endIndex)
      Return the substring from beginIndex, inclusive, to endIndex, exclusive.
      byte[] toByteArray()
      Copies bytes to a byte[].
      java.lang.String toString()  
      java.lang.String toString​(java.lang.String charsetName)
      Constructs a new String by decoding the bytes using the specified charset.
      java.lang.String toString​(java.nio.charset.Charset charset)
      Constructs a new String by decoding the bytes using the specified charset.
      protected abstract java.lang.String toStringInternal​(java.nio.charset.Charset charset)
      Constructs a new String by decoding the bytes using the specified charset.
      java.lang.String toStringUtf8()
      Constructs a new String by decoding the bytes as UTF-8.
      static java.util.Comparator<ByteString> unsignedLexicographicalComparator()
      Returns a Comparator which compares ByteString-s lexicographically as sequences of unsigned bytes (i.e.
      abstract void writeTo​(java.io.OutputStream out)
      Writes a copy of the contents of this byte string to the specified output stream argument.
      • Methods inherited from class java.lang.Object

        clone, finalize, getClass, notify, notifyAll, wait, wait, wait
      • Methods inherited from interface java.lang.Iterable

        forEach, spliterator
    • Field Detail

      • EMPTY

        public static final ByteString EMPTY
        Empty ByteString.
    • Method Detail

      • byteAt

        public abstract byte byteAt​(int index)
        Gets the byte at the given index. This method should be used only for random access to individual bytes. To access bytes sequentially, use the ByteString.ByteIterator returned by iterator(), and call substring(int, int) first if necessary.
        Parameters:
        index - index of byte
        Returns:
        the value
        Throws:
        java.lang.IndexOutOfBoundsException - index < 0 or index >= size
      • size

        public abstract int size()
        Gets the number of bytes.
        Returns:
        size in bytes
      • isEmpty

        public final boolean isEmpty()
        Returns true if the size is 0, false otherwise.
        Returns:
        true if this is zero bytes long
      • unsignedLexicographicalComparator

        public static java.util.Comparator<ByteString> unsignedLexicographicalComparator()
        Returns a Comparator which compares ByteString-s lexicographically as sequences of unsigned bytes (i.e. values between 0 and 255, inclusive).

        For example, (byte) -1 is considered to be greater than (byte) 1 because it is interpreted as an unsigned value, 255:

        • `-1` -> 0b11111111 (two's complement) -> 255
        • `1` -> 0b00000001 -> 1
      • substring

        public final ByteString substring​(int beginIndex)
        Return the substring from beginIndex, inclusive, to the end of the string.
        Parameters:
        beginIndex - start at this index
        Returns:
        substring sharing underlying data
        Throws:
        java.lang.IndexOutOfBoundsException - if beginIndex < 0 or beginIndex > size().
      • substring

        public abstract ByteString substring​(int beginIndex,
                                             int endIndex)
        Return the substring from beginIndex, inclusive, to endIndex, exclusive.
        Parameters:
        beginIndex - start at this index
        endIndex - the last character is the one before this index
        Returns:
        substring sharing underlying data
        Throws:
        java.lang.IndexOutOfBoundsException - if beginIndex < 0, endIndex > size(), or beginIndex > endIndex.
      • startsWith

        public final boolean startsWith​(ByteString prefix)
        Tests if this bytestring starts with the specified prefix. Similar to String.startsWith(String)
        Parameters:
        prefix - the prefix.
        Returns:
        true if the byte sequence represented by the argument is a prefix of the byte sequence represented by this string; false otherwise.
      • endsWith

        public final boolean endsWith​(ByteString suffix)
        Tests if this bytestring ends with the specified suffix. Similar to String.endsWith(String)
        Parameters:
        suffix - the suffix.
        Returns:
        true if the byte sequence represented by the argument is a suffix of the byte sequence represented by this string; false otherwise.
      • copyFrom

        public static ByteString copyFrom​(byte[] bytes,
                                          int offset,
                                          int size)
        Copies the given bytes into a ByteString.
        Parameters:
        bytes - source array
        offset - offset in source array
        size - number of bytes to copy
        Returns:
        new ByteString
        Throws:
        java.lang.IndexOutOfBoundsException - if offset or size are out of bounds
      • copyFrom

        public static ByteString copyFrom​(byte[] bytes)
        Copies the given bytes into a ByteString.
        Parameters:
        bytes - to copy
        Returns:
        new ByteString
      • copyFrom

        public static ByteString copyFrom​(java.nio.ByteBuffer bytes,
                                          int size)
        Copies the next size bytes from a java.nio.ByteBuffer into a ByteString.
        Parameters:
        bytes - source buffer
        size - number of bytes to copy
        Returns:
        new ByteString
        Throws:
        java.lang.IndexOutOfBoundsException - if size > bytes.remaining()
      • copyFrom

        public static ByteString copyFrom​(java.nio.ByteBuffer bytes)
        Copies the remaining bytes from a java.nio.ByteBuffer into a ByteString.
        Parameters:
        bytes - sourceBuffer
        Returns:
        new ByteString
      • copyFrom

        public static ByteString copyFrom​(java.lang.String text,
                                          java.lang.String charsetName)
                                   throws java.io.UnsupportedEncodingException
        Encodes text into a sequence of bytes using the named charset and returns the result as a ByteString.
        Parameters:
        text - source string
        charsetName - encoding to use
        Returns:
        new ByteString
        Throws:
        java.io.UnsupportedEncodingException - if the encoding isn't found
      • copyFrom

        public static ByteString copyFrom​(java.lang.String text,
                                          java.nio.charset.Charset charset)
        Encodes text into a sequence of bytes using the named charset and returns the result as a ByteString.
        Parameters:
        text - source string
        charset - encode using this charset
        Returns:
        new ByteString
      • copyFromUtf8

        public static ByteString copyFromUtf8​(java.lang.String text)
        Encodes text into a sequence of UTF-8 bytes and returns the result as a ByteString.
        Parameters:
        text - source string
        Returns:
        new ByteString
      • readFrom

        public static ByteString readFrom​(java.io.InputStream streamToDrain)
                                   throws java.io.IOException
        Completely reads the given stream's bytes into a ByteString, blocking if necessary until all bytes are read through to the end of the stream.

        Performance notes: The returned ByteString is an immutable tree of byte arrays ("chunks") of the stream data. The first chunk is small, with subsequent chunks each being double the size, up to 8K.

        Each byte read from the input stream will be copied twice to ensure that the resulting ByteString is truly immutable.

        Parameters:
        streamToDrain - The source stream, which is read completely but not closed.
        Returns:
        A new ByteString which is made up of chunks of various sizes, depending on the behavior of the underlying stream.
        Throws:
        java.io.IOException - IOException is thrown if there is a problem reading the underlying stream.
      • readFrom

        public static ByteString readFrom​(java.io.InputStream streamToDrain,
                                          int chunkSize)
                                   throws java.io.IOException
        Completely reads the given stream's bytes into a ByteString, blocking if necessary until all bytes are read through to the end of the stream.

        Performance notes: The returned ByteString is an immutable tree of byte arrays ("chunks") of the stream data. The chunkSize parameter sets the size of these byte arrays.

        Each byte read from the input stream will be copied twice to ensure that the resulting ByteString is truly immutable.

        Parameters:
        streamToDrain - The source stream, which is read completely but not closed.
        chunkSize - The size of the chunks in which to read the stream.
        Returns:
        A new ByteString which is made up of chunks of the given size.
        Throws:
        java.io.IOException - IOException is thrown if there is a problem reading the underlying stream.
      • readFrom

        public static ByteString readFrom​(java.io.InputStream streamToDrain,
                                          int minChunkSize,
                                          int maxChunkSize)
                                   throws java.io.IOException
        Throws:
        java.io.IOException
      • concat

        public final ByteString concat​(ByteString other)
        Concatenate the given ByteString to this one. Short concatenations, of total size smaller than CONCATENATE_BY_COPY_SIZE, are produced by copying the underlying bytes (as per Rope.java, BAP95 . In general, the concatenate involves no copying.
        Parameters:
        other - string to concatenate
        Returns:
        a new ByteString instance
      • copyFrom

        public static ByteString copyFrom​(java.lang.Iterable<ByteString> byteStrings)
        Concatenates all byte strings in the iterable and returns the result. This is designed to run in O(list size), not O(total bytes).

        The returned ByteString is not necessarily a unique object. If the list is empty, the returned object is the singleton empty ByteString. If the list has only one element, that ByteString will be returned without copying.

        Parameters:
        byteStrings - strings to be concatenated
        Returns:
        new ByteString
      • copyTo

        public void copyTo​(byte[] target,
                           int offset)
        Copies bytes into a buffer at the given offset.

        To copy a subset of bytes, you call this method on the return value of substring(int, int). Example: byteString.substring(start, end).copyTo(target, offset)

        Parameters:
        target - buffer to copy into
        offset - in the target buffer
        Throws:
        java.lang.IndexOutOfBoundsException - if the offset is negative or too large
      • copyTo

        @Deprecated
        public final void copyTo​(byte[] target,
                                 int sourceOffset,
                                 int targetOffset,
                                 int numberToCopy)
        Deprecated.
        Instead, call byteString.substring(sourceOffset, sourceOffset + numberToCopy).copyTo(target, targetOffset)
        Copies bytes into a buffer.
        Parameters:
        target - buffer to copy into
        sourceOffset - offset within these bytes
        targetOffset - offset within the target buffer
        numberToCopy - number of bytes to copy
        Throws:
        java.lang.IndexOutOfBoundsException - if an offset or size is negative or too large
      • copyToInternal

        protected abstract void copyToInternal​(byte[] target,
                                               int sourceOffset,
                                               int targetOffset,
                                               int numberToCopy)
        Internal (package private) implementation of copyTo(byte[],int,int,int). It assumes that all error checking has already been performed and that numberToCopy > 0.
      • copyTo

        public abstract void copyTo​(java.nio.ByteBuffer target)
        Copies bytes into a ByteBuffer.

        To copy a subset of bytes, you call this method on the return value of substring(int, int). Example: byteString.substring(start, end).copyTo(target)

        Parameters:
        target - ByteBuffer to copy into.
        Throws:
        java.nio.ReadOnlyBufferException - if the target is read-only
        java.nio.BufferOverflowException - if the target's remaining() space is not large enough to hold the data.
      • toByteArray

        public final byte[] toByteArray()
        Copies bytes to a byte[].
        Returns:
        copied bytes
      • writeTo

        public abstract void writeTo​(java.io.OutputStream out)
                              throws java.io.IOException
        Writes a copy of the contents of this byte string to the specified output stream argument.
        Parameters:
        out - the output stream to which to write the data.
        Throws:
        java.io.IOException - if an I/O error occurs.
      • asReadOnlyByteBuffer

        public abstract java.nio.ByteBuffer asReadOnlyByteBuffer()
        Constructs a read-only java.nio.ByteBuffer whose content is equal to the contents of this byte string. The result uses the same backing array as the byte string, if possible.
        Returns:
        wrapped bytes
      • asReadOnlyByteBufferList

        public abstract java.util.List<java.nio.ByteBuffer> asReadOnlyByteBufferList()
        Constructs a list of read-only java.nio.ByteBuffer objects such that the concatenation of their contents is equal to the contents of this byte string. The result uses the same backing arrays as the byte string.

        By returning a list, implementations of this method may be able to avoid copying even when there are multiple backing arrays.

        Returns:
        a list of wrapped bytes
      • toString

        public final java.lang.String toString​(java.lang.String charsetName)
                                        throws java.io.UnsupportedEncodingException
        Constructs a new String by decoding the bytes using the specified charset.
        Parameters:
        charsetName - encode using this charset
        Returns:
        new string
        Throws:
        java.io.UnsupportedEncodingException - if charset isn't recognized
      • toString

        public final java.lang.String toString​(java.nio.charset.Charset charset)
        Constructs a new String by decoding the bytes using the specified charset. Returns the same empty String if empty.
        Parameters:
        charset - encode using this charset
        Returns:
        new string
      • toStringInternal

        protected abstract java.lang.String toStringInternal​(java.nio.charset.Charset charset)
        Constructs a new String by decoding the bytes using the specified charset.
        Parameters:
        charset - encode using this charset
        Returns:
        new string
      • toStringUtf8

        public final java.lang.String toStringUtf8()
        Constructs a new String by decoding the bytes as UTF-8.
        Returns:
        new string using UTF-8 encoding
      • isValidUtf8

        public abstract boolean isValidUtf8()
        Tells whether this ByteString represents a well-formed UTF-8 byte sequence, such that the original bytes can be converted to a String object and then round tripped back to bytes without loss.

        More precisely, returns true whenever:

        
         Arrays.equals(byteString.toByteArray(),
             new String(byteString.toByteArray(), "UTF-8").getBytes("UTF-8"))
         

        This method returns false for "overlong" byte sequences, as well as for 3-byte sequences that would map to a surrogate character, in accordance with the restricted definition of UTF-8 introduced in Unicode 3.1. Note that the UTF-8 decoder included in Oracle's JDK has been modified to also reject "overlong" byte sequences, but (as of 2011) still accepts 3-byte surrogate character byte sequences.

        See the Unicode Standard,
        Table 3-6. UTF-8 Bit Distribution,
        Table 3-7. Well Formed UTF-8 Byte Sequences.

        Returns:
        whether the bytes in this ByteString are a well-formed UTF-8 byte sequence
      • partialIsValidUtf8

        protected abstract int partialIsValidUtf8​(int state,
                                                  int offset,
                                                  int length)
        Tells whether the given byte sequence is a well-formed, malformed, or incomplete UTF-8 byte sequence. This method accepts and returns a partial state result, allowing the bytes for a complete UTF-8 byte sequence to be composed from multiple ByteString segments.
        Parameters:
        state - either 0 (if this is the initial decoding operation) or the value returned from a call to a partial decoding method for the previous bytes
        offset - offset of the first byte to check
        length - number of bytes to check
        Returns:
        -1 if the partial byte sequence is definitely malformed, 0 if it is well-formed (no additional input needed), or, if the byte sequence is "incomplete", i.e. apparently terminated in the middle of a character, an opaque integer "state" value containing enough information to decode the character when passed to a subsequent invocation of a partial decoding method.
      • equals

        public abstract boolean equals​(java.lang.Object o)
        Overrides:
        equals in class java.lang.Object
      • hashCode

        public final int hashCode()
        Compute the hashCode using the traditional algorithm from ByteString.
        Overrides:
        hashCode in class java.lang.Object
        Returns:
        hashCode value
      • newInput

        public abstract java.io.InputStream newInput()
        Creates an InputStream which can be used to read the bytes.

        The InputStream returned by this method is guaranteed to be completely non-blocking. The method InputStream.available() returns the number of bytes remaining in the stream. The methods InputStream.read(byte[]), InputStream.read(byte[],int,int) and InputStream.skip(long) will read/skip as many bytes as are available. The method InputStream.markSupported() returns true.

        The methods in the returned InputStream might not be thread safe.

        Returns:
        an input stream that returns the bytes of this byte string.
      • newOutput

        public static ByteString.Output newOutput​(int initialCapacity)
        Creates a new ByteString.Output with the given initial capacity. Call ByteString.Output.toByteString() to create the ByteString instance.

        A ByteString.Output offers the same functionality as a ByteArrayOutputStream, except that it returns a ByteString rather than a byte array.

        Parameters:
        initialCapacity - estimate of number of bytes to be written
        Returns:
        OutputStream for building a ByteString
      • getTreeDepth

        protected abstract int getTreeDepth()
        Return the depth of the tree representing this ByteString, if any, whose root is this node. If this is a leaf node, return 0.
        Returns:
        tree depth or zero
      • isBalanced

        protected abstract boolean isBalanced()
        Return true if this ByteString is literal (a leaf node) or a flat-enough tree in the sense of RopeByteString.
        Returns:
        true if the tree is flat enough
      • peekCachedHashCode

        protected final int peekCachedHashCode()
        Return the cached hash code if available.
        Returns:
        value of cached hash code or 0 if not computed yet
      • partialHash

        protected abstract int partialHash​(int h,
                                           int offset,
                                           int length)
        Compute the hash across the value bytes starting with the given hash, and return the result. This is used to compute the hash across strings represented as a set of pieces by allowing the hash computation to be continued from piece to piece.
        Parameters:
        h - starting hash value
        offset - offset into this value to start looking at data values
        length - number of data values to include in the hash computation
        Returns:
        ending hash value
      • toString

        public final java.lang.String toString()
        Overrides:
        toString in class java.lang.Object