io
Throw to indicate that bitOrder changed, but not on a byte boundary.
Simple realization of the DataInputStream API
Simple realization of the DataInputStream API
Underlying representation is just a ByteBuffer containing all input data.
In many cases this can be a direct byte buffer, and so doesn't occupy storage in the java heap.
At top level (when creating one for all the data), the supplied byte buffer must have position 0 and its limit should be equal to its capacity.
The backward compatibility layers, however, until removed, make copies of these objects, and those may have non-zero positions. E.g., see the def makeACopy.
Hex/Bits and text dump formats for debug/trace purposes.
Hex/Bits and text dump formats for debug/trace purposes.
By definition this is a dump, so doesn't know much about where the fields in the data are. (To do that you'd need a format description language, like DFDL, but this is here to help debug DFDL descriptions, so it really cannot exploit any information about the data format)
There is an asymmetry between DataInputStream and DataOutputStream with respect to the positions and limits in the bit stream.
There is an asymmetry between DataInputStream and DataOutputStream with respect to the positions and limits in the bit stream.
For the DataInputStream, we have this concept of the current bitPos0b, and optionally there may be abound called bitLimit0b. There are 1b variants of these.
For parsing, these are always absolute values, that is they contain bit position relative the ultimate start of the input stream where parsing began.
For DataOutputStream, we have slightly different concepts.
There are absolute and relative variants. The absolute bitPosOb or absBitPos0b is symmetric to the parser's bitPos0b. It's the position relative to the ultimate start of the output stream.
However, we often do not know this value. So the UState and DataOutputStream have a maybeAbsBitPos0b which can be MaybeULong.Nope if the value isn't known.
In addition we have the relative or relBitPos0b. This is relative to the start of whatever buffer we are doing unparsing into.
When unparsing, we often have to unparse into a buffer where the ultimate actual absolute position isn't yet known, but we have to do the unparsing anyway, for example so that we can measure exactly how long something is.
Conversely, sometimes we simply must have the absolute output bit position, for example, when computing the number of bits to insert to achieve the required alignment.
Hence we have relBitPos0b - always known and is a value >= 0, and we have maybeAbsBitPos0b which is a MaybeULong. If known it is >=0.
Corresponding to bit position we have bit limit, which is measured in the same 0b or 1b units, but is *always* a maybe type, because even in the case where we know the absolute position, we still may or may not have any limit in place. Hence the UState and DataOutputStream have a
maybeRelBitLimit0b
and
maybeAbsBitLimit0b.
One invariant is this: when the absolute bit pos is known, then it is the same as the relative bit pos. Similarly when the absolute bit limit is known, then the relative bit limit is known and is equal.
This is an interface trait, and it defines methods shared by both DataInputStream and DataOutputStream.
This is an interface trait, and it defines methods shared by both DataInputStream and DataOutputStream.
Implementation (partial) is in DataStreamCommonImplMixin.
Shared by both DataInputStream and DataOutputStream implementations
To support dfdl:outputValueCalc, we must suspend output.
To support dfdl:outputValueCalc, we must suspend output. This is done by taking the current "direct" output, and splitting it into a still direct part, and a following buffered output.
The direct part waits for the OVC calculation to complete, when that is written, it is finished and collapses into the following, which was buffered, but becomes direct as a result of this collapsing.
Hence, most output will be to direct data output streams, with some, while an OVC is pending, will be buffered, but this is eliminated as soon as possible.
A Buffered DOS can be finished or not. Not finished means that it might still be appended to. Not concurrently, but by other code invoked from this thread of control (which might traverse different co-routine "stack" threads, but it's still one thread of control).
Finished means that the Buffered DOS can never be appended to again.
Has two modes of operation, buffering or direct. When buffering, all output goes into a buffer. When direct, all output goes into a "real" DataOutputStream.
Abstract interface to obtain format properties or values derived from properties.
Abstract interface to obtain format properties or values derived from properties.
This includes anything the I/O layer needs, which includes properties that can be runtime-valued expressions, or that depend on such.
By passing in an object that provides quick access to these, we avoid the need to have setters/getters that call setters that change state in the I/O layer.
Warning: Only mix this into thread-local state objects.
Warning: Only mix this into thread-local state objects. If mixed into a regular class this will end up sharing the local stack object across threads, which is a very bad idea (not thread safe).
The state that must be saved and restored by mark/reset calls
When unparsing, we reuse all the DFA logic to identify delimiters within the data that need to be escaped, so we need to treat the string data being unparsed as a DataInputStream.
Mixin to classes that are supposed to exist 1 to 1 with threads.
Mixin to classes that are supposed to exist 1 to 1 with threads. Such as DataInputStream derived classes and DataOutputStream derived classes.
Factory for creating this type of DataInputStream
Factory for creating this type of DataInputStream
Examines channels and input streams to see if they are associated to files. If so it memory maps the file.
This trait defines the low level API called by Daffodil's Parsers.
This trait defines the low level API called by Daffodil's Parsers.
It has features to support
A goal is that this API does not allocate objects as I/O operations are performed unless boxed objects are being returned. For example getSignedLong(...) should not allocate anything per call; however, getSignedBigInt(...) does, because a BigInt is a heap-allocated object.
Internal buffers and such may be dropped/resized/reallocated as needed during method calls. The point is not that you never allocate. It's that the per-I/O operation overhead does not require object allocation for every data-accessing method call.
Similarly, text data can be retrieved into a char buffer, and the char buffer can provide a limit on size (available capacity of the char buffer) in characters. The text can be examined in the char buffer, or a string can be created from the char buffer's contents when needed.
This API is very stateful, and not-thread-safe i.e., each thread must have its own object. Some of this is inherent in this API style, and some is inherited from the underlying objects this API uses (such as CharsetDecoder).
This API is also intended to support some very highly optimized implementations. For example, if the schemas is all text, and the encoding is known to be iso-8859-1, then there is no notion of a decode error, and every byte value, extended to a Char value, *is* the Unicode codepoint. No decoder needs to be used in this case and this API becomes a quite thin layer on top of a java.io.BufferedStream.
Terminology:
Available Data - this is the data that is between the current bit position, and some limit. The limit can either be set (via setBitLimit calls), or it can be limited by tunable values, or implementation-specific upper limits, or it can simply be the end of the data stream.
Different kinds of DataInputStreams can have different limits. For example, a File-based DataInputStream may have no limit on the forward speculation distance, because the file can be randomly accessed if necessary. Contrasting that with a data stream that is directly connected to a network socket may have a upper limit on the amount of data that it is willing to buffer.
None of this is a commitment that this API will in fact have multiple specialized implementations. It's just a possibility for the future.
Implementation Note: It is the implementation of this interface which implements the Bucket Algorithm as described on the Daffodil Wiki. All of that bucket stuff is beneath this API.
In general, this API tries to return a value rather than throw exceptions whenever the behavior may be very common. This leaves it up to the caller to decide whether or not to throw an exception, and avoids the overhead of try-catch blocks. The exception to this rule are the methods that involve character decoding for textual data. These methods may throw CharacterCodingException when the encoding error policy is 'error'.
Highly optimized converter for Ascii to Unicode
Throw to indicate that bitOrder changed, but not on a byte boundary.
Must be caught at higher level and turned into a RuntimeSDE where we have the context to do so.
All calls to setFinished should, somewhere, be surrounded by a catch of this.