processors
Runtime valued properties that are enums would all work like ByteOrder here.
A combinator is a processor that orchestrates the operation of other processors.
State used when compiling Evaluatable[T] objects So they don't require a "real" state.
State used when compiling Evaluatable[T] objects So they don't require a "real" state.
This serves two purposes. First it lets us obey the regular API for evaluation, so we don't need one way to evaluate and another very similar thing for analyzing expressions to see if they are constant.
Second, it serves as a detector of when an expression is non-constant by blowing up when things inconsistent with constant-value are attempted to be extracted from the state. By "blow up" it throws a structured set of exceptions, typically children of InfosetException or VariableException.
The very last aspects of compilation, and the start of the back-end runtime.
Takes a compiled expression and a conversion, and if the expression is a constant, runs the conversion once saving the converted result.
Takes a compiled expression and a conversion, and if the expression is a constant, runs the conversion once saving the converted result. This saves work if this happens at compilation time, not runtime.
If the expression is dynamic, then it runs the expression then the conversion once, and saves the result.
Note that the "only once" aspect of this isn't terribly important, as one would not expect parse/unparse operations to go back and ask for these expressions repeatedly. They should be keeping it in a local val.
Encoding is a string, so there is no converter.
This is the object we serialize.
This is the object we serialize.
At compile time we will create an encodingInfo for ourselves supplying as context a schema component.
At runtime we will create an encodingInfo supplying as context a TermRuntimeData object.
Rv is a "runtime value" as in, it is *always* evaluated, never cached, never even considered as a possibility for a constant.
Evaluatable - things that could be runtime-valued, but also could be compile-time constants are instances of Ev.
Use for expressions that produce results which have to subsequently be converted into say, an enum value (so that "bigEndian" becomes ByteOrder.BigEndian) or that have to be interpreted as DFDL string literals * of various kinds.
Use for expressions that produce results which have to subsequently be converted into say, an enum value (so that "bigEndian" becomes ByteOrder.BigEndian) or that have to be interpreted as DFDL string literals * of various kinds.
See the cookers for string literals such as TextStandardInfinityRepCooker.
Use for expressions when what you want out really is just the string value of the property.
Definitions that are the same whether we're in the schema compiler or runtime are on this trait.
Provides the length of the element, in bits, for Explicit or Implicit length elements, including deals with LengthUnits.Characters and fixed width encodings.
Provides the length of the element, in bits, for Explicit or Implicit length elements, including deals with LengthUnits.Characters and fixed width encodings.
Does NOT take into account minLength and maxLength or textOutputMinLength for unparsing of Explicit length elements. See ElementTargetLengthInBitsEv.
Used when lengthKind='explicit', computes the length in bits or Nope if the length depends on encoding, and the encoding is variable width.
Used when lengthKind='explicit', computes the length in bits or Nope if the length depends on encoding, and the encoding is variable width.
Handles the case of LengthUnits.Characters by returning Nope if the charset is variable width. Otherwise it computes the number of bits using the width.
Nope does NOT mean that we cannot yet compute the length expression. It means the length depends on variable-width character encoding (and since encoding is runtime-valued sometimes, we need an Ev for this.
Sometimes a length cannot be computed yet - happens during unparsing, for an element whose length expression refers back to an element with a dfdl:outputValueCalc that has not yet been computed, or a variable that has not yet been computed.
In that case, the access to the infoset throws particular exceptions descended from the RetryableException trait.
This is a method of caching that requires the parsers to create new cache slots, evaluate, then invalidate the cache slots when it goes out of scope.
This is a method of caching that requires the parsers to create new cache slots, evaluate, then invalidate the cache slots when it goes out of scope. This differs from the InfosetCachedEvaluatable, which caches values in the infoset and relies on backtracking and removal of infoset nodes to invalidate the cache.
The Evaluatable implementing this must have an MStack stored in the PState or UState, and getCacheStack will be used to retrieve that stack. Before calling evaluate for the first time, the parser must call newCache. All other calls to evaluate will use this evaluated value. When the evaluatable goes out of scope, the parser must then invalidate this cache by calling invalidateCace.
The motivating use case for this is for DFAFieldEV and EscapeEschemeEV. DFAField is dependent on the escape scheme. The logical way to handle is to make the EscapeSchemeEv a runtime dependent on the DFAFieldEv. This is convenient since if the EscapeSchemeEv is constant, we can calcuclate the DFAFieldEv at compile time.
However, say the EscapeSchemeEv is not constant. The problem is that it needs to be evaluated in a different scope than when the DFAFieldEv is evaluated. So this manual caching thing is a way to force evaluation of the Ev at the appropriate time (via the DynamicEscapeSchemeParser), and cache it in the PState. Then when the DFAField is evaluated it can use that cached value. Note that the problem with storing the cached value in the infoset (ala InfosetCachedEvaluatable) is that the current infoset element may have changed in between the time the EscapeSchemeEv was evaluated and the time the DFAField is evaluated. So if we cached on the infoset, when the DFAField asks for the EscapeSchemeEv, it will look at the currentInfoset cache and it wouldn't be there, and then evaluate the escape scheme again in the wrong scope.
Since the minimum text length might be specified in LengthUnits.Characters for the minLength facet, and for the textOutputMinLength property, then converting those to bits requires possibly not knowing the encoding until runtime.
Since the minimum text length might be specified in LengthUnits.Characters for the minLength facet, and for the textOutputMinLength property, then converting those to bits requires possibly not knowing the encoding until runtime.
Hence, we have to compute this similarly at runtime.
Singleton Ok is for returning from checks and such which will either throw SDE or succeed.
Singleton Ok is for returning from checks and such which will either throw SDE or succeed. It means "check succeeded"
A parser takes a state, and returns an updated state
A parser takes a state, and returns an updated state
The fact that there are side-effects/mutations on parts of the state enables us to reuse low-level java primitives that mutate streams.
The goal however, is to hide that fact so that the only places that have to know are the places doing the mutation, and the places rolling them back which should be isolated to the alternative parser, and repParsers, i.e., places where points-of-uncertainty are handled.
A Prim or Primitive processor does not orchestrate the operation of other processors, it actually does the work of parsing/unparsing.
A Prim or Primitive processor does not orchestrate the operation of other processors, it actually does the work of parsing/unparsing.
Most PrimProcessor actually manipulate data to/from the data stream. Some (NoData) do not.
A PrimProcessor which does other work than manipulating the data, such as evaluating DPath expressions.
Captures common members for any processor, parser or unparser.
The resulting state after invoking a Processor
This is the DataProcessor constructed from a saved Parser.
Trait mixed into the PState.Mark object class and the ParseOrUnparseState
Trait mixed into the PState.Mark object class and the ParseOrUnparseState
contains member functions for everything the debugger needs to be able to observe.
SuspendableOperation is used for suspending and retrying things that aren't expressions.
SuspendableOperation is used for suspending and retrying things that aren't expressions. Example is an alignmentFill unparser. Until we know the absolute start bit positon, we can't lay down alignment fill bits.
This has to be suspended and retried later, but it's not an expression being evaluated that has forward references.
The suspension object keeps track of the state of the task, i.e., whether it is done, whether it is making forward progress when run or not.
The suspension object keeps track of the state of the task, i.e., whether it is done, whether it is making forward progress when run or not.
A suspension" may block, by which we mean it may set isDone to false, and return.
Running the suspension again tries again and will either block or complete.
must mixin to all processors that deal with text
BriefXML is XML-style output, but intended for specific purposes.
BriefXML is XML-style output, but intended for specific purposes. It is NOT an XML serialization of the data structure. It's an XML-style string, suitable to manipulate, by people, in XML tooling. E.g., can stick into an XML editor to then get it all indented nicely, use a structure editor to expand/collapse subregions, but it is NOT intended to capture all of the state of the object.
Used for unparsing, to determine whether we have a target length from the dfdl:length property and Explicit length, or for fixed length from facets or textOutputMinLength.
Used for unparsing, to determine whether we have a target length from the dfdl:length property and Explicit length, or for fixed length from facets or textOutputMinLength.
Evaluates to Nope if variable-width encoding prevents computing the targetLengthInBits.
Nope does NOT MEAN the length cannot be computed yet. It means variable-width encoding prevents computing the target length in bits units.
Core tuple of a pure functional "state" for variables.
Provides one more indirection to the variable map.
Provides one more indirection to the variable map.
Needed so that when unparsing multiple clones of a UState can share and modify, the same VMap.
This is a new mechanism, which allows for less rel-allocation of VariableMaps. There's a box that the variable map lives in, called vbox for convention. by having two UState items point to the same vbox they can share the variables. This is fine for unparsing, because all shared uses are not alternatives to each other which need independent variables, but are just different parts of the same unparse, which need the same variables.
Copying excessively in order to deal with the forward-referencing, and not-yet-computed stuff in unparsing is a battle. This vbox thing is one tiny improvement there. More will be needed.
Pure functional data structure for implementing DFDL's variables.
Pure functional data structure for implementing DFDL's variables.
Key concepts: DFDL variables are single-assignment. Once they have been set, they may not be set again. Furthermore, they have default values, and if the default value has been read, then they may not subsequently be set.
These constraints insure that the variables do NOT become a sneaky way to get generalized accumulators and hence turing-complete semantics (and complexity) into designing and debugging DFDL schemas. They also allow for parallel implementations since order of evaluation does not matter.
The DPath implementation must be made to implement the no-set-after-default-value-has-been-read behavior. This requires that reading the variables causes a state transition.
Implementation mixin - provides simple helper methods
Generates unique int for use as key into EvalCache
Singleton.
Singleton. If found as the default value, means to use nil as the default value instead of an actual value.
Used when unparsing.
Used when unparsing. A setVariable or newVariableInstance may be underway in the sense that the value of the variable is being computed but is blocked.
Readers in this situation should also block, not get any default value nor any error.
A combinator is a processor that orchestrates the operation of other processors.
Combinators must be pure. They cannot both touch the data stream directly and also orchestrate other processors.