This object is concerned with lexemes: these are tokens that are
treated as "words", such that whitespace will be consumed after each
has been parsed.
Ideally, a wider parser should not be concerned with
handling whitespace, as it is responsible for dealing with a stream
of tokens. With parser combinators, however, it is usually not the
case that there is a separate distinction between the parsing phase
and the lexing phase. That said, it is good practice to establish
a logical separation between the two worlds. As such, this object
contains parsers that parse tokens, and these are whitespace-aware.
This means that whitespace will be consumed after any of these
parsers are parsed. It is not, however, required that whitespace be
present.
This object contains lexing functionality relevant to the parsing
of numbers.
This object contains lexing functionality relevant to the parsing
of numbers. This is sub-divided into different categories:
integers (both signed and unsigned)
reals (signed only)
a combination of the two (signed and unsigned)
These contain relevant functionality for the processing of
decimal, hexadecimal, octal, and binary literals; or some
mixed combination thereof (as specified by desc.numericDesc).
Additionally, it is possible to ensure literals represent known
sizes or precisions.
This combinator turns a non-lexeme parser into a lexeme one by
ensuring whitespace is consumed after the parser.
This combinator turns a non-lexeme parser into a lexeme one by
ensuring whitespace is consumed after the parser.
When using parser combinators, it is important to establish a
consistent whitespace consumption scheme: ideally, there is no
wasteful parsing, and whitespace consumption should not impact
backtracking. This leads to a convention that whitespace must
only be consumed after a token, and only once at the very
start of the parser (see fully). When manually
constructing tokens that are not supported by this lexer, use
this combinator to ensure it also follows the whitespace convention.
Attributes
p
the token parser to ensure consumes trailing whitespace.
This object contains lexing functionality relevant to the parsing
of names, which include operators or identifiers.
This object contains lexing functionality relevant to the parsing
of names, which include operators or identifiers.
The parsing of names is mostly concerned with finding the longest
valid name that is not a reserved name, such as a hard keyword or
a special operator.
This object contains lexing functionality relevant to the parsing
of atomic symbols.
This object contains lexing functionality relevant to the parsing
of atomic symbols.
Symbols are characterised by their "unitness", that is, every parser
inside returns Unit. This is because they all parse a specific
known entity, and, as such, the result of the parse is irrelevant.
These can be things such as reserved names, or small symbols like
parentheses. This object also contains a means of creating new symbols
as well as implicit conversions to allow for Scala's string literals to serve
as symbols within a parser.