parsley.token
Members list
Type members
Classlikes
This class provides a large selection of functionality concerned with lexing.
This class provides a large selection of functionality concerned with lexing.
This class provides lexing functionality to parsley
, however
it is guaranteed that nothing in this class is not implementable
purely using parsley
's pre-existing functionality. These are
regular parsers, but constructed in such a way that they create
a clear and logical separation from the rest of the parser.
The class is broken up into several internal "modules" that group
together similar kinds of functionality. Importantly, the lexemes
and nonlexemes
objects separate the underlying token implementations
based on whether or not they consume whitespace or not. Functionality
is broadly duplicated across both of these modules: lexemes
should
be used by a wider parser, to ensure whitespace is handled uniformly;
and nonlexemes
should be used to define further composite tokens or
in special circumstances where whitespace should not be consumed.
It is possible that some of the implementations of parsers found within this class may have been hand-optimised for performance: care will have been taken to ensure these implementations precisely match the semantics of the originals.
Attributes
- Source:
- Lexer.scala
- Graph
- Supertypes
This module contains functionality to describe character predicates, which can be used to determine what characters are valid for different tokens.
This module contains functionality to describe character predicates, which can be used to determine what characters are valid for different tokens.
Attributes
- Since:
4.0.0
- Source:
- predicate.scala
- Graph
- Supertypes
- Self type
- predicate.type