package token
This package provides a wealth of functionality for performing common lexing tasks.
It is organised as follows:
- the main parsing functionality is accessed via
Lexer
, which provides implementations for the combinators found in the sub-packages given aLexicalDesc
. - the
descriptions
sub-package is how a lexical structure can be described, providing the configuration that alters the behaviour of the parsers produced by theLexer
. - the other sub-packages contain the high-level interfaces that the
Lexer
exposes, which can be used to pass whitespace-aware and non-whitespace-aware combinators around in a uniform way. - the
predicate
module contains functionality to help define boolean predicates on characters or unicode codepoints.
- Source
- package.scala
- Alphabetic
- By Inheritance
- token
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Package Members
- package descriptions
This package contains the descriptions of various lexical structures to be fed to
Lexer
.This package contains the descriptions of various lexical structures to be fed to
Lexer
.- Since
4.0.0
- package errors
- package names
This package contains the abstract parsers for parsing identifiers and operators.
This package contains the abstract parsers for parsing identifiers and operators.
- Since
4.0.0
- package numeric
This package contains the abstract parsers for parsing numeric literals, like integers and reals.
This package contains the abstract parsers for parsing numeric literals, like integers and reals.
- Since
4.0.0
- package symbol
This package contains the abstract parsers for parsing symbolic tokens like keywords.
This package contains the abstract parsers for parsing symbolic tokens like keywords.
- Since
4.0.0
- package text
This package contains the abstract parsers for parsing string and character literals.
This package contains the abstract parsers for parsing string and character literals.
- Since
4.0.0
Type Members
- class Lexer extends AnyRef
This class provides a large selection of functionality concerned with lexing.
This class provides a large selection of functionality concerned with lexing.
This class provides lexing functionality to
parsley
, however it is guaranteed that nothing in this class is not implementable purely usingparsley
's pre-existing functionality. These are regular parsers, but constructed in such a way that they create a clear and logical separation from the rest of the parser.The class is broken up into several internal "modules" that group together similar kinds of functionality. Importantly, the
lexemes
andnonlexemes
objects separate the underlying token implementations based on whether or not they consume whitespace or not. Functionality is broadly duplicated across both of these modules:lexemes
should be used by a wider parser, to ensure whitespace is handled uniformly; andnonlexemes
should be used to define further composite tokens or in special circumstances where whitespace should not be consumed.It is possible that some of the implementations of parsers found within this class may have been hand-optimised for performance: care will have been taken to ensure these implementations precisely match the semantics of the originals.
- Annotations
- @deprecatedInheritance()
Value Members
- object predicate
This module contains functionality to describe character predicates, which can be used to determine what characters are valid for different tokens.
This module contains functionality to describe character predicates, which can be used to determine what characters are valid for different tokens.
- Since
4.0.0