package token
- Alphabetic
- By Inheritance
- token
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Type Members
- sealed trait Impl extends AnyRef
The Impl trait is used to provide implementation of the parser requirements from
LanguageDef
The Impl trait is used to provide implementation of the parser requirements from
LanguageDef
- Since
2.2.0
- case class LanguageDef(commentStart: String, commentEnd: String, commentLine: String, nestedComments: Boolean, identStart: Impl, identLetter: Impl, opStart: Impl, opLetter: Impl, keywords: Set[String], operators: Set[String], caseSensitive: Boolean, space: Impl) extends Product with Serializable
This class is required to construct a TokenParser.
This class is required to construct a TokenParser. It defines the various characteristics of the language to be tokenised. Where a parameter can be either a
Set[Char]
or aParsley
object, prefer theSet
where possible. It will unlock a variety of faster intrinsic versions of the parsers, which will greatly improve tokenisation performance! In addition, the Sets are one time converted to heavily optimised BitSets, though that has up to 8KB memory usage associated but at least doubles the execution speed for that instruction. Seeparsley.Impl
.- commentStart
For multi-line comments; how does the comment start? (If this or
commentEnd
is the empty string, multi-line comments are disabled)- commentEnd
For multi-line comments; how does the comment end? (If this or
commentEnd
is the empty string, multi-line comments are disabled)- commentLine
For single-line comments; how does the comment start? (This this is the empty string, single-line comments are disabled)
- nestedComments
Are multi-line comments allowed to be nested inside each other? E.g. If
{-
and-}
are opening and closing comments, is the following valid syntax:{-{-hello -}-}
? Note in C this is not the case.- identStart
What characters can an identifier in the language start with?
- identLetter
What characters can an identifier in the language consist of after the starting character?
- opStart
What characters can an operator in the language start with?
- opLetter
What characters can an operator in the language consist of after the starting character?
- keywords
What keywords does the language contain?
- operators
What operators does the language contain?
- caseSensitive
Is the language case-sensitive. I.e. is IF equivalent to if?
- space
What characters count as whitespace in the language?
- Since
2.2.0
- class Lexer extends AnyRef
When provided with a
LanguageDef
, this class will produce a large variety of parsers that can be used for tokenisation of a language.When provided with a
LanguageDef
, this class will produce a large variety of parsers that can be used for tokenisation of a language. This includes parsing numbers and strings in their various formats and ensuring that all operations consume whitespace after them (so-called lexeme parsers). These are very useful in parsing programming languages. This class also has a large number of hand-optimised intrinsic parsers to improve performance!- Since
2.2.0
- final case class Parser(p: Parsley[_]) extends Impl with Product with Serializable
The implementation provided is a parser which parses the required token.
The implementation provided is a parser which parses the required token.
- p
The parser which will parse the token
- Since
2.2.0
- final case class Predicate(f: (Char) => Boolean) extends Impl with Product with Serializable
The implementation provided is a function which matches on the input streams characters
The implementation provided is a function which matches on the input streams characters
- f
The predicate that input tokens are tested against
- Since
2.2.0
Value Members
- object BitGen
This implementation uses a predicate to generate a BitSet.
This implementation uses a predicate to generate a BitSet. This should be preferred over
Predicate
when the function in question is expensive to execute and the parser itself is expected to be used many times. If the predicate is cheap, this is unlikely to provide any performance improvements, but will instead incur heavy space costs- Since
2.2.0
- object CharSet
This implementation uses a set of valid tokens.
This implementation uses a set of valid tokens. It is converted to a high-performance BitSet.
- Since
2.2.0
- object LanguageDef extends Serializable
This object contains any preconfigured language definitions
This object contains any preconfigured language definitions
- Since
2.2.0
- case object NotRequired extends Impl with Product with Serializable
This implementation states that the required functionality is not required.
This implementation states that the required functionality is not required. If it is used it will raise an error at parse-time
- Since
2.2.0
This is the documentation for Parsley.
Package structure
The parsley package contains the
Parsley
class, as well as theResult
,Success
, andFailure
types. In addition to these, it also contains the following packages and "modules" (a module is defined as being an object which mocks a package):parsley.Parsley
contains the bulk of the core "function-style" combinators, as well as the implicit classes which enable the "method-style" combinators.parsley.combinator
contains many helpful combinators that simplify some common parser patterns.parsley.character
contains the combinators needed to read characters and strings, as well as combinators to match specific sub-sets of characters.parsley.debug
contains debugging combinators, helpful for identifying faults in parsers.parsley.expr
contains the following sub modules:parsley.expr.chain
contains combinators used in expression parsingparsley.expr.precedence
is a builder for expression parsers built on a precedence table.parsley.implicits
contains several implicits to add syntactic sugar to the combinators, such as being able to use character and string literals directly as parsers, as well as enabling lifting of functions to work on parsers.parsley.lift
contains functions which lift functions that work on regular types to those which now combine the results of parsers returning those same types. these are ubiquitous.parsley.registers
contains combinators that interact with the context-sensitive functionality in the form of registers.parsley.token
contains theLexer
class that provides a host of helpful lexing combinators when provided with the description of a language.parsley.unsafe
contains unsafe (and not thread-safe) ways of speeding up the execution of a parser.In addition to the modules and packages outlined above, this version of Parsley (up to version 3.0), also includes the so-called
old-style
API, which is deprecated (see the Parsley wiki for a discussion of these differences). You should use the modules described above, and avoid the following:parsley.BitGen
parsley.Char
parsley.CharSet
parsley.Combinator
parsley.ExpressionParser
parsley.Impl
parsley.Implicits
parsley.LanguageDef
parsley.NotRequired
parsley.Parser
parsley.Predicate
parsley.Reg
parsley.TokenParser