object lexeme extends Lexeme
This object is concerned with lexemes: these are tokens that are treated as "words", such that whitespace will be consumed after each has been parsed.
Ideally, a wider parser should not be concerned with handling whitespace, as it is responsible for dealing with a stream of tokens. With parser combinators, however, it is usually not the case that there is a separate distinction between the parsing phase and the lexing phase. That said, it is good practice to establish a logical separation between the two worlds. As such, this object contains parsers that parse tokens, and these are whitespace-aware. This means that whitespace will be consumed after any of these parsers are parsed. It is not, however, required that whitespace be present.
- Source
- Lexer.scala
- Since
4.0.0
- Alphabetic
- By Inheritance
- lexeme
- Lexeme
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- def angles[A](p: => Parsley[A]): Parsley[A]
This combinator parses a
penclosed within angle brackets.This combinator parses a
penclosed within angle brackets.First parse an open bracket, any whitespace, then parse,
p, producingx. Finally, parse a closing bracket and any whitespace. If all three parts succeeded, then returnx. If any of them failed, this combinator fails.- p
the parser to parse between parentheses.
- returns
a parser that reads an open bracket, then
p, then a closing bracket and returns the result ofp.
scala> ... scala> val p = lexer.nonlexeme.enclosing.brackets(int) scala> p.parse("< 5>") val res0 = Success(5) scala> p.parse("<5") val res1 = Failure(...) scala> p.parse("5>") val res2 = Failure(...)
- Since
4.5.0
Example: - def apply[A](p: Parsley[A]): Parsley[A]
This combinator turns a non-lexeme parser into a lexeme one by ensuring whitespace is consumed after the parser.
This combinator turns a non-lexeme parser into a lexeme one by ensuring whitespace is consumed after the parser.
When using parser combinators, it is important to establish a consistent whitespace consumption scheme: ideally, there is no wasteful parsing, and whitespace consumption should not impact backtracking. This leads to a convention that whitespace must only be consumed after a token, and only once at the very start of the parser (see
fully). When manually constructing tokens that are not supported by this lexer, use this combinator to ensure it also follows the whitespace convention.- p
the token parser to ensure consumes trailing whitespace.
- Definition Classes
- lexeme → Lexeme
- Since
4.0.0
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def braces[A](p: => Parsley[A]): Parsley[A]
This combinator parses a
penclosed within braces.This combinator parses a
penclosed within braces.First parse an open brace, any whitespace, then parse,
p, producingx. Finally, parse a closing brace and any whitespace. If all three parts succeeded, then returnx. If any of them failed, this combinator fails.- p
the parser to parse between parentheses.
- returns
a parser that reads an open brace, then
p, then a closing brace and returns the result ofp.
scala> ... scala> val p = lexer.nonlexeme.enclosing.braces(int) scala> p.parse("{ 5}") val res0 = Success(5) scala> p.parse("{5") val res1 = Failure(...) scala> p.parse("5}") val res2 = Failure(...)
- Since
4.5.0
Example: - def brackets[A](p: => Parsley[A]): Parsley[A]
This combinator parses a
penclosed within square brackets.This combinator parses a
penclosed within square brackets.First parse an open bracket, any whitespace, then parse,
p, producingx. Finally, parse a closing bracket and any whitespace. If all three parts succeeded, then returnx. If any of them failed, this combinator fails.- p
the parser to parse between parentheses.
- returns
a parser that reads an open bracket, then
p, then a closing bracket and returns the result ofp.
scala> ... scala> val p = lexer.nonlexeme.enclosing.brackets(int) scala> p.parse("[ 5]") val res0 = Success(5) scala> p.parse("[5") val res1 = Failure(...) scala> p.parse("5]") val res2 = Failure(...)
- Since
4.5.0
Example: - def character: CharacterParsers
This is a collection of parsers concerned with handling character literals.
This is a collection of parsers concerned with handling character literals.
Character literals are described generally as follows:
desc.textDesc.characterLiteralEnd: the character that starts and ends the literal (for example in many languages this is')desc.textDesc.graphicCharacter: describes the legal characters that may appear in the literal directly. Usually, this excludes control characters and newlines, but permits most other things. Escape sequences can represent non-graphic charactersdesc.textDesc.escapeSequences: describes the legal escape sequences that that can appear in a character literal (for example\nor\u000a)
Aside from the generic configuration, characters can be parsed in accordance with varying levels of unicode support, from ASCII-only to full UTF-16 characters. Parsers for each of four different vareties are exposed by this object.
- Since
4.5.0
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- def commaSep[A](p: Parsley[A]): Parsley[List[A]]
This combinator parses zero or more occurrences of
p, separated by commas.This combinator parses zero or more occurrences of
p, separated by commas.Behaves just like
commaSep1, except does not require an initialp, returning the empty list instead.- p
the parser whose results are collected into a list.
- returns
a parser that parses
pdelimited by commas, returning the list ofp's results.
scala> ... scala> val stmts = lexer.lexeme.separators.commaSep(int) scala> stmts.parse("7, 3,2") val res0 = Success(List(7, 3, 2)) scala> stmts.parse("") val res1 = Success(Nil) scala> stmts.parse("1") val res2 = Success(List(1)) scala> stmts.parse("1, 2, ") val res3 = Failure(..) // no trailing comma allowed
- Since
4.5.0
Example: - def commaSep1[A](p: Parsley[A]): Parsley[List[A]]
This combinator parses one or more occurrences of
p, separated by commas.This combinator parses one or more occurrences of
p, separated by commas.First parses a
p. Then parses a comma followed bypuntil there are no more commas. The results of thep's,x1throughxn, are returned asList(x1, .., xn). Ifpfails having consumed input, the whole parser fails. Requires at least onepto have been parsed.- p
the parser whose results are collected into a list.
- returns
a parser that parses
pdelimited by commas, returning the list ofp's results.
scala> ... scala> val stmts = lexer.lexeme.separators.commaSep1(int) scala> stmts.parse("7, 3,2") val res0 = Success(List(7, 3, 2)) scala> stmts.parse("") val res1 = Failure(..) scala> stmts.parse("1") val res2 = Success(List(1)) scala> stmts.parse("1, 2, ") val res3 = Failure(..) // no trailing comma allowed
- Since
4.5.0
Example: - final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- def floating: RealParsers
This is a collection of parsers concerned with handling signed real numbers (like floats and doubles).
This is a collection of parsers concerned with handling signed real numbers (like floats and doubles).
These literals consist of a (possibly optional) integer prefix, with at least one of a fractional component (with
.) or an exponential component.Real numbers are an extension of signed integers with the following additional configuration:
desc.numericDesc.leadingDotAllowed: determines whether a literal like.0would be considered legaldesc.numericDesc.trailingDotAllowed: determines whether a literal like0.would be considered legaldesc.numericDesc.realNumbersCanBe{Hexadecimal/Octal/Binary}: these flags control what kind of literals can appear within thenumberparser. Each type of literal may still be individually parsed with its corresponding parser, regardless of the value of the flagdesc.numericDesc.{decimal/hexadecimal/octal/binary}ExponentDesc: describes how the exponential syntax works for each kind of base. If the syntax is legal, then this describes: which characters start it (classically, this would beeorEfor decimals); whether or not it is compulsory for the literal (in Java and C, hexadecimal floats are only valid when they have an exponent attached); and whether or not a+sign is mandatory, optional, or illegal for positive exponents
Additional to the parsing of decimal, hexadecimal, octal, and binary floating literals, each parser can be given a precision of IEEE 754 float or double. This can either be achieved by rounding to the nearest representable value, or by ensuring that the literal must be precisely representable as one of these numbers (which is defined as being one of binary, decimal or exact
floatanddoublevalues as described by Java) - final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def integer: IntegerParsers
This is a collection of parsers concerned with handling signed integer literals.
This is a collection of parsers concerned with handling signed integer literals.
Signed integer literals are an extension of unsigned integer literals with the following extra configuration:
desc.numericDesc.positiveSign: describes whether or not literals are allowed to omit+for positive literals, must write a+, or can never write a+.
- Since
4.5.0
- See also
naturalfor a full description of integer configuration
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def multiString: StringParsers
This is a collection of parsers concerned with handling multi-line string literals.
This is a collection of parsers concerned with handling multi-line string literals.
String literals are described generally as follows:
desc.textDesc.multiStringEnds: the sequence of characters that can begin or end a multi-line string literal. Regardless of which of these is used for a specific literal, the end of the literal must use the same sequencedesc.textDesc.graphicCharacter: describes the legal characters that may appear in the literal directly. Usually, this excludes control characters and newlines, but permits most other things. Escape sequences can represent non-graphic characters for non-raw stringsdesc.textDesc.escapeSequences: describes the legal escape sequences that that can appear in a string literal (for example\nor\u000a)
- Since
4.5.0
- val names: Names
This object contains lexing functionality relevant to the parsing of names, which include operators or identifiers.
This object contains lexing functionality relevant to the parsing of names, which include operators or identifiers.
The parsing of names is mostly concerned with finding the longest valid name that is not a reserved name, such as a hard keyword or a special operator.
- Since
4.0.0
- def natural: IntegerParsers
This is a collection of parsers concerned with handling unsigned (positive) integer literals.
This is a collection of parsers concerned with handling unsigned (positive) integer literals.
Natural numbers are described generally as follows:
desc.numericDesc.literalBreakChar: determines whether or not it is legal to "break up" the digits within a literal, for example: is1_000_000allowed? If this is legal, describes what the break character is, and whether it can appear after a hexadecimal/octal/binary prefixdesc.numericDesc.leadingZerosAllowed: determines whether or not it is possible to add extraneous zero digits onto the front of a number or not. In some languages, like C, this is disallowed, as numbers starting with0are octal numbers.desc.numericDesc.integerNumbersCanBe{Hexadecimal/Octal/Binary}: these flags control what kind of literals can appear within thenumberparser. Each type of literal can be individually parsed with its corresponding parser, regardless of the value of the flagdesc.numericDesc.{hexadecimal/octal/binary}Leads: controls what character must follow a0when starting a number to change it from decimal into another base. This set may be empty, in which case the literal is described purely with leading zero (C style octals would setoctalLeadstoSet.empty)
Additional to the parsing of decimal, hexadecimal, octal, and binary literals, each parser can be given a bit-width from 8- to 64-bit: this will check the parsed literal to ensure it is a legal literal of that size.
- Since
4.5.0
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- def parens[A](p: => Parsley[A]): Parsley[A]
This combinator parses a
penclosed within parentheses.This combinator parses a
penclosed within parentheses.First parse an open parenthesis, any whitespace, then parse,
p, producingx. Finally, parse a closing parenthesis and any whitespace. If all three parts succeeded, then returnx. If any of them failed, this combinator fails.- p
the parser to parse between parentheses.
- returns
a parser that reads an open parenthesis, then
p, then a closing parenthesis and returns the result ofp.
scala> ... scala> val p = lexer.nonlexeme.enclosing.parens(int) scala> p.parse("( 5)") val res0 = Success(5) scala> p.parse("(5") val res1 = Failure(...) scala> p.parse("5)") val res2 = Failure(...)
- Since
4.5.0
Example: - def rawMultiString: StringParsers
This is a collection of parsers concerned with handling multi-line string literals.
This is a collection of parsers concerned with handling multi-line string literals.
String literals are described generally as follows:
desc.textDesc.multiStringEnds: the sequence of characters that can begin or end a multi-line string literal. Regardless of which of these is used for a specific literal, the end of the literal must use the same sequencedesc.textDesc.graphicCharacter: describes the legal characters that may appear in the literal directly. Usually, this excludes control characters and newlines, but permits most other things. Escape sequences can represent non-graphic characters for non-raw stringsdesc.textDesc.escapeSequences: describes the legal escape sequences that that can appear in a string literal (for example\nor\u000a)
- Since
4.5.0
- Note
this will be parsed without handling any escape sequences, this includes literal-end characters and the escape prefix (often
"and\respectively)
- def rawString: StringParsers
This is a collection of parsers concerned with handling single-line string literals.
This is a collection of parsers concerned with handling single-line string literals.
String literals are described generally as follows:
desc.textDesc.stringEnds: the sequence of characters that can begin or end a string literal. Regardless of which of these is used for a specific literal, the end of the literal must use the same sequencedesc.textDesc.graphicCharacter: describes the legal characters that may appear in the literal directly. Usually, this excludes control characters and newlines, but permits most other things. Escape sequences can represent non-graphic characters for non-raw stringsdesc.textDesc.escapeSequences: describes the legal escape sequences that that can appear in a string literal (for example\nor\u000a)
- Since
4.5.0
- Note
this will be parsed without handling any escape sequences, this includes literal-end characters and the escape prefix (often
"and\respectively)
- def real: RealParsers
This is a collection of parsers concerned with handling signed real numbers (like floats and doubles).
This is a collection of parsers concerned with handling signed real numbers (like floats and doubles).
These literals consist of a (possibly optional) integer prefix, with at least one of a fractional component (with
.) or an exponential component.Real numbers are an extension of signed integers with the following additional configuration:
desc.numericDesc.leadingDotAllowed: determines whether a literal like.0would be considered legaldesc.numericDesc.trailingDotAllowed: determines whether a literal like0.would be considered legaldesc.numericDesc.realNumbersCanBe{Hexadecimal/Octal/Binary}: these flags control what kind of literals can appear within thenumberparser. Each type of literal may still be individually parsed with its corresponding parser, regardless of the value of the flagdesc.numericDesc.{decimal/hexadecimal/octal/binary}ExponentDesc: describes how the exponential syntax works for each kind of base. If the syntax is legal, then this describes: which characters start it (classically, this would beeorEfor decimals); whether or not it is compulsory for the literal (in Java and C, hexadecimal floats are only valid when they have an exponent attached); and whether or not a+sign is mandatory, optional, or illegal for positive exponents
Additional to the parsing of decimal, hexadecimal, octal, and binary floating literals, each parser can be given a precision of IEEE 754 float or double. This can either be achieved by rounding to the nearest representable value, or by ensuring that the literal must be precisely representable as one of these numbers (which is defined as being one of binary, decimal or exact
floatanddoublevalues as described by Java) - def semiSep[A](p: Parsley[A]): Parsley[List[A]]
This combinator parses zero or more occurrences of
p, separated by semi-colons.This combinator parses zero or more occurrences of
p, separated by semi-colons.Behaves just like
semiSep1, except does not require an initialp, returning the empty list instead.- p
the parser whose results are collected into a list.
- returns
a parser that parses
pdelimited by semi-colons, returning the list ofp's results.
scala> ... scala> val stmts = lexer.lexeme.separators.semiSep(int) scala> stmts.parse("7; 3;2") val res0 = Success(List(7; 3; 2)) scala> stmts.parse("") val res1 = Success(Nil) scala> stmts.parse("1") val res2 = Success(List(1)) scala> stmts.parse("1; 2; ") val res3 = Failure(..) // no trailing semi-colon allowed
- Since
4.5.0
Example: - def semiSep1[A](p: Parsley[A]): Parsley[List[A]]
This combinator parses one or more occurrences of
p, separated by semi-colons.This combinator parses one or more occurrences of
p, separated by semi-colons.First parses a
p. Then parses a semi-colon followed bypuntil there are no more semi-colons. The results of thep's,x1throughxn, are returned asList(x1, .., xn). Ifpfails having consumed input, the whole parser fails. Requires at least onepto have been parsed.- p
the parser whose results are collected into a list.
- returns
a parser that parses
pdelimited by semi-colons, returning the list ofp's results.
scala> ... scala> val stmts = lexer.lexeme.separators.semiSep1(int) scala> stmts.parse("7; 3;2") val res0 = Success(List(7; 3; 2)) scala> stmts.parse("") val res1 = Failure(..) scala> stmts.parse("1") val res2 = Success(List(1)) scala> stmts.parse("1; 2; ") val res3 = Failure(..) // no trailing semi-colon allowed
- Since
4.5.0
Example: - def signed: IntegerParsers
This is a collection of parsers concerned with handling signed integer literals.
This is a collection of parsers concerned with handling signed integer literals.
Signed integer literals are an extension of unsigned integer literals with the following extra configuration:
desc.numericDesc.positiveSign: describes whether or not literals are allowed to omit+for positive literals, must write a+, or can never write a+.
- def signedCombined: CombinedParsers
This is a collection of parsers concerned with handling numeric literals that may either be signed integers or signed reals.
This is a collection of parsers concerned with handling numeric literals that may either be signed integers or signed reals.
There is no additional configuration offered over that found in
integerorreal.the bit-bounds and precision of the integer or real parts of the result can be specified in any pairing.
- Since
4.5.0
- def string: StringParsers
This is a collection of parsers concerned with handling single-line string literals.
This is a collection of parsers concerned with handling single-line string literals.
String literals are described generally as follows:
desc.textDesc.stringEnds: the sequence of characters that can begin or end a string literal. Regardless of which of these is used for a specific literal, the end of the literal must use the same sequencedesc.textDesc.graphicCharacter: describes the legal characters that may appear in the literal directly. Usually, this excludes control characters and newlines, but permits most other things. Escape sequences can represent non-graphic characters for non-raw stringsdesc.textDesc.escapeSequences: describes the legal escape sequences that that can appear in a string literal (for example\nor\u000a)
- Since
4.5.0
- val symbol: Symbol
This object contains lexing functionality relevant to the parsing of atomic symbols.
This object contains lexing functionality relevant to the parsing of atomic symbols.
Symbols are characterised by their "unitness", that is, every parser inside returns
Unit. This is because they all parse a specific known entity, and, as such, the result of the parse is irrelevant. These can be things such as reserved names, or small symbols like parentheses. This object also contains a means of creating new symbols as well as implicit conversions to allow for Scala's string literals to serve as symbols within a parser.- Since
4.0.0
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- def unsigned: IntegerParsers
This is a collection of parsers concerned with handling unsigned (positive) integer literals.
This is a collection of parsers concerned with handling unsigned (positive) integer literals.
Natural numbers are described generally as follows:
desc.numericDesc.literalBreakChar: determines whether or not it is legal to "break up" the digits within a literal, for example: is1_000_000allowed? If this is legal, describes what the break character is, and whether it can appear after a hexadecimal/octal/binary prefixdesc.numericDesc.leadingZerosAllowed: determines whether or not it is possible to add extraneous zero digits onto the front of a number or not. In some languages, like C, this is disallowed, as numbers starting with0are octal numbers.desc.numericDesc.integerNumbersCanBe{Hexadecimal/Octal/Binary}: these flags control what kind of literals can appear within thenumberparser. Each type of literal can be individually parsed with its corresponding parser, regardless of the value of the flagdesc.numericDesc.{hexadecimal/octal/binary}Leads: controls what character must follow a0when starting a number to change it from decimal into another base. This set may be empty, in which case the literal is described purely with leading zero (C style octals would setoctalLeadstoSet.empty)
Additional to the parsing of decimal, hexadecimal, octal, and binary literals, each parser can be given a bit-width from 8- to 64-bit: this will check the parsed literal to ensure it is a legal literal of that size.
- Since
4.5.0
- Note
alias for
natural.
- def unsignedCombined: CombinedParsers
This is a collection of parsers concerned with handling numeric literals that may either be unsigned integers or unsigned reals.
This is a collection of parsers concerned with handling numeric literals that may either be unsigned integers or unsigned reals.
There is no additional configuration offered over that found in
naturalorreal.the bit-bounds and precision of the integer or real parts of the result can be specified in any pairing.
- Since
4.5.0
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
This is the documentation for Parsley.
Package structure
The parsley package contains the
Parsleyclass, as well as theResult,Success, andFailuretypes. In addition to these, it also contains the following packages and "modules" (a module is defined as being an object which mocks a package):parsley.Parsleycontains the bulk of the core "function-style" combinators.parsley.combinatorcontains many helpful combinators that simplify some common parser patterns.parsley.charactercontains the combinators needed to read characters and strings, as well as combinators to match specific sub-sets of characters.parsley.debugcontains debugging combinators, helpful for identifying faults in parsers.parsley.exprcontains the following sub modules:parsley.expr.chaincontains combinators used in expression parsingparsley.expr.precedenceis a builder for expression parsers built on a precedence table.parsley.expr.infixcontains combinators used in expression parsing, but with more permissive types than their equivalents inchain.parsley.expr.mixedcontains combinators that can be used for expression parsing, but where different fixities may be mixed on the same level: this is rare in practice.parsley.syntaxcontains several implicits to add syntactic sugar to the combinators. These are sub-categorised into the following sub modules:parsley.syntax.charactercontains implicits to allow you to use character and string literals as parsers.parsley.syntax.liftenables postfix application of the lift combinator onto a function (or value).parsley.syntax.zippedenables boths a reversed form of lift where the function appears on the right and is applied on a tuple (useful when type inference has failed) as well as a.zippedmethod for building tuples out of several combinators.parsley.syntax.extensioncontains syntactic sugar combinators exposed as implicit classes.parsley.errorscontains modules to deal with error messages, their refinement and generation.parsley.errors.combinatorprovides combinators that can be used to either produce more detailed errors as well as refine existing errors.parsley.errors.tokenextractorsprovides mixins for common token extraction strategies during error message generation: these can be used to avoid implementingunexpectedTokenin theErrorBuilder.parsley.liftcontains functions which lift functions that work on regular types to those which now combine the results of parsers returning those same types. these are ubiquitous.parsley.apcontains functions which allow for the application of a parser returning a function to several parsers returning each of the argument types.parsley.statecontains combinators that interact with the context-sensitive functionality in the form of state.parsley.tokencontains theLexerclass that provides a host of helpful lexing combinators when provided with the description of a language.parsley.positioncontains parsers for extracting position information.parsley.genericcontains some basic implementations of the Parser Bridge pattern (see Design Patterns for Parser Combinators in Scala, or the parsley wiki): these can be used before more specialised generic bridge traits can be constructed.