scala.util.parsing.combinator.lexical

StdLexical

class StdLexical extends Lexical with StdTokens

This component provides a standard lexical parser for a simple, Scala-like language. It parses keywords and identifiers, numeric literals (integers), strings, and delimiters.

To distinguish between identifiers and keywords, it uses a set of reserved identifiers: every string contained in reserved' is returned as a keyword token. (Note that "=>" is hard-coded as a keyword.) Additionally, the kinds of delimiters can be specified by the delimiters' set.

Usually this component is used to break character-based input into bigger tokens, which are then passed to a token-parser {@see TokenParsers}.

known subclasses: Lexer

Inherits

  1. StdTokens
  2. Lexical
  3. Tokens
  4. Scanners
  5. Parsers
  6. AnyRef
  7. Any

Type Members

  1. type Elem = Char

    the type of input elements

    the type of input elements

    definition classes: ScannersParsers
  2. class Error(msg: String, next: Reader[Elem]) extends NoSuccess with Product

    The fatal failure case of ParseResult: contains an error-message and the remaining input

  3. class ErrorToken(msg: String) extends Token with Product

    A class of error tokens

  4. class Failure(msg: String, next: Reader[Elem]) extends NoSuccess with Product

    The failure case of ParseResult: contains an error-message and the remaining input

  5. class Identifier(chars: String) extends Token with Product

    The class of identifier tokens

  6. type Input = Reader[Char]

    The parser input is an abstract reader of input elements

    The parser input is an abstract reader of input elements

    definition classes: Parsers
  7. class Keyword(chars: String) extends Token with Product

    The class of keyword tokens

  8. class NoSuccess extends ParseResult[Nothing]

    A common super-class for unsuccessful parse results

  9. class NumericLit(chars: String) extends Token with Product

    The class of numeric literal tokens

  10. trait OnceParser[+T] extends Parser[T]

    A parser whose ~ combinator disallows back-tracking

  11. class ParseResult[+T] extends AnyRef

    A base class for parser results

  12. class Parser[+T] extends (Reader[Elem]) ⇒ ParseResult[T]

    The root class of parsers

  13. class Scanner extends Reader[Token]

    Scanner is essentially(*) a parser that produces Token's from a stream of characters

  14. class StringLit(chars: String) extends Token with Product

    The class of string literal tokens

  15. class Success[+T](result: T, next: Reader[Elem]) extends ParseResult[T] with Product

    The success case of ParseResult: contains the result and the remaining input

  16. class Token extends AnyRef

    Objects of this type are produced by a lexical parser or scanner, and consumed by a parser {@see scala

  17. class ~[+a, +b](_1: a, _2: b) extends Product

Value Members

  1. object EOF extends Token with Product

    A class for end-of-file tokens

  2. object NoSuccess extends AnyRef

    An extractor so NoSuccess(msg, next) can be used in matches

  3. def OnceParser[T](f: (Reader[Char]) ⇒ ParseResult[T]): Parser[T] with OnceParser[T]

  4. def Parser[T](f: (Reader[Char]) ⇒ ParseResult[T]): Parser[T]

  5. def accept[U](expected: String, f: PartialFunction[Char, U]): Parser[U]

    The parser that matches an element in the domain of the partial function f' If f' is defined on the first element in the input, f' is applied to it to produce this parser's result

    The parser that matches an element in the domain of the partial function f'

    If f' is defined on the first element in the input, f' is applied to it to produce this parser's result.

    Example: The parser accept("name", {case Identifier(n) => Name(n)}) accepts an Identifier(n) and returns a Name(n).

    expected

    a description of the kind of element this parser expects (for error messages)

    f

    a partial function that determines when this parser is successful and what its output is

    returns

    A parser that succeeds if f' is applicable to the first element of the input, applying f' to it to produce the result.

    definition classes: Parsers
  6. def accept[ES](es: ES)(arg0: (ES) ⇒ List[Char]): Parser[List[Char]]

    A parser that matches only the given list of element es' accept(es) succeeds if the input subsequently provides the elements in the list es'

    A parser that matches only the given list of element es'

    accept(es) succeeds if the input subsequently provides the elements in the list es'.

    es

    the list of expected elements

    returns

    a Parser that recognizes a specified list of elements

    definition classes: Parsers
  7. def accept(e: Char): Parser[Char]

    A parser that matches only the given element e' The method is implicit so that elements can automatically be lifted to their parsers

    A parser that matches only the given element e'

    The method is implicit so that elements can automatically be lifted to their parsers. For example, when parsing Token's, Identifier("new") (which is a Token') can be used directly, instead of first creating a Parser' using accept(Identifier("new")).

    e

    the Elem' that must be the next piece of input for the returned parser to succeed

    returns

    a tParser' that succeeds if e' is the next available input.

    attributes: implicit
    definition classes: Parsers
  8. def acceptIf(p: (Char) ⇒ Boolean)(err: (Char) ⇒ String): Parser[Char]

  9. def acceptMatch[U](expected: String, f: PartialFunction[Char, U]): Parser[U]

  10. def acceptSeq[ES](es: ES)(arg0: (ES) ⇒ Iterable[Char]): Parser[List[Char]]

  11. def chainl1[T, U](first: ⇒ Parser[T], p: ⇒ Parser[U], q: ⇒ Parser[(T, U) ⇒ T]): Parser[T]

    A parser generator that, roughly, generalises the rep1sep generator so that q', which parses the separator, produces a left-associative function that combines the elements it separates

    A parser generator that, roughly, generalises the rep1sep generator so that q', which parses the separator, produces a left-associative function that combines the elements it separates.

    first

    a parser that parses the first element

    p

    a parser that parses the subsequent elements

    q

    a parser that parses the token(s) separating the elements, yielding a left-associative function that combines two elements into one

    definition classes: Parsers
  12. def chainl1[T](p: ⇒ Parser[T], q: ⇒ Parser[(T, T) ⇒ T]): Parser[T]

    A parser generator that, roughly, generalises the rep1sep generator so that q', which parses the separator, produces a left-associative function that combines the elements it separates

    A parser generator that, roughly, generalises the rep1sep generator so that q', which parses the separator, produces a left-associative function that combines the elements it separates.

    From: J. Fokker. Functional parsers. In J. Jeuring and E. Meijer, editors, Advanced Functional Programming, volume 925 of Lecture Notes in Computer Science, pages 1--23. Springer, 1995.

    p

    a parser that parses the elements

    q

    a parser that parses the token(s) separating the elements, yielding a left-associative function that combines two elements into one

    definition classes: Parsers
  13. def chainr1[T, U](p: ⇒ Parser[T], q: ⇒ Parser[(T, U) ⇒ U], combine: (T, U) ⇒ U, first: U): Parser[U]

    A parser generator that generalises the rep1sep generator so that q', which parses the separator, produces a right-associative function that combines the elements it separates

    A parser generator that generalises the rep1sep generator so that q', which parses the separator, produces a right-associative function that combines the elements it separates. Additionally, The right-most (last) element and the left-most combinating function have to be supplied. rep1sep(p: Parser[T], q) corresponds to chainr1(p, q ^^ cons, cons, Nil) (where val cons = (x: T, y: List[T]) => x :: y)

    p

    a parser that parses the elements

    q

    a parser that parses the token(s) separating the elements, yielding a right-associative function that combines two elements into one

    combine

    the "last" (left-most) combination function to be applied

    first

    the "first" (right-most) element to be combined

    definition classes: Parsers
  14. def chrExcept(cs: Char*): Parser[Char]

    A character-parser that matches any character except the ones given in cs' (and returns it)

    A character-parser that matches any character except the ones given in cs' (and returns it)

    definition classes: Lexical
  15. def commit[T](p: ⇒ Parser[T]): Parser[T]

    Wrap a parser so that its failures become errors (the | combinator will give up as soon as it encounters an error, on failure it simply tries the next alternative)

    Wrap a parser so that its failures become errors (the | combinator will give up as soon as it encounters an error, on failure it simply tries the next alternative)

    definition classes: Parsers
  16. val delimiters: HashSet[String]

    The set of delimiters (ordering does not matter)

    The set of delimiters (ordering does not matter)

  17. def digit: Parser[Char]

    A character-parser that matches a digit (and returns it)

    A character-parser that matches a digit (and returns it)

    definition classes: Lexical
  18. def elem(e: Char): Parser[Char]

    A parser that matches only the given element e' elem(e) succeeds if the input starts with an element e'

    A parser that matches only the given element e'

    elem(e) succeeds if the input starts with an element e'

    e

    the Elem' that must be the next piece of input for the returned parser to succeed

    returns

    a Parser' that succeeds if e' is the next available input (and returns it).

    definition classes: Parsers
  19. def elem(kind: String, p: (Char) ⇒ Boolean): Parser[Char]

    A parser matching input elements that satisfy a given predicate

    A parser matching input elements that satisfy a given predicate

    elem(kind, p) succeeds if the input starts with an element e' for which p(e) is true.

    kind

    The element kind, used for error messages

    p

    A predicate that determines which elements match. @return

    definition classes: Parsers
  20. def equals(arg0: Any): Boolean

    This method is used to compare the receiver object (this) with the argument object (arg0) for equivalence

    This method is used to compare the receiver object (this) with the argument object (arg0) for equivalence.

    The default implementations of this method is an equivalence relation:

    • It is reflexive: for any instance x of type Any, x.equals(x) should return true.
    • It is symmetric: for any instances x and y of type Any, x.equals(y) should return true if and only if y.equals(x) returns true.
    • It is transitive: for any instances x, y, and z of type AnyRef if x.equals(y) returns true and y.equals(z) returns true, then x.equals(z) should return true.

    If you override this method, you should verify that your implementation remains an equivalence relation. Additionally, when overriding this method it is often necessary to override hashCode to ensure that objects that are "equal" (o1.equals(o2) returns true) hash to the same Int (o1.hashCode.equals(o2.hashCode)).

    arg0

    the object to compare against this object for equality.

    returns

    true if the receiver object is equivalent to the argument; false otherwise.

    definition classes: AnyRef ⇐ Any
  21. def err(msg: String): Parser[Nothing]

    A parser that results in an error

    A parser that results in an error

    msg

    The error message describing the failure.

    returns

    A parser that always fails with the specified error message.

    definition classes: Parsers
  22. def errorToken(msg: String): Token

    This token is produced by a scanner {@see Scanner} when scanning failed

    This token is produced by a scanner {@see Scanner} when scanning failed.

    definition classes: Tokens
  23. def failure(msg: String): Parser[Nothing]

    A parser that always fails

    A parser that always fails

    msg

    The error message describing the failure.

    returns

    A parser that always fails with the specified error message.

    definition classes: Parsers
  24. def guard[T](p: ⇒ Parser[T]): Parser[T]

    A parser generator for guard expressions

    A parser generator for guard expressions. The resulting parser will fail or succeed just like the one given as parameter but it will not consume any input.

    p

    a Parser' that is to be applied to the input

    returns

    A parser that returns success if and only if 'p' succeeds but never consumes any input

    definition classes: Parsers
  25. def hashCode(): Int

    Returns a hash code value for the object

    Returns a hash code value for the object.

    The default hashing algorithm is platform dependent.

    Note that it is allowed for two objects to have identical hash codes (o1.hashCode.equals(o2.hashCode)) yet not be equal (o1.equals(o2) returns false). A degenerate implementation could always return 0. However, it is required that if two objects are equal (o1.equals(o2) returns true) that they have identical hash codes (o1.hashCode.equals(o2.hashCode)). Therefore, when overriding this method, be sure to verify that the behavior is consistent with the equals method.

    definition classes: AnyRef ⇐ Any
  26. def identChar: Parser[Char]

  27. var lastNoSuccess: NoSuccess

  28. def letter: Parser[Char]

    A character-parser that matches a letter (and returns it)

    A character-parser that matches a letter (and returns it)

    definition classes: Lexical
  29. def log[T](p: ⇒ Parser[T])(name: String): Parser[T]

  30. def mkList[T]: (~[T, List[T]]) ⇒ List[T]

  31. def not[T](p: ⇒ Parser[T]): Parser[Unit]

    Wrap a parser so that its failures&errors become success and vice versa -- it never consumes any input

    Wrap a parser so that its failures&errors become success and vice versa -- it never consumes any input

    definition classes: Parsers
  32. def opt[T](p: ⇒ Parser[T]): Parser[Option[T]]

    A parser generator for optional sub-phrases

    A parser generator for optional sub-phrases.

    opt(p) is a parser that returns Some(x)' if p' returns x' and None' if p' fails

    p

    A Parser' that is tried on the input

    returns

    a Parser' that always succeeds: either with the result provided by p' or with the empty result

    definition classes: Parsers
  33. def phrase[T](p: Parser[T]): Parser[T]

    A parser generator delimiting whole phrases (i

    A parser generator delimiting whole phrases (i.e. programs).

    phrase(p) succeeds if p succeeds and no input is left over after p.

    p

    the parser that must consume all input for the resulting parser to succeed.

    returns

    a parser that has the same result as p', but that only succeeds if p consumed all the input.

    definition classes: Parsers
  34. def positioned[T <: Positional](p: ⇒ Parser[T]): Parser[T]

    positioned' decorates a parser's result with the start position of the input it consumed

    positioned' decorates a parser's result with the start position of the input it consumed.

    p

    a Parser' whose result conforms to Positional'.

    returns

    A parser that has the same behaviour as p', but which marks its result with the start position of the input it consumed, if it didn't already have a position.

    definition classes: Parsers
  35. def rep[T](p: ⇒ Parser[T]): Parser[List[T]]

    A parser generator for repetitions

    A parser generator for repetitions.

    rep(p) repeatedly uses p' to parse the input until p' fails (the result is a List of the consecutive results of p')

    p

    a Parser' that is to be applied successively to the input

    returns

    A parser that returns a list of results produced by repeatedly applying p' to the input.

    definition classes: Parsers
  36. def rep1[T](first: ⇒ Parser[T], p: ⇒ Parser[T]): Parser[List[T]]

    A parser generator for non-empty repetitions

    A parser generator for non-empty repetitions.

    rep1(f, p) first uses f' (which must succeed) and then repeatedly uses p' to parse the input until p' fails (the result is a List' of the consecutive results of f' and p')

    first

    a Parser' that parses the first piece of input

    p

    a Parser' that is to be applied successively to the rest of the input (if any)

    returns

    A parser that returns a list of results produced by first applying f' and then repeatedly p' to the input (it only succeeds if f' matches).

    definition classes: Parsers
  37. def rep1[T](p: ⇒ Parser[T]): Parser[List[T]]

    A parser generator for non-empty repetitions

    A parser generator for non-empty repetitions.

    rep1(p) repeatedly uses p' to parse the input until p' fails -- p' must succeed at least once (the result is a List' of the consecutive results of p')

    p

    a Parser' that is to be applied successively to the input

    returns

    A parser that returns a list of results produced by repeatedly applying p' to the input (and that only succeeds if p' matches at least once).

    definition classes: Parsers
  38. def rep1sep[T](p: ⇒ Parser[T], q: ⇒ Parser[Any]): Parser[List[T]]

    A parser generator for non-empty repetitions

    A parser generator for non-empty repetitions.

    rep1sep(p, q) repeatedly applies p' interleaved with q' to parse the input, until p' fails. The parser p' must succeed at least once.

    p

    a Parser' that is to be applied successively to the input

    q

    a Parser' that parses the elements that separate the elements parsed by p' (interleaved with q')

    returns

    A parser that returns a list of results produced by repeatedly applying p' to the input (and that only succeeds if p' matches at least once). The results of p' are collected in a list. The results of q' are discarded.

    definition classes: Parsers
  39. def repN[T](n: Int, p: ⇒ Parser[T]): Parser[List[T]]

    A parser generator for a specified number of repetitions

    A parser generator for a specified number of repetitions.

    repN(n, p) uses p' exactly n' time to parse the input (the result is a List' of the n' consecutive results of p')

    n

    the exact number of times p' must succeed

    p

    a Parser' that is to be applied successively to the input

    returns

    A parser that returns a list of results produced by repeatedly applying p' to the input (and that only succeeds if p' matches exactly n' times).

    definition classes: Parsers
  40. def repsep[T](p: ⇒ Parser[T], q: ⇒ Parser[Any]): Parser[List[T]]

    A parser generator for interleaved repetitions

    A parser generator for interleaved repetitions.

    repsep(p, q) repeatedly uses p' interleaved with q' to parse the input, until p' fails. (The result is a List' of the results of p'.)

    Example: repsep(term, ",") parses a comma-separated list of term's, yielding a list of these terms

    p

    a Parser' that is to be applied successively to the input

    q

    a Parser' that parses the elements that separate the elements parsed by p'

    returns

    A parser that returns a list of results produced by repeatedly applying p' (interleaved with q') to the input. The results of p' are collected in a list. The results of q' are discarded.

    definition classes: Parsers
  41. val reserved: HashSet[String]

    The set of reserved identifiers: these will be returned as Keyword's

    The set of reserved identifiers: these will be returned as Keyword's

  42. def success[T](v: T): Parser[T]

    A parser that always succeeds

    A parser that always succeeds

    v

    The result for the parser

    returns

    A parser that always succeeds, with the given result v'

    definition classes: Parsers
  43. def toString(): String

    Returns a string representation of the object

    Returns a string representation of the object.

    The default representation is platform dependent.

    definition classes: AnyRef ⇐ Any
  44. def token: Parser[Token]

    a parser that produces a token (from a stream of characters)

    a parser that produces a token (from a stream of characters)

  45. def whitespace: Parser[Any]

    a parser for white-space -- its result will be discarded

    a parser for white-space -- its result will be discarded

  46. def whitespaceChar: Parser[Char]

    A character-parser that matches a white-space character (and returns it)

    A character-parser that matches a white-space character (and returns it)

    definition classes: Lexical

Instance constructors

  1. new StdLexical()