LookaheadScanner

dotty.tools.dotc.parsing.Scanners.Scanner.LookaheadScanner
class LookaheadScanner(val allowIndent: Boolean) extends Scanner

Attributes

Graph
Supertypes
class Scanner
trait TokenData
class Object
trait Matchable
class Any
Show all

Members list

Type members

Inherited classlikes

Attributes

Inherited from:
CharArrayReader
Supertypes
class Object
trait Matchable
class Any
class LookaheadScanner(val allowIndent: Boolean) extends Scanner

Attributes

Inherited from:
Scanner
Supertypes
class Scanner
trait TokenData
class Object
trait Matchable
class Any
Show all

Value members

Concrete methods

Attributes

Definition Classes

Inherited methods

def adjustSepRegions(lastToken: Token): Unit

Attributes

Inherited from:
Scanner

Attributes

Inherited from:
Scanner

Attributes

Inherited from:
Scanner
def charLitOr(op: => Token): Unit

Parse character literal if current character is followed by ', or follow with given op and return a symbol literal token

Parse character literal if current character is followed by ', or follow with given op and return a symbol literal token

Attributes

Inherited from:
Scanner

Attributes

Inherited from:
Scanner

Attributes

Inherited from:
ScannerCommon

Attributes

Inherited from:
Scanner

Return a list of all the comments

Return a list of all the comments

Attributes

Inherited from:
Scanner
def copyFrom(td: TokenData): Unit

Attributes

Inherited from:
TokenData
protected def decodeUni: Boolean

Switch whether unicode should be decoded

Switch whether unicode should be decoded

Attributes

Inherited from:
CharArrayReader

Attributes

Inherited from:
Scanner
def error(msg: Message, off: Offset): Unit

Generate an error at the given offset

Generate an error at the given offset

Attributes

Inherited from:
ScannerCommon
protected def error(msg: String, off: Offset): Unit

Implements CharArrayReader's error method

Implements CharArrayReader's error method

Attributes

Inherited from:
ScannerCommon
def errorButContinue(msg: Message, off: Offset): Unit

Attributes

Inherited from:
ScannerCommon

Attributes

Inherited from:
Scanner
inline def finishNamed(): Unit

Finish an IDENTIFIER with this.name.

Finish an IDENTIFIER with this.name.

Attributes

Inherited from:
ScannerCommon
def finishNamedToken(idtoken: Token, target: TokenData): Unit

Clear buffer and set name and token. If target is different from this, don't treat identifiers as end tokens.

Clear buffer and set name and token. If target is different from this, don't treat identifiers as end tokens.

Attributes

Inherited from:
ScannerCommon

Returns the closest docstring preceding the position supplied

Returns the closest docstring preceding the position supplied

Attributes

Inherited from:
Scanner
def getc(): Char

Attributes

Inherited from:
CharArrayReader
def handleNewLine(lastToken: Token): Unit

Handle newlines, possibly inserting an INDENT, OUTDENT, NEWLINE, or NEWLINES token in front of the current token. This depends on whether indentation is significant or not.

Handle newlines, possibly inserting an INDENT, OUTDENT, NEWLINE, or NEWLINES token in front of the current token. This depends on whether indentation is significant or not.

Indentation is significant if indentSyntax is set, and we are not inside a {...}, [...], (...), case ... => pair, nor in a if/while condition (i.e. currentRegion is empty).

There are three rules:

  1. Insert NEWLINE or NEWLINES if

    • the closest enclosing sepRegion is { ... } or for ... do/yield, or we are on the toplevel, i.e. currentRegion is empty, and
    • the previous token can end a statement, and
    • the current token can start a statement, and
    • the current token is not a leading infix operator, and
    • if indentation is significant then the current token starts at the current indentation width or to the right of it.

    The inserted token is NEWLINES if the current token is preceded by a whitespace line, or NEWLINE otherwise.

  2. Insert INDENT if

    • indentation is significant, and
    • the last token can start an indentation region.
    • the indentation of the current token is strictly greater than the previous indentation width, or the two widths are the same and the current token is one of : or match.

    The following tokens can start an indentation region:

    : = => <- if then else while do try catch finally for yield match throw return with

    Inserting an INDENT starts a new indentation region with the indentation of the current token as indentation width.

  3. Insert OUTDENT if

    • indentation is significant, and
    • the indentation of the current token is strictly less than the previous indentation width,
    • the current token is not a leading infix operator.

    Inserting an OUTDENT closes an indentation region. In this case, issue an error if the indentation of the current token does not match the indentation of some previous line in an enclosing indentation region.

    If a token is inserted and consumed, the original source token is still considered to start a new line, so the process that inserts an OUTDENT might repeat several times.

Indentation widths are strings consisting of spaces and tabs, ordered by the prefix relation. I.e. a <= b iff b.startsWith(a). If indentation is significant it is considered an error if the current indentation width and the indentation of the current token are incomparable.

Attributes

Inherited from:
Scanner

Is the current token in a position where a modifier is allowed?

Is the current token in a position where a modifier is allowed?

Attributes

Inherited from:
Scanner

signal an error where the input ended in the middle of a token

signal an error where the input ended in the middle of a token

Attributes

Inherited from:
ScannerCommon

The indentation width of the given offset.

The indentation width of the given offset.

Attributes

Inherited from:
Scanner
def insert(token: Token, offset: Int): Unit

Insert token at assumed offset in front of current one.

Insert token at assumed offset in front of current one.

Attributes

Inherited from:
Scanner

Is current token first one after a newline?

Is current token first one after a newline?

Attributes

Inherited from:
TokenData

Attributes

Inherited from:
TokenData

Attributes

Inherited from:
CharArrayReader

Attributes

Inherited from:
TokenData

Attributes

Inherited from:
Scanner
def isIdent(name: Name): Boolean

Attributes

Inherited from:
TokenData

Attributes

Inherited from:
TokenData
def isLeadingInfixOperator(nextWidth: IndentWidth, inConditional: Boolean): Boolean

A leading symbolic or backquoted identifier is treated as an infix operator if

A leading symbolic or backquoted identifier is treated as an infix operator if

  • it does not follow a blank line, and
  • it is followed by at least one whitespace character and a token that can start an expression.
  • if the operator appears on its own line, the next line must have at least the same indentation width as the operator. See pos/i12395 for a test where this matters. If a leading infix operator is found and the source version is 3.0-migration, emit a change warning.

Attributes

Inherited from:
Scanner

Attributes

Inherited from:
TokenData

Attributes

Inherited from:
TokenData

Attributes

Inherited from:
TokenData

Attributes

Inherited from:
ScannerCommon

Attributes

Inherited from:
TokenData

Attributes

Inherited from:
Scanner

Attributes

Inherited from:
Scanner

Attributes

Inherited from:
Scanner

Attributes

Inherited from:
TokenData

Is last character a unicode escape \uxxxx?

Is last character a unicode escape \uxxxx?

Attributes

Inherited from:
CharArrayReader

Attributes

Inherited from:
Scanner

The next token after this one. The token is computed via fetchToken, so complex two word tokens such as CASECLASS are not recognized. Newlines and indent/unindent tokens are skipped. Restriction: lookahead is illegal if the current token is INTERPOLATIONID

The next token after this one. The token is computed via fetchToken, so complex two word tokens such as CASECLASS are not recognized. Newlines and indent/unindent tokens are skipped. Restriction: lookahead is illegal if the current token is INTERPOLATIONID

Attributes

Inherited from:
Scanner

Attributes

Inherited from:
CharArrayReader

A new reader that takes off at the current character position

A new reader that takes off at the current character position

Attributes

Inherited from:
CharArrayReader

Attributes

Inherited from:
Scanner
final def nextChar(): Unit

Advance one character; reducing CR;LF pairs to just LF

Advance one character; reducing CR;LF pairs to just LF

Attributes

Inherited from:
CharArrayReader
final def nextRawChar(): Unit

Advance one character, leaving CR;LF pairs intact. This is for use in multi-line strings, so there are no "potential line ends" here.

Advance one character, leaving CR;LF pairs intact. This is for use in multi-line strings, so there are no "potential line ends" here.

Attributes

Inherited from:
CharArrayReader
def nextToken(): Unit

Produce next token, filling TokenData fields of Scanner.

Produce next token, filling TokenData fields of Scanner.

Attributes

Inherited from:
Scanner
def observeColonEOL(inTemplate: Boolean): Unit

Attributes

Inherited from:
Scanner

Attributes

Inherited from:
Scanner

Insert an token if next token closes an indentation region. Exception: continue if indentation region belongs to a match and next token is case.

Insert an token if next token closes an indentation region. Exception: continue if indentation region belongs to a match and next token is case.

Attributes

Inherited from:
Scanner
def peekAhead(): Unit

Attributes

Inherited from:
Scanner
def postProcessToken(lastToken: Token, lastName: SimpleName): Unit
  • Join CASE + CLASS => CASECLASS, CASE + OBJECT => CASEOBJECT SEMI + ELSE => ELSE, COLON following id/)/] => COLONfollow
  • Insert missing OUTDENTs at EOF

Attributes

Inherited from:
Scanner

Attributes

Inherited from:
Scanner
final def printState(): Unit

Attributes

Inherited from:
Scanner
protected def putChar(c: Char): Unit

append Unicode character to "litBuf" buffer

append Unicode character to "litBuf" buffer

Attributes

Inherited from:
ScannerCommon

Attributes

Inherited from:
ScannerCommon
def reset(): Unit

Attributes

Inherited from:
Scanner
def resume(lastTokenData: TokenData): Unit

Attributes

Inherited from:
Scanner
def setStrVal(): Unit

Clear buffer and set string

Clear buffer and set string

Attributes

Inherited from:
ScannerCommon
def show: String

Attributes

Inherited from:
Scanner
def skip(): Unit

Skip on error to next safe point.

Skip on error to next safe point.

Attributes

Inherited from:
Scanner
final def skipParens(multiple: Boolean): Unit

Skip matching pairs of (...) or [...] parentheses.

Skip matching pairs of (...) or [...] parentheses.

Attributes

Inherited from:
Scanner
def skipToken[T](result: T): T

Attributes

Inherited from:
Scanner
def skipToken(): Offset

read next token and return last offset

read next token and return last offset

Attributes

Inherited from:
Scanner

Attributes

Inherited from:
ScannerCommon
def spaceTabMismatchMsg(lastWidth: IndentWidth, nextWidth: IndentWidth): Message

Attributes

Inherited from:
Scanner
override def toString: String

Returns a string representation of the object.

Returns a string representation of the object.

The default representation is platform dependent.

Attributes

Returns

a string representation of the object.

Definition Classes
Scanner -> Any
Inherited from:
Scanner
def toToken(identifier: SimpleName): Token

The token for given name. Either IDENTIFIER or a keyword.

The token for given name. Either IDENTIFIER or a keyword.

Attributes

Inherited from:
Scanner

Concrete fields

Inherited fields

var base: Int

the base of a number

the base of a number

Attributes

Inherited from:
TokenData
val buf: Array[Char]

Attributes

Inherited from:
ScannerCommon
var ch: Char

the last read character

the last read character

Attributes

Inherited from:
CharArrayReader

The offset one past the last read character

The offset one past the last read character

Attributes

Inherited from:
CharArrayReader

The current region. This is initially an Indented region with zero indentation width.

The current region. This is initially an Indented region with zero indentation width.

Attributes

Inherited from:
Scanner

Attributes

Inherited from:
Scanner

the last error offset

the last error offset

Attributes

Inherited from:
ScannerCommon

Attributes

Inherited from:
Scanner

Attributes

Inherited from:
Scanner

The offset before the last read character

The offset before the last read character

Attributes

Inherited from:
CharArrayReader

the offset of the character following the token preceding this one

the offset of the character following the token preceding this one

Attributes

Inherited from:
TokenData

the offset of the newline immediately preceding the token, or -1 if token is not preceded by a newline.

the offset of the newline immediately preceding the token, or -1 if token is not preceded by a newline.

Attributes

Inherited from:
TokenData

The start offset of the current line

The start offset of the current line

Attributes

Inherited from:
CharArrayReader
protected val litBuf: CharBuffer

A character buffer for literals

A character buffer for literals

Attributes

Inherited from:
ScannerCommon

the name of an identifier

the name of an identifier

Attributes

Inherited from:
TokenData

Attributes

Inherited from:
Scanner

We need one token lookahead and one token history

We need one token lookahead and one token history

Attributes

Inherited from:
Scanner

Attributes

Inherited from:
Scanner
var offset: Offset

the offset of the first character of the current token

the offset of the first character of the current token

Attributes

Inherited from:
TokenData

Attributes

Inherited from:
Scanner

Attributes

Inherited from:
Scanner

Attributes

Inherited from:
Scanner

Attributes

Inherited from:
Scanner

Attributes

Inherited from:
Scanner
override val startFrom: Offset

Attributes

Inherited from:
Scanner
var strVal: String

the string value of a literal

the string value of a literal

Attributes

Inherited from:
TokenData
var token: Token

the next token

the next token

Attributes

Inherited from:
TokenData