scala3-compiler-bootstrapped
API
Generated with
Copyright (c) 2002-2023, LAMP/EPFL
Copyright (c) 2002-2023, LAMP/EPFL
scala3-compiler-bootstrapped/dotty.tools/dotty.tools.dotc/dotty.tools.dotc.parsing/Scanners/Scanner

Scanner

dotty.tools.dotc.parsing.Scanners$.Scanner
class Scanner(source: SourceFile, val startFrom: Offset, profile: Profile, allowIndent: Boolean)(using x$5: Context) extends ScannerCommon

Attributes

Graph
Supertypes
trait TokenData
class Object
trait Matchable
class Any
Show all

Members list

Type members

Inherited classlikes

class CharArrayLookaheadReader extends CharArrayReader

Attributes

Inherited from:
CharArrayReader
Supertypes
class Object
trait Matchable
class Any

Value members

Concrete methods

def adjustSepRegions(lastToken: Token): Unit
def charLitOr(op: => Token): Unit

Parse character literal if current character is followed by ', or follow with given op and return a symbol literal token

Parse character literal if current character is followed by ', or follow with given op and return a symbol literal token

Attributes

def commentSpans: List[Span]

Return a list of all the comment positions

Return a list of all the comment positions

Attributes

def getDocComment(pos: Int): Option[Comment]

Returns the closest docstring preceding the position supplied

Returns the closest docstring preceding the position supplied

Attributes

def handleNewLine(lastToken: Token): Unit

Handle newlines, possibly inserting an INDENT, OUTDENT, NEWLINE, or NEWLINES token in front of the current token. This depends on whether indentation is significant or not.

Handle newlines, possibly inserting an INDENT, OUTDENT, NEWLINE, or NEWLINES token in front of the current token. This depends on whether indentation is significant or not.

Indentation is significant if indentSyntax is set, and we are not inside a {...}, [...], (...), case ... => pair, nor in a if/while condition (i.e. currentRegion is empty).

There are three rules:

  1. Insert NEWLINE or NEWLINES if

    • the closest enclosing sepRegion is { ... } or for ... do/yield, or we are on the toplevel, i.e. currentRegion is empty, and
    • the previous token can end a statement, and
    • the current token can start a statement, and
    • the current token is not a leading infix operator, and
    • if indentation is significant then the current token starts at the current indentation width or to the right of it.

    The inserted token is NEWLINES if the current token is preceded by a whitespace line, or NEWLINE otherwise.

  2. Insert INDENT if

    • indentation is significant, and
    • the last token can start an indentation region.
    • the indentation of the current token is strictly greater than the previous indentation width, or the two widths are the same and the current token is one of : or match.

    The following tokens can start an indentation region:

    : = => <- if then else while do try catch finally for yield match

    Inserting an INDENT starts a new indentation region with the indentation of the current token as indentation width.

  3. Insert OUTDENT if

    • indentation is significant, and
    • the indentation of the current token is strictly less than the previous indentation width,
    • the current token is not a leading infix operator.

    Inserting an OUTDENT closes an indentation region. In this case, issue an error if the indentation of the current token does not match the indentation of some previous line in an enclosing indentation region.

    If a token is inserted and consumed, the original source token is still considered to start a new line, so the process that inserts an OUTDENT might repeat several times.

Indentation widths are strings consisting of spaces and tabs, ordered by the prefix relation. I.e. a <= b iff b.startsWith(a). If indentation is significant it is considered an error if the current indentation width and the indentation of the current token are incomparable.

Attributes

def inModifierPosition(): Boolean

Is the current token in a position where a modifier is allowed?

Is the current token in a position where a modifier is allowed?

Attributes

def indentWidth(offset: Offset): IndentWidth

The indentation width of the given offset.

The indentation width of the given offset.

Attributes

def insert(token: Token, offset: Int): Unit

Insert token at assumed offset in front of current one.

Insert token at assumed offset in front of current one.

Attributes

def isLeadingInfixOperator(nextWidth: IndentWidth, inConditional: Boolean): Boolean

A leading symbolic or backquoted identifier is treated as an infix operator if

A leading symbolic or backquoted identifier is treated as an infix operator if

  • it does not follow a blank line, and
  • it is followed by at least one whitespace character and a token that can start an expression.
  • if the operator appears on its own line, the next line must have at least the same indentation width as the operator. See pos/i12395 for a test where this matters. If a leading infix operator is found and the source version is 3.0-migration, emit a change warning.

Attributes

def lookahead: TokenData

The next token after this one. The token is computed via fetchToken, so complex two word tokens such as CASECLASS are not recognized. Newlines and indent/unindent tokens are skipped. Restriction: lookahead is illegal if the current token is INTERPOLATIONID

The next token after this one. The token is computed via fetchToken, so complex two word tokens such as CASECLASS are not recognized. Newlines and indent/unindent tokens are skipped. Restriction: lookahead is illegal if the current token is INTERPOLATIONID

Attributes

def nextToken(): Unit

Produce next token, filling TokenData fields of Scanner.

Produce next token, filling TokenData fields of Scanner.

Attributes

def observeColonEOL(inTemplate: Boolean): Unit
def observeOutdented(): Unit

Insert an token if next token closes an indentation region. Exception: continue if indentation region belongs to a match and next token is case.

Insert an token if next token closes an indentation region. Exception: continue if indentation region belongs to a match and next token is case.

Attributes

def peekAhead(): Unit
def postProcessToken(lastToken: Token, lastName: SimpleName): Unit
  • Join CASE + CLASS => CASECLASS, CASE + OBJECT => CASEOBJECT SEMI + ELSE => ELSE, COLON following id/)/] => COLONfollow
  • Insert missing OUTDENTs at EOF

Attributes

final def printState(): Unit
def reset(): Unit
def resume(lastTokenData: TokenData): Unit
def show: String
def skip(): Unit

Skip on error to next safe point.

Skip on error to next safe point.

Attributes

final def skipParens(multiple: Boolean): Unit

Skip matching pairs of (...) or [...] parentheses.

Skip matching pairs of (...) or [...] parentheses.

Attributes

def skipToken(): Offset

read next token and return last offset

read next token and return last offset

Attributes

def skipToken[T](result: T): T
def spaceTabMismatchMsg(lastWidth: IndentWidth, nextWidth: IndentWidth): Message
override def toString: String

Returns a string representation of the object.

Returns a string representation of the object.

The default representation is platform dependent.

Attributes

Returns

a string representation of the object.

Definition Classes
Any
def toToken(identifier: SimpleName): Token

The token for given name. Either IDENTIFIER or a keyword.

The token for given name. Either IDENTIFIER or a keyword.

Attributes

Inherited methods

def checkNoTrailingSeparator(): Unit

Attributes

Inherited from:
ScannerCommon
def copyFrom(td: TokenData): Unit

Attributes

Inherited from:
TokenData
protected def decodeUni: Boolean

Switch whether unicode should be decoded

Switch whether unicode should be decoded

Attributes

Inherited from:
CharArrayReader
def error(msg: Message, off: Offset): Unit

Generate an error at the given offset

Generate an error at the given offset

Attributes

Inherited from:
ScannerCommon
protected def error(msg: String, off: Offset): Unit

Implements CharArrayReader's error method

Implements CharArrayReader's error method

Attributes

Inherited from:
ScannerCommon
def errorButContinue(msg: Message, off: Offset): Unit

Attributes

Inherited from:
ScannerCommon
inline def finishNamed(): Unit

Finish an IDENTIFIER with this.name.

Finish an IDENTIFIER with this.name.

Attributes

Inherited from:
ScannerCommon
def finishNamedToken(idtoken: Token, target: TokenData): Unit

Clear buffer and set name and token. If target is different from this, don't treat identifiers as end tokens.

Clear buffer and set name and token. If target is different from this, don't treat identifiers as end tokens.

Attributes

Inherited from:
ScannerCommon
def getc(): Char

Attributes

Inherited from:
CharArrayReader
def incompleteInputError(msg: Message): Unit

signal an error where the input ended in the middle of a token

signal an error where the input ended in the middle of a token

Attributes

Inherited from:
ScannerCommon
protected def initialCharBufferSize: Int

Attributes

Inherited from:
ScannerCommon
def isAfterLineEnd: Boolean

Is current token first one after a newline?

Is current token first one after a newline?

Attributes

Inherited from:
TokenData
def isArrow: Boolean

Attributes

Inherited from:
TokenData
def isAtEnd: Boolean

Attributes

Inherited from:
CharArrayReader
def isColon: Boolean

Attributes

Inherited from:
TokenData
def isIdent(name: Name): Boolean

Attributes

Inherited from:
TokenData
def isIdent: Boolean

Attributes

Inherited from:
TokenData
def isNestedEnd: Boolean

Attributes

Inherited from:
TokenData
def isNestedStart: Boolean

Attributes

Inherited from:
TokenData
def isNewLine: Boolean

Attributes

Inherited from:
TokenData
def isNumberSeparator(c: Char): Boolean

Attributes

Inherited from:
ScannerCommon
def isOperator: Boolean

Attributes

Inherited from:
TokenData
def isStatSep: Boolean

Attributes

Inherited from:
TokenData
def isUnicodeEscape: Boolean

Is last character a unicode escape \uxxxx?

Is last character a unicode escape \uxxxx?

Attributes

Inherited from:
CharArrayReader
def lookaheadChar(): Char

Attributes

Inherited from:
CharArrayReader
def lookaheadReader(): CharArrayLookaheadReader

A new reader that takes off at the current character position

A new reader that takes off at the current character position

Attributes

Inherited from:
CharArrayReader
final def nextChar(): Unit

Advance one character; reducing CR;LF pairs to just LF

Advance one character; reducing CR;LF pairs to just LF

Attributes

Inherited from:
CharArrayReader
final def nextRawChar(): Unit

Advance one character, leaving CR;LF pairs intact. This is for use in multi-line strings, so there are no "potential line ends" here.

Advance one character, leaving CR;LF pairs intact. This is for use in multi-line strings, so there are no "potential line ends" here.

Attributes

Inherited from:
CharArrayReader
protected def putChar(c: Char): Unit

append Unicode character to "litBuf" buffer

append Unicode character to "litBuf" buffer

Attributes

Inherited from:
ScannerCommon
def removeNumberSeparators(s: String): String

Attributes

Inherited from:
ScannerCommon
def setStrVal(): Unit

Clear buffer and set string

Clear buffer and set string

Attributes

Inherited from:
ScannerCommon
def sourcePos(off: Offset): SourcePosition

Attributes

Inherited from:
ScannerCommon

Concrete fields

var currentRegion: Region

The current region. This is initially an Indented region with zero indentation width.

The current region. This is initially an Indented region with zero indentation width.

Attributes

val next: TokenData

We need one token lookahead and one token history

We need one token lookahead and one token history

Attributes

override val startFrom: Offset

Inherited fields

var base: Int

the base of a number

the base of a number

Attributes

Inherited from:
TokenData
val buf: Array[Char]

Attributes

Inherited from:
ScannerCommon
var ch: Char

the last read character

the last read character

Attributes

Inherited from:
CharArrayReader
var charOffset: Int

The offset one past the last read character

The offset one past the last read character

Attributes

Inherited from:
CharArrayReader
var errOffset: Offset

the last error offset

the last error offset

Attributes

Inherited from:
ScannerCommon
var lastCharOffset: Int

The offset before the last read character

The offset before the last read character

Attributes

Inherited from:
CharArrayReader
var lastOffset: Offset

the offset of the character following the token preceding this one

the offset of the character following the token preceding this one

Attributes

Inherited from:
TokenData
var lineOffset: Offset

the offset of the newline immediately preceding the token, or -1 if token is not preceded by a newline.

the offset of the newline immediately preceding the token, or -1 if token is not preceded by a newline.

Attributes

Inherited from:
TokenData
var lineStartOffset: Int

The start offset of the current line

The start offset of the current line

Attributes

Inherited from:
CharArrayReader
protected val litBuf: CharBuffer

A character buffer for literals

A character buffer for literals

Attributes

Inherited from:
ScannerCommon
var name: SimpleName

the name of an identifier

the name of an identifier

Attributes

Inherited from:
TokenData
var offset: Offset

the offset of the first character of the current token

the offset of the first character of the current token

Attributes

Inherited from:
TokenData
var strVal: String

the string value of a literal

the string value of a literal

Attributes

Inherited from:
TokenData
var token: Token

the next token

the next token

Attributes

Inherited from:
TokenData
Generated with
Copyright (c) 2002-2023, LAMP/EPFL
Copyright (c) 2002-2023, LAMP/EPFL