org.antlr.v4.runtime

Interface Token

All Known Subinterfaces:

WritableToken

All Known Implementing Classes:

CommonToken, RuleTagToken, TokenTagToken

public interface Token

A token has properties: text, type, line, character position in the line (so we can ignore tabs), token channel, index, and source from which we obtained this token.

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_CHANNEL All tokens go to the parser (unless skip() is called in that rule) on a particular "channel".
static int	EOF
static int	EPSILON During lookahead operations, this "token" signifies we hit rule end ATN state and did not follow it despite needing to.
static int	HIDDEN_CHANNEL Anything on different channel than DEFAULT_CHANNEL is not parsed by parser.
static int	INVALID_TYPE
static int	MIN_USER_TOKEN_TYPE

Method Summary

Methods

Modifier and Type	Method and Description
int	getChannel() Return the channel this token.
int	getCharPositionInLine() The index of the first character of this token relative to the beginning of the line at which it occurs, 0..n-1
CharStream	getInputStream() Gets the CharStream from which this token was derived.
int	getLine() The line number on which the 1st character of this token was matched, line=1..n
int	getStartIndex() The starting character index of the token This method is optional; return -1 if not implemented.
int	getStopIndex() The last character index of the token.
String	getText() Get the text of the token.
int	getTokenIndex() An index from 0..n-1 of the token object in the input stream.
TokenSource	getTokenSource() Gets the TokenSource which created this token.
int	getType() Get the token type of the token

Field Detail

INVALID_TYPE

static final int INVALID_TYPE

See Also:

Constant Field Values

EPSILON

static final int EPSILON

During lookahead operations, this "token" signifies we hit rule end ATN state and did not follow it despite needing to.

See Also:

Constant Field Values

MIN_USER_TOKEN_TYPE

static final int MIN_USER_TOKEN_TYPE

See Also:

Constant Field Values

EOF

static final int EOF

See Also:

Constant Field Values

DEFAULT_CHANNEL

static final int DEFAULT_CHANNEL

All tokens go to the parser (unless skip() is called in that rule) on a particular "channel". The parser tunes to a particular channel so that whitespace etc... can go to the parser on a "hidden" channel.

See Also:

Constant Field Values

HIDDEN_CHANNEL

static final int HIDDEN_CHANNEL

Anything on different channel than DEFAULT_CHANNEL is not parsed by parser.

See Also:

Constant Field Values

Method Detail

getText

String getText()

Get the text of the token.

getType

int getType()

Get the token type of the token

getLine

int getLine()

The line number on which the 1st character of this token was matched, line=1..n

getCharPositionInLine

int getCharPositionInLine()

The index of the first character of this token relative to the beginning of the line at which it occurs, 0..n-1

getChannel

int getChannel()

Return the channel this token. Each token can arrive at the parser on a different channel, but the parser only "tunes" to a single channel. The parser ignores everything not on DEFAULT_CHANNEL.

getTokenIndex

int getTokenIndex()

An index from 0..n-1 of the token object in the input stream. This must be valid in order to print token streams and use TokenRewriteStream. Return -1 to indicate that this token was conjured up since it doesn't have a valid index.

getStartIndex

int getStartIndex()

The starting character index of the token This method is optional; return -1 if not implemented.

getStopIndex

int getStopIndex()

The last character index of the token. This method is optional; return -1 if not implemented.

getTokenSource

TokenSource getTokenSource()

Gets the TokenSource which created this token.

getInputStream

CharStream getInputStream()

Gets the CharStream from which this token was derived.