public class CommonToken extends Object implements WritableToken, Serializable
Modifier and Type | Field and Description |
---|---|
protected int |
channel
This is the backing field for
getChannel() and
setChannel(int) . |
protected int |
charPositionInLine
This is the backing field for
getCharPositionInLine() and
setCharPositionInLine(int) . |
protected static Pair<TokenSource,CharStream> |
EMPTY_SOURCE
|
protected int |
index
This is the backing field for
getTokenIndex() and
setTokenIndex(int) . |
protected int |
line
This is the backing field for
getLine() and setLine(int) . |
protected Pair<TokenSource,CharStream> |
source
This is the backing field for
getTokenSource() and
getInputStream() . |
protected int |
start
This is the backing field for
getStartIndex() and
setStartIndex(int) . |
protected int |
stop
This is the backing field for
getStopIndex() and
setStopIndex(int) . |
protected String |
text
This is the backing field for
getText() when the token text is
explicitly set in the constructor or via setText(java.lang.String) . |
protected int |
type
This is the backing field for
getType() and setType(int) . |
DEFAULT_CHANNEL, EOF, EPSILON, HIDDEN_CHANNEL, INVALID_TYPE, MIN_USER_CHANNEL_VALUE, MIN_USER_TOKEN_TYPE
Constructor and Description |
---|
CommonToken(int type)
Constructs a new
CommonToken with the specified token type. |
CommonToken(int type,
String text)
Constructs a new
CommonToken with the specified token type and
text. |
CommonToken(Pair<TokenSource,CharStream> source,
int type,
int channel,
int start,
int stop) |
CommonToken(Token oldToken)
Constructs a new
CommonToken as a copy of another Token . |
Modifier and Type | Method and Description |
---|---|
int |
getChannel()
Return the channel this token.
|
int |
getCharPositionInLine()
The index of the first character of this token relative to the
beginning of the line at which it occurs, 0..n-1
|
CharStream |
getInputStream()
Gets the
CharStream from which this token was derived. |
int |
getLine()
The line number on which the 1st character of this token was matched,
line=1..n
|
int |
getStartIndex()
The starting character index of the token
This method is optional; return -1 if not implemented.
|
int |
getStopIndex()
The last character index of the token.
|
String |
getText()
Get the text of the token.
|
int |
getTokenIndex()
An index from 0..n-1 of the token object in the input stream.
|
TokenSource |
getTokenSource()
Gets the
TokenSource which created this token. |
int |
getType()
Get the token type of the token
|
void |
setChannel(int channel) |
void |
setCharPositionInLine(int charPositionInLine) |
void |
setLine(int line) |
void |
setStartIndex(int start) |
void |
setStopIndex(int stop) |
void |
setText(String text)
Explicitly set the text for this token.
|
void |
setTokenIndex(int index) |
void |
setType(int type) |
String |
toString() |
protected static final Pair<TokenSource,CharStream> EMPTY_SOURCE
protected int type
getType()
and setType(int)
.protected int line
getLine()
and setLine(int)
.protected int charPositionInLine
getCharPositionInLine()
and
setCharPositionInLine(int)
.protected int channel
getChannel()
and
setChannel(int)
.protected Pair<TokenSource,CharStream> source
getTokenSource()
and
getInputStream()
.
These properties share a field to reduce the memory footprint of
CommonToken
. Tokens created by a CommonTokenFactory
from
the same source and input stream share a reference to the same
Pair
containing these values.
protected String text
getText()
when the token text is
explicitly set in the constructor or via setText(java.lang.String)
.getText()
protected int index
getTokenIndex()
and
setTokenIndex(int)
.protected int start
getStartIndex()
and
setStartIndex(int)
.protected int stop
getStopIndex()
and
setStopIndex(int)
.public CommonToken(int type)
CommonToken
with the specified token type.type
- The token type.public CommonToken(Pair<TokenSource,CharStream> source, int type, int channel, int start, int stop)
public CommonToken(int type, String text)
CommonToken
with the specified token type and
text.type
- The token type.text
- The text of the token.public CommonToken(Token oldToken)
CommonToken
as a copy of another Token
.
If oldToken
is also a CommonToken
instance, the newly
constructed token will share a reference to the text
field and
the Pair
stored in source
. Otherwise, text
will
be assigned the result of calling getText()
, and source
will be constructed from the result of Token.getTokenSource()
and
Token.getInputStream()
.
oldToken
- The token to copy.public int getType()
Token
public void setLine(int line)
setLine
in interface WritableToken
public void setText(String text)
null
, then getText()
will return this value rather than
extracting the text from the input.setText
in interface WritableToken
text
- The explicit text of the token, or null
if the text
should be obtained from the input along with the start and stop indexes
of the token.public int getLine()
Token
public int getCharPositionInLine()
Token
getCharPositionInLine
in interface Token
public void setCharPositionInLine(int charPositionInLine)
setCharPositionInLine
in interface WritableToken
public int getChannel()
Token
getChannel
in interface Token
public void setChannel(int channel)
setChannel
in interface WritableToken
public void setType(int type)
setType
in interface WritableToken
public int getStartIndex()
Token
getStartIndex
in interface Token
public void setStartIndex(int start)
public int getStopIndex()
Token
getStopIndex
in interface Token
public void setStopIndex(int stop)
public int getTokenIndex()
Token
getTokenIndex
in interface Token
public void setTokenIndex(int index)
setTokenIndex
in interface WritableToken
public TokenSource getTokenSource()
Token
TokenSource
which created this token.getTokenSource
in interface Token
public CharStream getInputStream()
Token
CharStream
from which this token was derived.getInputStream
in interface Token
Copyright © 1992-2015 ANTLR. All Rights Reserved.