Class UnbufferedCharStream
- java.lang.Object
-
- org.antlr.v4.runtime.UnbufferedCharStream
-
- All Implemented Interfaces:
CharStream
,IntStream
public class UnbufferedCharStream extends Object implements CharStream
Do not buffer up the entire char stream. It does keep a small buffer for efficiency and also buffers while a mark exists (set by the lookahead prediction in parser). "Unbuffered" here refers to fact that it doesn't buffer all data, not that's it's on demand loading of char. Before 4.7, this class used the default environment encoding to convert bytes to UTF-16, and held the UTF-16 bytes in the buffer as chars. As of 4.7, the class uses UTF-8 by default, and the buffer holds Unicode code points in the buffer as ints.
-
-
Field Summary
Fields Modifier and Type Field Description protected int
currentCharIndex
Absolute character index.protected int[]
data
A moving window buffer of the data being scanned.protected Reader
input
protected int
lastChar
This is theLA(-1)
character for the current position.protected int
lastCharBufferStart
protected int
n
The number of characters currently indata
.String
name
The name or source of this char stream.protected int
numMarkers
protected int
p
0..n-1 index intodata
of next character.-
Fields inherited from interface org.antlr.v4.runtime.IntStream
EOF, UNKNOWN_SOURCE_NAME
-
-
Constructor Summary
Constructors Constructor Description UnbufferedCharStream()
Useful for subclasses that pull char from other than this.input.UnbufferedCharStream(int bufferSize)
Useful for subclasses that pull char from other than this.input.UnbufferedCharStream(InputStream input)
UnbufferedCharStream(InputStream input, int bufferSize)
UnbufferedCharStream(InputStream input, int bufferSize, Charset charset)
UnbufferedCharStream(Reader input)
UnbufferedCharStream(Reader input, int bufferSize)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description protected void
add(int c)
void
consume()
Consumes the current symbol in the stream.protected int
fill(int n)
Addn
characters to the buffer.protected int
getBufferStartIndex()
String
getSourceName()
Gets the name of the underlying symbol source.String
getText(Interval interval)
This method returns the text for a range of characters within this input stream.int
index()
Return the index into the stream of the input symbol referred to byLA(1)
.int
LA(int i)
Gets the value of the symbol at offseti
from the current position.int
mark()
Return a marker that we can release later.protected int
nextChar()
Override to provide different source of characters thaninput
.void
release(int marker)
Decrement number of markers, resetting buffer if we hit 0.void
seek(int index)
Seek to absolute character index, which might not be in the current sliding window.int
size()
Returns the total number of symbols in the stream, including a single EOF symbol.protected void
sync(int want)
Make sure we have 'need' elements from current positionp
.
-
-
-
Field Detail
-
data
protected int[] data
A moving window buffer of the data being scanned. While there's a marker, we keep adding to buffer. Otherwise,consume()
resets so we start filling at index 0 again.
-
n
protected int n
The number of characters currently indata
.This is not the buffer capacity, that's
data.length
.
-
p
protected int p
0..n-1 index intodata
of next character.The
LA(1)
character isdata[p]
. Ifp == n
, we are out of buffered characters.
-
numMarkers
protected int numMarkers
-
lastChar
protected int lastChar
This is theLA(-1)
character for the current position.
-
lastCharBufferStart
protected int lastCharBufferStart
WhennumMarkers > 0
, this is theLA(-1)
character for the first character indata
. Otherwise, this is unspecified.
-
currentCharIndex
protected int currentCharIndex
Absolute character index. It's the index of the character about to be read viaLA(1)
. Goes from 0 to the number of characters in the entire stream, although the stream size is unknown before the end is reached.
-
input
protected Reader input
-
name
public String name
The name or source of this char stream.
-
-
Constructor Detail
-
UnbufferedCharStream
public UnbufferedCharStream()
Useful for subclasses that pull char from other than this.input.
-
UnbufferedCharStream
public UnbufferedCharStream(int bufferSize)
Useful for subclasses that pull char from other than this.input.
-
UnbufferedCharStream
public UnbufferedCharStream(InputStream input)
-
UnbufferedCharStream
public UnbufferedCharStream(Reader input)
-
UnbufferedCharStream
public UnbufferedCharStream(InputStream input, int bufferSize)
-
UnbufferedCharStream
public UnbufferedCharStream(InputStream input, int bufferSize, Charset charset)
-
UnbufferedCharStream
public UnbufferedCharStream(Reader input, int bufferSize)
-
-
Method Detail
-
consume
public void consume()
Description copied from interface:IntStream
Consumes the current symbol in the stream. This method has the following effects:- Forward movement: The value of
index()
before calling this method is less than the value ofindex()
after calling this method. - Ordered lookahead: The value of
LA(1)
before calling this method becomes the value ofLA(-1)
after calling this method.
index()
is incremented by exactly 1, as that would preclude the ability to implement filtering streams (e.g.CommonTokenStream
which distinguishes between "on-channel" and "off-channel" tokens). - Forward movement: The value of
-
sync
protected void sync(int want)
Make sure we have 'need' elements from current positionp
. Last validp
index isdata.length-1
.p+need-1
is the char index 'need' elements ahead. If we need 1 element,(p+1-1)==p
must be less thandata.length
.
-
fill
protected int fill(int n)
Addn
characters to the buffer. Returns the number of characters actually added to the buffer. If the return value is less thann
, then EOF was reached beforen
characters could be added.
-
nextChar
protected int nextChar() throws IOException
Override to provide different source of characters thaninput
.- Throws:
IOException
-
add
protected void add(int c)
-
LA
public int LA(int i)
Description copied from interface:IntStream
Gets the value of the symbol at offseti
from the current position. Wheni==1
, this method returns the value of the current symbol in the stream (which is the next symbol to be consumed). Wheni==-1
, this method returns the value of the previously read symbol in the stream. It is not valid to call this method withi==0
, but the specific behavior is unspecified because this method is frequently called from performance-critical code.This method is guaranteed to succeed if any of the following are true:
i>0
i==-1
andindex()
returns a value greater than the value ofindex()
after the stream was constructed andLA(1)
was called in that order. Specifying the currentindex()
relative to the index after the stream was created allows for filtering implementations that do not return every symbol from the underlying source. Specifying the call toLA(1)
allows for lazily initialized streams.LA(i)
refers to a symbol consumed within a marked region that has not yet been released.
If
i
represents a position at or beyond the end of the stream, this method returnsIntStream.EOF
.The return value is unspecified if
i<0
and fewer than-i
calls toconsume()
have occurred from the beginning of the stream before calling this method.
-
mark
public int mark()
Return a marker that we can release later.The specific marker value used for this class allows for some level of protection against misuse where
seek()
is called on a mark orrelease()
is called in the wrong order.
-
release
public void release(int marker)
Decrement number of markers, resetting buffer if we hit 0.- Specified by:
release
in interfaceIntStream
- Parameters:
marker
-- See Also:
IntStream.mark()
-
index
public int index()
Description copied from interface:IntStream
Return the index into the stream of the input symbol referred to byLA(1)
.The behavior of this method is unspecified if no call to an
initializing method
has occurred after this stream was constructed.
-
seek
public void seek(int index)
Seek to absolute character index, which might not be in the current sliding window. Movep
toindex-bufferStartIndex
.
-
size
public int size()
Description copied from interface:IntStream
Returns the total number of symbols in the stream, including a single EOF symbol.
-
getSourceName
public String getSourceName()
Description copied from interface:IntStream
Gets the name of the underlying symbol source. This method returns a non-null, non-empty string. If such a name is not known, this method returnsIntStream.UNKNOWN_SOURCE_NAME
.- Specified by:
getSourceName
in interfaceIntStream
-
getText
public String getText(Interval interval)
Description copied from interface:CharStream
This method returns the text for a range of characters within this input stream. This method is guaranteed to not throw an exception if the specifiedinterval
lies entirely within a marked range. For more information about marked ranges, seeIntStream.mark()
.- Specified by:
getText
in interfaceCharStream
- Parameters:
interval
- an interval within the stream- Returns:
- the text of the specified interval
-
getBufferStartIndex
protected final int getBufferStartIndex()
-
-