org.jruby.lexer.yacc

Class LexerSource

java.lang.Object

org.jruby.lexer.yacc.LexerSource

Direct Known Subclasses:

ByteArrayLexerSource, InputStreamLexerSource

public abstract class LexerSource
extends Object

This class is what feeds the lexer. It is primarily a wrapper around a Reader that can unread() data back onto the source. Originally, I thought about using the PushBackReader to handle read/unread, but I realized that some extremely pathological case could overflow the pushback buffer. Better safe than sorry. I could have combined this implementation with a PushbackBuffer, but the added complexity did not seem worth it.

Field Summary

Fields

Modifier and Type	Field and Description
protected ISourcePosition	lastPosition
protected int	line
protected int	lineOffset
protected int	offset

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	LexerSource(String sourceName, List<String> list, int lineOffset) Create our food-source for the lexer

Method Summary

Methods

Modifier and Type	Method and Description
protected void	captureFeature(int c)
String	getCurrentLine()
String	getFilename() What file are we lexing?
int	getLine() What line are we at?
int	getOffset() The location of the last byte we read from the source.
ISourcePosition	getPosition() Where is the reader within the source {filename,row} Node that this method updates some internal state - it's not just a constructor
ISourcePosition	getPosition(ISourcePosition startPosition) Where is the reader within the source {filename,row} Node that this method updates some internal state - it's not just a constructor
abstract InputStream	getRemainingAsStream()
static LexerSource	getSource(String name, byte[] content, List<String> list, ParserConfiguration configuration)
static LexerSource	getSource(String name, InputStream content, List<String> list, ParserConfiguration configuration) Create a source.
int	getVirtualLine()
abstract boolean	lastWasBeginOfLine()
protected String	makePointer(int length)
abstract boolean	matchMarker(org.jruby.util.ByteList marker, boolean indent, boolean withNewline) Match marker against input consumering lexer source as it goes...Unless it does not match then it reverts lexer source back to point when this method was invoked.
abstract boolean	peek(int c)
abstract int	read()
int	readCodepoint(int first, org.jcodings.Encoding encoding)
abstract org.jruby.util.ByteList	readLineBytes()
abstract org.jruby.util.ByteList	readUntil(char c)
abstract int	skipUntil(int c)
void	startOfToken()
protected void	uncaptureFeature(int c)
abstract void	unread(int c)
abstract void	unreadMany(CharSequence line)
abstract boolean	wasBeginOfLine()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

line

protected int line

lineOffset

protected int lineOffset

offset

protected int offset

lastPosition

protected ISourcePosition lastPosition

Constructor Detail

LexerSource

protected LexerSource(String sourceName,
           List<String> list,
           int lineOffset)

Create our food-source for the lexer

Parameters:

sourceName - is the file we are reading

Method Detail

getFilename

public String getFilename()

What file are we lexing?

Returns:

the files name

getLine

public int getLine()

What line are we at?

Returns:

the line number 0...line_size-1

getVirtualLine

public int getVirtualLine()

getOffset

public int getOffset()

The location of the last byte we read from the source.

Returns:

current location of source

startOfToken

public void startOfToken()

getPosition

public ISourcePosition getPosition(ISourcePosition startPosition)

Where is the reader within the source {filename,row} Node that this method updates some internal state - it's not just a constructor

Returns:

the current position

getPosition

public ISourcePosition getPosition()

Where is the reader within the source {filename,row} Node that this method updates some internal state - it's not just a constructor

Returns:

the current position

getSource

public static LexerSource getSource(String name,
                    InputStream content,
                    List<String> list,
                    ParserConfiguration configuration)

Create a source.

Parameters:

name - the name of the source (e.g a filename: foo.rb)

content - the data of the source

Returns:

the new source

getSource

public static LexerSource getSource(String name,
                    byte[] content,
                    List<String> list,
                    ParserConfiguration configuration)

captureFeature

protected void captureFeature(int c)

uncaptureFeature

protected void uncaptureFeature(int c)

getCurrentLine

public String getCurrentLine()

makePointer

protected String makePointer(int length)

readCodepoint

public int readCodepoint(int first,
                org.jcodings.Encoding encoding)
                  throws IOException

Throws:

IOException

matchMarker

public abstract boolean matchMarker(org.jruby.util.ByteList marker,
                  boolean indent,
                  boolean withNewline)
                             throws IOException

Match marker against input consumering lexer source as it goes...Unless it does not match then it reverts lexer source back to point when this method was invoked.

Parameters:

marker - to match against

indent - eat any leading whitespace

withNewline - includes a check that marker is followed by newline or EOF

Returns:

true if marker matches...false otherwise

Throws:

IOException - if an error occurred reading from underlying IO source

read

public abstract int read()
                  throws IOException

Throws:

IOException

readUntil

public abstract org.jruby.util.ByteList readUntil(char c)
                                           throws IOException

Throws:

IOException

readLineBytes

public abstract org.jruby.util.ByteList readLineBytes()
                                               throws IOException

Throws:

IOException

skipUntil

public abstract int skipUntil(int c)
                       throws IOException

Throws:

IOException

unread

public abstract void unread(int c)

unreadMany

public abstract void unreadMany(CharSequence line)

peek

public abstract boolean peek(int c)
                      throws IOException

Throws:

IOException

lastWasBeginOfLine

public abstract boolean lastWasBeginOfLine()

wasBeginOfLine

public abstract boolean wasBeginOfLine()

getRemainingAsStream

public abstract InputStream getRemainingAsStream()
                                          throws IOException

Throws:

IOException