org.apache.cassandra.index.sasi.analyzer

Interface StandardTokenizerInterface

public interface StandardTokenizerInterface

Internal interface for supporting versioned grammars.

Method Summary

Modifier and Type	Method and Description
char[]	getArray()
byte[]	getBytes()
int	getNextToken() Resumes scanning until the next regular expression is matched, the end of input is encountered or an I/O-Error occurs.
java.lang.String	getText()
long	yychar() Returns the current position.
int	yylength() Returns the length of the matched text region.
void	yyreset(java.io.Reader reader) Resets the scanner to read from a new input stream.

Method Detail

getText

java.lang.String getText()

getArray

char[] getArray()

getBytes

byte[] getBytes()

yychar

long yychar()

Returns the current position.

yylength

int yylength()

Returns the length of the matched text region.

getNextToken

int getNextToken()
          throws java.io.IOException

Resumes scanning until the next regular expression is matched, the end of input is encountered or an I/O-Error occurs.

Returns:

the next token, StandardTokenizerImpl.YYEOF on end of stream

Throws:

java.io.IOException - if any I/O-Error occurs

yyreset

void yyreset(java.io.Reader reader)

Resets the scanner to read from a new input stream. Does not close the old reader. All internal variables are reset, the old input stream cannot be reused (internal buffer is discarded and lost). Lexical state is set to ZZ_INITIAL.

Parameters:

reader - the new input stream