public final class Tokenizer extends Object
Modifier and Type | Class and Description |
---|---|
static interface |
Tokenizer.CharacterStream
Stream of chars.
|
static class |
Tokenizer.Token
Simple token
|
static class |
Tokenizer.TokenType
token type
|
Modifier and Type | Field and Description |
---|---|
static Tokenizer.Token |
BRACKET_CLOSE |
static Tokenizer.Token |
BRACKET_OPEN |
static Tokenizer.Token |
DIVIDE |
static Tokenizer.Token |
END |
static Tokenizer.Token |
EXPONENT |
static Tokenizer.Token |
MINUS |
static Tokenizer.Token |
MULTIPLY |
static Tokenizer.Token |
PLUS |
static Tokenizer.Token |
SPACE |
Constructor and Description |
---|
Tokenizer(Tokenizer.CharacterStream stream)
Create tokenizer of a given char stream
|
Modifier and Type | Method and Description |
---|---|
static Tokenizer.CharacterStream |
concat(Tokenizer.CharacterStream a,
Tokenizer.CharacterStream b)
Concat char streams
|
static Tokenizer.CharacterStream |
mkCharacterStream(String string,
Character terminateChar)
Create character stream from string
|
static Tokenizer |
mkTokenizer(String string)
Create string tokenizer
|
static Tokenizer |
mkTokenizer(String string,
Character terminateChar)
Create string tokenizer
|
Tokenizer.Token |
nextToken()
Get the next token from stream
|
public static final Tokenizer.Token END
public static final Tokenizer.Token PLUS
public static final Tokenizer.Token MINUS
public static final Tokenizer.Token MULTIPLY
public static final Tokenizer.Token DIVIDE
public static final Tokenizer.Token EXPONENT
public static final Tokenizer.Token BRACKET_OPEN
public static final Tokenizer.Token BRACKET_CLOSE
public static final Tokenizer.Token SPACE
public Tokenizer(Tokenizer.CharacterStream stream)
public Tokenizer.Token nextToken()
public static Tokenizer.CharacterStream concat(Tokenizer.CharacterStream a, Tokenizer.CharacterStream b)
public static Tokenizer.CharacterStream mkCharacterStream(String string, Character terminateChar)
terminateChar
- if a non-null value specified, stream will terminate on the last char preceding the terminateChar
public static Tokenizer mkTokenizer(String string, Character terminateChar)
terminateChar
- if a non-null value specified, stream will terminate on the last char preceding the terminateChar
Copyright © 2018. All rights reserved.