public class SpecialTokens extends Object
Modifier and Type | Class and Description |
---|---|
static class |
SpecialTokens.SpecialToken
An immutable special token
|
Constructor and Description |
---|
SpecialTokens()
Creates a null list of special tokens
|
SpecialTokens(String name) |
Modifier and Type | Method and Description |
---|---|
void |
addSpecialToken(String token,
String replace)
Adds a special token to this
|
void |
freeze() |
String |
getName()
Returns the name of this special tokens list
|
int |
size()
Returns the number of special tokens in this
|
SpecialTokens.SpecialToken |
tokenize(String string,
boolean substring)
Returns the special token starting at the start of the given string, or null if no
special token starts at this string
|
public SpecialTokens()
public SpecialTokens(String name)
public String getName()
public void addSpecialToken(String token, String replace)
token
- the special token string to addreplace
- the token to replace instances of the special token with,
or null to keep the tokenpublic SpecialTokens.SpecialToken tokenize(String string, boolean substring)
string
- the string to search for a special token at the start positionsubstring
- true to allow the special token to be followed by a character which does not
mark the end of a tokenpublic int size()
public void freeze()
Copyright © 2018. All rights reserved.