Class SpecialTokens


  • public class SpecialTokens
    extends java.lang.Object
    A list of special tokens - string that should be threated as word no matter what they contain. Special tokens are case insensitive.
    Author:
    bratseth
    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description
      static class  SpecialTokens.SpecialToken
      An immutable special token
    • Constructor Summary

      Constructors 
      Constructor Description
      SpecialTokens()
      Creates a null list of special tokens
      SpecialTokens​(java.lang.String name)  
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void addSpecialToken​(java.lang.String token, java.lang.String replace)
      Adds a special token to this
      void freeze()  
      java.lang.String getName()
      Returns the name of this special tokens list
      int size()
      Returns the number of special tokens in this
      SpecialTokens.SpecialToken tokenize​(java.lang.String string, boolean substring)
      Returns the special token starting at the start of the given string, or null if no special token starts at this string
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Constructor Detail

      • SpecialTokens

        public SpecialTokens()
        Creates a null list of special tokens
      • SpecialTokens

        public SpecialTokens​(java.lang.String name)
    • Method Detail

      • getName

        public java.lang.String getName()
        Returns the name of this special tokens list
      • addSpecialToken

        public void addSpecialToken​(java.lang.String token,
                                    java.lang.String replace)
        Adds a special token to this
        Parameters:
        token - the special token string to add
        replace - the token to replace instances of the special token with, or null to keep the token
      • tokenize

        public SpecialTokens.SpecialToken tokenize​(java.lang.String string,
                                                   boolean substring)
        Returns the special token starting at the start of the given string, or null if no special token starts at this string
        Parameters:
        string - the string to search for a special token at the start position
        substring - true to allow the special token to be followed by a character which does not mark the end of a token
      • size

        public int size()
        Returns the number of special tokens in this
      • freeze

        public void freeze()