Class Token


  • public class Token
    extends java.lang.Object
    Simple token class.
    Author:
    David Hovemeyer
    See Also:
    Tokenizer
    • Field Summary

      Fields 
      Modifier and Type Field Description
      static int COMMENT
      A comment.
      static int EOF
      End of file.
      static int EOL
      End of line.
      static int SINGLE
      A single character token.
      static int STRING
      A string or character literal.
      static int WORD
      An ordinary word, number, etc.
    • Constructor Summary

      Constructors 
      Constructor Description
      Token​(int kind)
      Constructor when there is no text.
      Token​(int kind, java.lang.String lexeme)
      Constructor.
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      int getKind()
      Get the kind of token.
      java.lang.String getLexeme()
      Get the text value of the token.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Constructor Detail

      • Token

        public Token​(int kind,
                     java.lang.String lexeme)
        Constructor.
        Parameters:
        kind - the kind of token
        lexeme - the text value of the token
      • Token

        public Token​(int kind)
        Constructor when there is no text. E.g., EOF and EOL.
        Parameters:
        kind - the kind of token
    • Method Detail

      • getKind

        public int getKind()
        Get the kind of token.
      • getLexeme

        public java.lang.String getLexeme()
        Get the text value of the token.