Class Tokenizer


  • public class Tokenizer
    extends java.lang.Object
    A simple tokenizer for Java source text. This is not intended to be a compliant lexer; instead, it is for quick and dirty scanning.
    Author:
    David Hovemeyer
    See Also:
    Token
    • Constructor Summary

      Constructors 
      Constructor Description
      Tokenizer​(java.io.Reader reader)
      Constructor.
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      Token next()
      Get the next Token in the stream.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Constructor Detail

      • Tokenizer

        public Tokenizer​(java.io.Reader reader)
        Constructor.
        Parameters:
        reader - the Reader for the Java source text
    • Method Detail

      • next

        public Token next()
                   throws java.io.IOException
        Get the next Token in the stream.
        Returns:
        the Token
        Throws:
        java.io.IOException