net.sourceforge.pmd.cpd
Class AbstractTokenizer

java.lang.Object
  net.sourceforge.pmd.cpd.AbstractTokenizer

All Implemented Interfaces:

Tokenizer

public abstract class AbstractTokenizer
extends java.lang.Object
implements Tokenizer

Author:

Zev Blut [email protected], Romain PELISSE [email protected]

Field Summary

protected java.util.List<java.lang.String>	ignorableCharacter
protected java.util.List<java.lang.String>	ignorableStmt
protected char	oneLineCommentChar
protected java.lang.Character	spanMultipleLinesLineContinuationCharacter
protected boolean	spanMultipleLinesString
protected java.util.List<java.lang.String>	stringToken

Fields inherited from interface net.sourceforge.pmd.cpd.Tokenizer

DEFAULT_SKIP_BLOCKS_PATTERN, IGNORE_ANNOTATIONS, IGNORE_IDENTIFIERS, IGNORE_LITERALS, OPTION_SKIP_BLOCKS, OPTION_SKIP_BLOCKS_PATTERN

Constructor Summary

AbstractTokenizer()

Method Summary

void	tokenize(SourceCode tokens, Tokens tokenEntries)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

stringToken

protected java.util.List<java.lang.String> stringToken

ignorableCharacter

protected java.util.List<java.lang.String> ignorableCharacter

ignorableStmt

protected java.util.List<java.lang.String> ignorableStmt

oneLineCommentChar

protected char oneLineCommentChar

spanMultipleLinesString

protected boolean spanMultipleLinesString

spanMultipleLinesLineContinuationCharacter

protected java.lang.Character spanMultipleLinesLineContinuationCharacter

Constructor Detail

AbstractTokenizer

public AbstractTokenizer()

Method Detail

tokenize

public void tokenize(SourceCode tokens,
                     Tokens tokenEntries)

Specified by:

tokenize in interface Tokenizer