net.sourceforge.pmd.cpd
Class AbstractTokenizer

java.lang.Object
  net.sourceforge.pmd.cpd.AbstractTokenizer

All Implemented Interfaces:

Tokenizer

public abstract class AbstractTokenizer
extends Object
implements Tokenizer

Author:

Zev Blut [email protected], Romain PELISSE [email protected]

Field Summary

protected List<String>	ignorableCharacter
protected List<String>	ignorableStmt
protected char	oneLineCommentChar
protected Character	spanMultipleLinesLineContinuationCharacter
protected boolean	spanMultipleLinesString
protected List<String>	stringToken

Fields inherited from interface net.sourceforge.pmd.cpd.Tokenizer

DEFAULT_SKIP_BLOCKS_PATTERN, IGNORE_ANNOTATIONS, IGNORE_IDENTIFIERS, IGNORE_LITERALS, IGNORE_USINGS, OPTION_SKIP_BLOCKS, OPTION_SKIP_BLOCKS_PATTERN

Constructor Summary

AbstractTokenizer()

Method Summary

void	tokenize(SourceCode tokens, Tokens tokenEntries)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

stringToken

protected List<String> stringToken

ignorableCharacter

protected List<String> ignorableCharacter

ignorableStmt

protected List<String> ignorableStmt

oneLineCommentChar

protected char oneLineCommentChar

spanMultipleLinesString

protected boolean spanMultipleLinesString

spanMultipleLinesLineContinuationCharacter

protected Character spanMultipleLinesLineContinuationCharacter

Constructor Detail

AbstractTokenizer

public AbstractTokenizer()

Method Detail

tokenize

public void tokenize(SourceCode tokens,
                     Tokens tokenEntries)

Specified by:

tokenize in interface Tokenizer