Uses of Interface
org.deeplearning4j.text.tokenization.tokenizer.TokenPreProcess

Packages that use TokenPreProcess

Package	Description
org.deeplearning4j.text.tokenization.tokenizer
org.deeplearning4j.text.tokenization.tokenizer.preprocessor
org.deeplearning4j.text.tokenization.tokenizerfactory

Uses of TokenPreProcess in org.deeplearning4j.text.tokenization.tokenizer

Methods in org.deeplearning4j.text.tokenization.tokenizer with parameters of type TokenPreProcess

Modifier and Type	Method and Description
void	NGramTokenizer.setTokenPreProcessor(TokenPreProcess tokenPreProcessor)
void	DefaultTokenizer.setTokenPreProcessor(TokenPreProcess tokenPreProcessor)
void	Tokenizer.setTokenPreProcessor(TokenPreProcess tokenPreProcessor) Set the token pre process
void	DefaultStreamTokenizer.setTokenPreProcessor(TokenPreProcess tokenPreProcessor)

Uses of TokenPreProcess in org.deeplearning4j.text.tokenization.tokenizer.preprocessor

Classes in org.deeplearning4j.text.tokenization.tokenizer.preprocessor that implement TokenPreProcess

Modifier and Type	Class and Description
class	CommonPreprocessor
class	EndingPreProcessor Gets rid of endings: ed,ing, ly, s, .
class	LowCasePreProcessor

Uses of TokenPreProcess in org.deeplearning4j.text.tokenization.tokenizerfactory

Methods in org.deeplearning4j.text.tokenization.tokenizerfactory that return TokenPreProcess

Modifier and Type	Method and Description
TokenPreProcess	NGramTokenizerFactory.getTokenPreProcessor() Returns TokenPreProcessor set for this TokenizerFactory instance
TokenPreProcess	DefaultTokenizerFactory.getTokenPreProcessor() Returns TokenPreProcessor set for this TokenizerFactory instance
TokenPreProcess	TokenizerFactory.getTokenPreProcessor() Returns TokenPreProcessor set for this TokenizerFactory instance

Methods in org.deeplearning4j.text.tokenization.tokenizerfactory with parameters of type TokenPreProcess

Modifier and Type	Method and Description
void	NGramTokenizerFactory.setTokenPreProcessor(TokenPreProcess preProcessor)
void	DefaultTokenizerFactory.setTokenPreProcessor(TokenPreProcess preProcessor)
void	TokenizerFactory.setTokenPreProcessor(TokenPreProcess preProcessor) Sets a token pre processor to be used with every tokenizer