org.deeplearning4j.text.tokenization.tokenizer

Interface TokenPreProcess

All Known Implementing Classes:

CommonPreprocessor, EndingPreProcessor, LowCasePreProcessor

public interface TokenPreProcess

Token preprocessing

Author:

Adam Gibson

Method Summary

Modifier and Type	Method and Description
String	preProcess(String token) Pre process a token

Method Detail

preProcess

String preProcess(String token)

Pre process a token

Parameters:

token - the token to pre process

Returns:

the preprocessed token