Package org.deeplearning4j.text.tokenization.tokenizer
-
Interface Summary Interface Description Tokenizer TokenPreProcess -
Class Summary Class Description BertWordPieceStreamTokenizer BertWordPieceTokenizer DefaultStreamTokenizer Tokenizer based on theStreamTokenizerDefaultTokenizer Default tokenizerNGramTokenizer