Package com.yahoo.language.process
Class SegmenterImpl
java.lang.Object
com.yahoo.language.process.SegmenterImpl
- All Implemented Interfaces:
Segmenter
- Author:
- Simon Thoresen Hult
-
Constructor Summary
-
Method Summary
-
Constructor Details
-
SegmenterImpl
-
-
Method Details
-
segment
Description copied from interface:Segmenter
Split input-string into tokens, and returned a list of tokens in unprocessed form (i.e. lowercased, normalized and stemmed if applicable, see @link{StemMode} for list of stemming options). It is assumed that the input only contains word-characters, any punctuation and spacing tokens will be removed.
-