public class OakWordTokenFilter extends CompoundWordTokenFilterBase
CompoundWordTokenFilterBase.CompoundToken
DEFAULT_MAX_SUBWORD_SIZE, DEFAULT_MIN_SUBWORD_SIZE, DEFAULT_MIN_WORD_SIZE, dictionary, maxSubwordSize, minSubwordSize, minWordSize, offsetAtt, onlyLongestMatch, termAtt, tokens
Constructor and Description |
---|
OakWordTokenFilter(org.apache.lucene.util.Version version,
org.apache.lucene.analysis.TokenStream in) |
OakWordTokenFilter(org.apache.lucene.util.Version version,
org.apache.lucene.analysis.TokenStream in,
char[] separators) |
Modifier and Type | Method and Description |
---|---|
protected void |
decompose()
Decomposes the current
CompoundWordTokenFilterBase.termAtt and places CompoundWordTokenFilterBase.CompoundToken instances in the CompoundWordTokenFilterBase.tokens list. |
incrementToken, reset
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState, toString
public OakWordTokenFilter(org.apache.lucene.util.Version version, org.apache.lucene.analysis.TokenStream in, char[] separators)
public OakWordTokenFilter(org.apache.lucene.util.Version version, org.apache.lucene.analysis.TokenStream in)
protected void decompose()
CompoundWordTokenFilterBase
CompoundWordTokenFilterBase.termAtt
and places CompoundWordTokenFilterBase.CompoundToken
instances in the CompoundWordTokenFilterBase.tokens
list.
The original token may not be placed in the list, as it is automatically passed through this filter.decompose
in class CompoundWordTokenFilterBase
Copyright © 2012–2020 The Apache Software Foundation. All rights reserved.