public final class XFuzzySuggester extends XAnalyzingSuggester
AnalyzingSuggester
. The similarity measurement is
based on the Damerau-Levenshtein (optimal string alignment) algorithm, though
you can explicitly choose classic Levenshtein by passing false
for the transpositions
parameter.
At most, this query will match terms up to
2
edits. Higher distances are not supported. Note that the
fuzzy distance is measured in "byte space" on the bytes
returned by the TokenStream
's TermToBytesRefAttribute
, usually UTF8. By default
the analyzed bytes must be at least 3 DEFAULT_MIN_FUZZY_LENGTH
bytes before any edits are
considered. Furthermore, the first 1 DEFAULT_NON_FUZZY_PREFIX
byte is not allowed to be
edited. We allow up to 1 (@link
#DEFAULT_MAX_EDITS} edit.
If unicodeAware
parameter in the constructor is set to true, maxEdits,
minFuzzyLength, transpositions and nonFuzzyPrefix are measured in Unicode code
points (actual letters) instead of bytes.*
NOTE: This suggester does not boost suggestions that required no edits over suggestions that did require edits. This is a known limitation.
Note: complex query analyzers can have a significant impact on the lookup performance. It's recommended to not use analyzers that drop or inject terms like synonyms to keep the complexity of the prefix intersection low for good lookup performance. At index time, complex analyzers can safely be used.
XAnalyzingSuggester.XBuilder
Modifier and Type | Field and Description |
---|---|
static int |
DEFAULT_MAX_EDITS
The default maximum number of edits for fuzzy
suggestions.
|
static int |
DEFAULT_MIN_FUZZY_LENGTH
The default minimum length of the key passed to
XAnalyzingSuggester.lookup(java.lang.CharSequence, java.util.Set<org.apache.lucene.util.BytesRef>, boolean, int) before any edits are allowed. |
static int |
DEFAULT_NON_FUZZY_PREFIX
The default prefix length where edits are not allowed.
|
static boolean |
DEFAULT_TRANSPOSITIONS
The default transposition value passed to
LevenshteinAutomata |
static boolean |
DEFAULT_UNICODE_AWARE
Measure maxEdits, minFuzzyLength, transpositions and nonFuzzyPrefix
parameters in Unicode code points (actual letters)
instead of bytes.
|
END_BYTE, EXACT_FIRST, HOLE_CHARACTER, PAYLOAD_SEP, PRESERVE_SEP, SEP_LABEL
Constructor and Description |
---|
XFuzzySuggester(org.apache.lucene.analysis.Analyzer analyzer)
Creates a
FuzzySuggester instance initialized with default values. |
XFuzzySuggester(org.apache.lucene.analysis.Analyzer indexAnalyzer,
org.apache.lucene.analysis.Analyzer queryAnalyzer)
Creates a
FuzzySuggester instance with an index & a query analyzer initialized with default values. |
XFuzzySuggester(org.apache.lucene.analysis.Analyzer indexAnalyzer,
org.apache.lucene.util.automaton.Automaton queryPrefix,
org.apache.lucene.analysis.Analyzer queryAnalyzer,
int options,
int maxSurfaceFormsPerAnalyzedForm,
int maxGraphExpansions,
int maxEdits,
boolean transpositions,
int nonFuzzyPrefix,
int minFuzzyLength,
boolean unicodeAware,
org.apache.lucene.util.fst.FST<org.apache.lucene.util.fst.PairOutputs.Pair<java.lang.Long,org.apache.lucene.util.BytesRef>> fst,
boolean hasPayloads,
int maxAnalyzedPathsForOneInput,
int sepLabel,
int payloadSep,
int endByte,
int holeCharacter)
Creates a
FuzzySuggester instance. |
Modifier and Type | Method and Description |
---|---|
protected org.apache.lucene.util.automaton.Automaton |
convertAutomaton(org.apache.lucene.util.automaton.Automaton a) |
protected java.util.List<org.apache.lucene.search.suggest.analyzing.FSTUtil.Path<org.apache.lucene.util.fst.PairOutputs.Pair<java.lang.Long,org.apache.lucene.util.BytesRef>>> |
getFullPrefixPaths(java.util.List<org.apache.lucene.search.suggest.analyzing.FSTUtil.Path<org.apache.lucene.util.fst.PairOutputs.Pair<java.lang.Long,org.apache.lucene.util.BytesRef>>> prefixPaths,
org.apache.lucene.util.automaton.Automaton lookupAutomaton,
org.apache.lucene.util.fst.FST<org.apache.lucene.util.fst.PairOutputs.Pair<java.lang.Long,org.apache.lucene.util.BytesRef>> fst)
Returns all completion paths to initialize the search.
|
org.apache.lucene.analysis.TokenStreamToAutomaton |
getTokenStreamToAutomaton() |
build, decodeWeight, encodeWeight, get, getCount, getMaxAnalyzedPathsForOneInput, getTempDir, load, load, lookup, ramBytesUsed, store, store, toFiniteStrings
public static final boolean DEFAULT_UNICODE_AWARE
public static final int DEFAULT_MIN_FUZZY_LENGTH
XAnalyzingSuggester.lookup(java.lang.CharSequence, java.util.Set<org.apache.lucene.util.BytesRef>, boolean, int)
before any edits are allowed.public static final int DEFAULT_NON_FUZZY_PREFIX
public static final int DEFAULT_MAX_EDITS
public static final boolean DEFAULT_TRANSPOSITIONS
LevenshteinAutomata
public XFuzzySuggester(org.apache.lucene.analysis.Analyzer analyzer)
FuzzySuggester
instance initialized with default values.analyzer
- the analyzer used for this suggesterpublic XFuzzySuggester(org.apache.lucene.analysis.Analyzer indexAnalyzer, org.apache.lucene.analysis.Analyzer queryAnalyzer)
FuzzySuggester
instance with an index & a query analyzer initialized with default values.indexAnalyzer
- Analyzer that will be used for analyzing suggestions while building the index.queryAnalyzer
- Analyzer that will be used for analyzing query text during lookuppublic XFuzzySuggester(org.apache.lucene.analysis.Analyzer indexAnalyzer, org.apache.lucene.util.automaton.Automaton queryPrefix, org.apache.lucene.analysis.Analyzer queryAnalyzer, int options, int maxSurfaceFormsPerAnalyzedForm, int maxGraphExpansions, int maxEdits, boolean transpositions, int nonFuzzyPrefix, int minFuzzyLength, boolean unicodeAware, org.apache.lucene.util.fst.FST<org.apache.lucene.util.fst.PairOutputs.Pair<java.lang.Long,org.apache.lucene.util.BytesRef>> fst, boolean hasPayloads, int maxAnalyzedPathsForOneInput, int sepLabel, int payloadSep, int endByte, int holeCharacter)
FuzzySuggester
instance.indexAnalyzer
- Analyzer that will be used for
analyzing suggestions while building the index.queryAnalyzer
- Analyzer that will be used for
analyzing query text during lookupoptions
- see XAnalyzingSuggester.EXACT_FIRST
, XAnalyzingSuggester.PRESERVE_SEP
maxSurfaceFormsPerAnalyzedForm
- Maximum number of
surface forms to keep for a single analyzed form.
When there are too many surface forms we discard the
lowest weighted ones.maxGraphExpansions
- Maximum number of graph paths
to expand from the analyzed form. Set this to -1 for
no limit.maxEdits
- must be >= 0 and <= LevenshteinAutomata.MAXIMUM_SUPPORTED_DISTANCE
.transpositions
- true
if transpositions should be treated as a primitive
edit operation. If this is false, comparisons will implement the classic
Levenshtein algorithm.nonFuzzyPrefix
- length of common (non-fuzzy) prefix (see default DEFAULT_NON_FUZZY_PREFIX
minFuzzyLength
- minimum length of lookup key before any edits are allowed (see default DEFAULT_MIN_FUZZY_LENGTH
)sepLabel
- separation labelpayloadSep
- payload separator byteendByte
- end byte marker byteprotected java.util.List<org.apache.lucene.search.suggest.analyzing.FSTUtil.Path<org.apache.lucene.util.fst.PairOutputs.Pair<java.lang.Long,org.apache.lucene.util.BytesRef>>> getFullPrefixPaths(java.util.List<org.apache.lucene.search.suggest.analyzing.FSTUtil.Path<org.apache.lucene.util.fst.PairOutputs.Pair<java.lang.Long,org.apache.lucene.util.BytesRef>>> prefixPaths, org.apache.lucene.util.automaton.Automaton lookupAutomaton, org.apache.lucene.util.fst.FST<org.apache.lucene.util.fst.PairOutputs.Pair<java.lang.Long,org.apache.lucene.util.BytesRef>> fst) throws java.io.IOException
XAnalyzingSuggester
getFullPrefixPaths
in class XAnalyzingSuggester
java.io.IOException
protected org.apache.lucene.util.automaton.Automaton convertAutomaton(org.apache.lucene.util.automaton.Automaton a)
convertAutomaton
in class XAnalyzingSuggester
public org.apache.lucene.analysis.TokenStreamToAutomaton getTokenStreamToAutomaton()
getTokenStreamToAutomaton
in class XAnalyzingSuggester