Package com.yahoo.search

Class Searcher

java.lang.Object

com.yahoo.component.AbstractComponent

com.yahoo.component.chain.ChainedComponent

com.yahoo.processing.Processor

com.yahoo.search.Searcher

All Implemented Interfaces:

com.yahoo.component.Component, java.lang.Comparable<com.yahoo.component.Component>

Direct Known Subclasses:

BlendingSearcher, BooleanSearcher, CacheControlSearcher, CJKSearcher, ClusterSearcher, CollapsePhraseSearcher, ConnectionControlSearcher, ContainerLatencySearcher, DefaultPositionSearcher, DocumentDb, DocumentSourceSearcher, ExampleSearcher, FieldCollapsingSearcher, FieldFiller, FieldFilter, FillSearcher, ForkingSearcher, GroupingExecutor, GroupingQueryParser, GroupingValidator, InputCheckingSearcher, JSONDebugSearcher, JuniperSearcher, LiteralBoostSearcher, LowercasingSearcher, MetricsSearcher, MinimalQueryInserter, MultipleResultsSearcher, NGramSearcher, NonPhrasingSearcher, NoRankingSearcher, NormalizingSearcher, PageTemplateSearcher, PeakQpsSearcher, PhrasingSearcher, PingableSearcher, PosSearcher, QueryRewriteSearcher, QuotingSearcher, RangeQueryOptimizer, RateLimitingSearcher, RecallSearcher, SearchChainDispatcherSearcher, SemanticSearcher, SortingDegrader, StatisticsSearcher, StemmingSearcher, UniqueGroupingSearcher, ValidateMatchPhaseSearcher, ValidateNearestNeighborSearcher, ValidatePredicateSearcher, ValidateSortingSearcher, WandSearcher

public abstract class Searcher
extends com.yahoo.processing.Processor

Superclass of all Components which produces Results in response to Queries by calling the search method.

Searchers are participants in chain of responsibility search chains where they passes the Queries downwards by synchroneously calling the next Searcher in the chain, and returns the Results back up as the response.

Any Searcher may

• Do modifications to the Query before passing it on (a query rerwiter)
• Do modifications to the Result before passing it on up, e.g removing altering, reorganizing or adding Hits (a result processor)
• Pass the Query on to multiple other search chains, either in series (by creating a new Execution for each chain), or in parallel (by creating a AsyncExecution) (a federator)
• Create a Result and pass it back up, either by calling some other node(s) to get the data, or by creating the Result from internal data (a source)
• Pass some query on downwards multiple times, or in different ways, typically each time depending of the Result returned the last time (a workflow)

...or some combination of the above of course. Note that as Searchers work synchronously, any information can be retained on the stack in the Searcher from the Query is received until the Result is returned simply by declaring variables for the data in the search method (or whatever it calls), and for the same reason workflows are implemented as Java code. However, searchers are executed by many threads, for different Queries, in parallell, so any mutable data shared between queries (and hence stored as instance members must be accessed multithread safely. In many cases, shared data can simply be instantiated in the constructor and used in read-only mode afterwards

Searcher lifecycle: A searcher has a simple life-cycle:

• Construction: While a constructor is running. A searcher is handed its id and configuration (if any) in the constructor. During construction, the searcher should build any in-memory structures needed. A new instance of the searcher will be created when the configuration is changed. Constructors are called with this priority:
  • The constructor taking a ComponentId, followed by the highest number of config classes (subclasses of ConfigInstance) as arguments.
  • The constructor taking a string id followed by the highest number of config classes as arguments.
  • The constructor taking only the highest number of config classes as arguments.
  • The constructor taking a ComponentId as the only argument
  • The constructor taking a string id as the only argument
  • The default (no-argument) constructor.
  If none of these constructors are declared, searcher construction will fail.
• In service: After the constructor has returned. In this phase, searcher service methods are called at any time by multiple threads in parallel. Implementations should avoid synchronization and access to volatiles as much as possible by keeping data structures build in construction read-only.
• Deconstruction: While deconstruct is running. All Searcher service method calls have completed when this method is called. When it returns, the searcher will be eligible for garbage collection.

Author:

bratseth

Field Summary

Fields inherited from class com.yahoo.component.AbstractComponent

isDeconstructable

Constructor Summary

Constructors

Constructor	Description
Searcher()
Searcher(com.yahoo.component.ComponentId id)	Creates a searcher from an id

Method Summary

Modifier and Type	Method	Description
void	ensureFilled(Result result, java.lang.String summaryClass, Execution execution)	Fills the result if it is not already filled for the given summary class.
void	fill(Result result, java.lang.String summaryClass, Execution execution)	Fill hit properties with data using the given summary class.
protected java.util.logging.Logger	getLogger()	Returns a logger unique for the instance subclass
com.yahoo.processing.Response	process(com.yahoo.processing.Request request, com.yahoo.processing.execution.Execution execution)	Use the search method in Searcher processors.
abstract Result	search(Query query, Execution execution)	Override this to implement your searcher.
java.lang.String	toString()	Returns "searcher 'getId()'"

Methods inherited from class com.yahoo.component.chain.ChainedComponent

getAnnotatedDependencies, getDefaultAnnotatedDependencies, getDependencies, initDependencies

Methods inherited from class com.yahoo.component.AbstractComponent

clone, compareTo, deconstruct, getClassName, getId, getIdString, hasInitializedId, initId, isDeconstructable, setIsDeconstructable

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Searcher

public Searcher()

Searcher

public Searcher(com.yahoo.component.ComponentId id)

Creates a searcher from an id

Method Detail

search

public abstract Result search(Query query,
                              Execution execution)

Override this to implement your searcher.

Searcher implementation subclasses will, depending on their type of logic, do one of the following:

• Query processors: Access the query, then call execution.search and return the result
• Result processors: Call execution.search to get the result, access it and return
• Sources (which produces results): Create a result, add the desired hits and return it.
• Federators (which forwards the search to multiple subchains): Call search on the desired subchains in parallel and get the results. Combine the results to one and return it.
• Workflows: Call execution.search as many times as desired, using different queries. Eventually return a result.

Hits come in two kinds - concrete hits are actual content of the kind requested by the user, meta hits are hits which provides information about the collection of hits, on the query, the service and so on.

The query specifies a window into a larger result list that must be returned from the searcher through hits and offset; Searchers which returns list of hits in the top level in the result must return at least hits number of hits (or if impossible; all that are available), starting at the given offset. In addition, searchers are allowed to return any number of meta hits (although this number is expected to be low). For hits contained in nested hit groups, the concept of a window defined by hits and offset is not well defined and does not apply.

Error handling in searchers:

• Unexpected events: Throw any RuntimeException. This query will fail with the exception message, and the error will be logged
• Expected events: Create (new Result(Query, ErrorMessage) or add result.setErrorIfNoOtherErrors(ErrorMessage) an error message to the Result.
• Recoverable user errors: Add a FeedbackHit explaining the condition and how to correct it.

Parameters:

query - the query

Returns:

the result of making this query

process

public final com.yahoo.processing.Response process(com.yahoo.processing.Request request,
                                                   com.yahoo.processing.execution.Execution execution)

Use the search method in Searcher processors. This forwards to it.

Specified by:

process in class com.yahoo.processing.Processor

fill

public void fill(Result result,
                 java.lang.String summaryClass,
                 Execution execution)

Fill hit properties with data using the given summary class. Calling this on already filled results has no cost.

This needs to be overridden by federating searchers to contact search sources again by propagating the fill call down through the search chain, and by source searchers which talks to fill capable backends to request the data to be filled. Other searchers do not need to override this.

Parameters:

result - the result to fill

summaryClass - the name of the collection of fields to fetch the values of, or null to use the default

ensureFilled

public final void ensureFilled(Result result,
                               java.lang.String summaryClass,
                               Execution execution)

Fills the result if it is not already filled for the given summary class. See the fill method.

getLogger

protected java.util.logging.Logger getLogger()

Returns a logger unique for the instance subclass

toString

public java.lang.String toString()

Returns "searcher 'getId()'"

Overrides:

toString in class com.yahoo.component.AbstractComponent