Package com.yahoo.search.query.rewrite

Class QueryRewriteSearcher

  • All Implemented Interfaces:
    com.yahoo.component.Component, java.lang.Comparable<com.yahoo.component.Component>
    Direct Known Subclasses:
    GenericExpansionRewriter, MisspellRewriter, NameRewriter
    public abstract class QueryRewriteSearcher
extends Searcher

    A template class for all rewriters

    All rewriters extending this class would need to implement the rewrite method which contains the rewriter's main logic, getSkipRewriterIfRewritten method which indicates whether this rewriter should be skipped if the query has been rewritten, getRewriterName method which returns the name of the rewriter used in query profile, configure method which contains any instance creation time configuration besides the default FSA loading, and getDefaultDicts method which return the pair of dictionary name and filename.

    Common rewrite features are in RewriterFeatures.java. Common rewriter utils are in RewriterUtils.java.

    Author:
    Karen Sze Wing Lee

    • Field Summary

      Fields 
      Modifier and Type Field Description
      protected java.util.logging.Logger logger  
      protected java.util.HashMap<java.lang.String,​java.lang.Object> rewriterDicts  

      • Fields inherited from class com.yahoo.component.AbstractComponent

        isDeconstructable

    • Constructor Summary

      Constructors 
      Modifier Constructor Description
      protected QueryRewriteSearcher()
      Empty constructor for unit test.
      protected QueryRewriteSearcher​(com.yahoo.component.ComponentId id)
      Empty constructor.
      protected QueryRewriteSearcher​(com.yahoo.component.ComponentId id, com.yahoo.filedistribution.fileacquirer.FileAcquirer fileAcquirer, RewritesConfig config)
      Constructor for this rewriter.
      protected QueryRewriteSearcher​(RewritesConfig config, java.util.HashMap<java.lang.String,​java.io.File> fileList)
      Constructor for unit test.

    • Method Summary

      All Methods Instance Methods Abstract Methods Concrete Methods 
      Modifier and Type Method Description
      abstract boolean configure​(com.yahoo.filedistribution.fileacquirer.FileAcquirer fileAcquirer, RewritesConfig config, java.util.HashMap<java.lang.String,​java.io.File> fileList)
      Perform instance creation time configuration besides the default FSA loading
      abstract java.util.HashMap<java.lang.String,​java.lang.String> getDefaultFSAs()
      Get default FSA dictionary names
      protected java.lang.String getQPConfig​(Query query, java.lang.String paramName)
      Get config parameter value set in query profile
      protected java.lang.String getRewriteFromFSA​(Query query, java.lang.String dictName, java.lang.String key)
      Retrieve rewrite from FSA given the original query
      abstract java.lang.String getRewriterName()
      Retrieve rewriter name It should match the name used in query profile
      protected abstract boolean getSkipRewriterIfRewritten()
      Check whether rewriter should be skipped if the query has been rewritten by other rewriter
      protected abstract java.util.HashMap<java.lang.String,​java.lang.Object> rewrite​(Query query, java.lang.String dictKey)
      Perform the main rewrite logic
      Result search​(Query query, Execution execution)
      Perform main rewrite logics for this searcher
      - Skip to next rewriter if query is previously rewritten and getSkipRewriterIfRewritten() is true for this rewriter
      - Execute rewriter's main rewrite logic
      - Pass to the next rewriter the query to be used for dictionary retrieval

      • Methods inherited from class com.yahoo.component.chain.ChainedComponent

        getAnnotatedDependencies, getDefaultAnnotatedDependencies, getDependencies, initDependencies

      • Methods inherited from class com.yahoo.component.AbstractComponent

        clone, compareTo, deconstruct, getClassName, getId, getIdString, hasInitializedId, initId, isDeconstructable, setIsDeconstructable

      • Methods inherited from class java.lang.Object

        equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    • Field Detail

      • logger

        protected final java.util.logging.Logger logger

      • rewriterDicts

        protected java.util.HashMap<java.lang.String,​java.lang.Object> rewriterDicts

    • Constructor Detail

      • QueryRewriteSearcher

        @Inject
protected QueryRewriteSearcher​(com.yahoo.component.ComponentId id,
                               com.yahoo.filedistribution.fileacquirer.FileAcquirer fileAcquirer,
                               RewritesConfig config)
        Constructor for this rewriter. Prepare the data needed by the rewriter
        Parameters:
        id - Component ID (see vespa's search container doc for more detail)
        fileAcquirer - Required param for retrieving file type config (see vespa's search container doc for more detail)
        config - Config from vespa-services.xml (see vespa's search container doc for more detail)

      • QueryRewriteSearcher

        protected QueryRewriteSearcher​(RewritesConfig config,
                               java.util.HashMap<java.lang.String,​java.io.File> fileList)
        Constructor for unit test. Prepare the data needed by the rewriter
        Parameters:
        config - Config from vespa-services.xml (see vespa's search container doc for more detail)
        fileList - pairs of file name and file handler for unit tests

      • QueryRewriteSearcher

        protected QueryRewriteSearcher​(com.yahoo.component.ComponentId id)
        Empty constructor. Do nothing at instance creation time

      • QueryRewriteSearcher

        protected QueryRewriteSearcher()
        Empty constructor for unit test. Do nothing at instance creation time

    • Method Detail

      • configure

        public abstract boolean configure​(com.yahoo.filedistribution.fileacquirer.FileAcquirer fileAcquirer,
                                  RewritesConfig config,
                                  java.util.HashMap<java.lang.String,​java.io.File> fileList)
                           throws java.lang.RuntimeException
        Perform instance creation time configuration besides the default FSA loading
        Parameters:
        fileAcquirer - Required param for retrieving file type config (see vespa's search container doc for more detail)
        config - Config from vespa-services.xml (see vespa's search container doc for more detail)
        fileList - pairs of file name and file handler for unit tests
        Returns:
        boolean true if loaded successfully, false otherwise
        Throws:
        java.lang.RuntimeException

      • search

        public Result search​(Query query,
                     Execution execution)
        Perform main rewrite logics for this searcher
        - Skip to next rewriter if query is previously rewritten and getSkipRewriterIfRewritten() is true for this rewriter
        - Execute rewriter's main rewrite logic
        - Pass to the next rewriter the query to be used for dictionary retrieval
        Specified by:
        search in class Searcher
        Parameters:
        query - the query
        Returns:
        the result of making this query

      • rewrite

        protected abstract java.util.HashMap<java.lang.String,​java.lang.Object> rewrite​(Query query,
                                                                                      java.lang.String dictKey)
                                                                               throws java.lang.RuntimeException
        Perform the main rewrite logic
        Parameters:
        query - Query object from searcher
        dictKey - the key passed from previous rewriter to be treated as "original query from user" For example, if previous is misspell rewriter, it would pass the corrected query as the "original query from user". For other rewriters which add variants, abbr, etc to the query, the original query should be passed as a key. This rewriter could still choose to ignore this key. This key is not the rewritten query itself. For example, if original query is (willl smith) and the rewritten query is (willl smith) OR (will smith) the key to be passed could be (will smith)
        Returns:
        HashMap which contains the key value pairs:
        - whether this query has been rewritten by this rewriter
        key: rewritten
        value: true or false
        - the key to be treated as "original query from user" in next rewriter downstream, for example, misspell rewriter would pass the corrected query as the "original query from user" to the next rewriter. For other rewriters which add variants, abbr, etc to the query, the original query should be passed as a key. This key is not necessarily consumed by the next rewriter. The next rewriter can still choose to ignore this key.
        key: newDictKey
        value: new dict key
        Throws:
        java.lang.RuntimeException

      • getSkipRewriterIfRewritten

        protected abstract boolean getSkipRewriterIfRewritten()
        Check whether rewriter should be skipped if the query has been rewritten by other rewriter
        Returns:
        boolean Whether rewriter should be skipped

      • getRewriterName

        public abstract java.lang.String getRewriterName()
        Retrieve rewriter name It should match the name used in query profile
        Returns:
        Name of the rewriter

      • getDefaultFSAs

        public abstract java.util.HashMap<java.lang.String,​java.lang.String> getDefaultFSAs()
        Get default FSA dictionary names
        Returns:
        Pair of FSA dictionary name and filename

      • getQPConfig

        protected java.lang.String getQPConfig​(Query query,
                                       java.lang.String paramName)
        Get config parameter value set in query profile
        Parameters:
        query - Query object from the searcher
        paramName - parameter to be retrieved
        Returns:
        parameter value or null if not found

      • getRewriteFromFSA

        protected java.lang.String getRewriteFromFSA​(Query query,
                                             java.lang.String dictName,
                                             java.lang.String key)
                                      throws java.lang.RuntimeException
        Retrieve rewrite from FSA given the original query
        Parameters:
        query - Query object from searcher
        dictName - FSA dictionary name
        key - The original query used to retrieve rewrite from the dictionary
        Returns:
        String The retrieved rewrites, null if query doesn't exist
        Throws:
        java.lang.RuntimeException