Package com.yahoo.search.cluster

Class ClusterSearcher<T>

  • All Implemented Interfaces:
    com.yahoo.component.Component, NodeManager<T>, java.lang.Comparable<com.yahoo.component.Component>
    public abstract class ClusterSearcher<T>
extends PingableSearcher
implements NodeManager<T>
    Implements clustering (failover and load balancing) over a set of client connections to a homogenuos cluster of nodes. Searchers which wants to make clustered connections to some service should use this.

    This replaces the usual searcher methods by ones which have the same contract and semantics but which takes an additional parameter which is the Connection selected by the cluster searcher which the method should use. Overrides of these connection methods must not call the super methods to pass on but must use the methods on execution.

    The type argument is the class (of any type) representing the connections. The connection objects should implement a good toString to ease diagnostics.

    Author:
    bratseth, Arne Bergene Fossaa

    • Field Summary

      • Fields inherited from class com.yahoo.component.AbstractComponent

        isDeconstructable

    • Constructor Summary

      Constructors 
      Modifier Constructor Description
        ClusterSearcher​(com.yahoo.component.ComponentId id, java.util.List<T> connections, boolean internal)
      Creates a new cluster searcher
        ClusterSearcher​(com.yahoo.component.ComponentId id, java.util.List<T> connections, Hasher<T> hasher, boolean internal)  
      protected ClusterSearcher​(com.yahoo.component.ComponentId id, java.util.List<T> connections, Hasher<T> hasher, boolean internal, boolean startPingThread)  

    • Method Summary

      All Methods Instance Methods Abstract Methods Concrete Methods 
      Modifier and Type Method Description
      void deconstruct()  
      void failed​(T node)
      NodeManager method, called from ClusterMonitor
      void fill​(Result result, java.lang.String summaryClass, Execution execution)
      Fill hit properties with data using the given summary class.
      protected abstract void fill​(Result result, java.lang.String summaryClass, Execution execution, T connection)
      Perform the fill against the given connection.
      protected T getFirstConnection​(Hasher.NodeList<T> nodes, int code, int trynum, Query query)  
      Hasher<T> getHasher()
      Returns the hasher used internally in this.
      ClusterMonitor<T> getMonitor()
      Returns the monitor of these nodes
      protected void log​(java.util.logging.Level level, java.lang.Object... objects)  
      protected abstract Pong ping​(Ping ping, T connection)
      Pings this connection.
      void ping​(ClusterMonitor<T> clusterMonitor, T p, java.util.concurrent.Executor executor)
      Pinging a node, called from ClusterMonitor
      protected Result robustSearch​(Query query, Execution execution, T connection)
      Call search(Query, Execution, T) and handle any exceptions returned which we do not want to propagate upwards By default this catches all runtime exceptions and puts them into the result
      Result search​(Query query, Execution execution)
      Override this to implement your searcher.
      protected Result search​(Query query, Execution execution, ErrorMessage message)
      This is called (instead of search(query, execution, connection) to handle searches where no (suitable) backend was available.
      protected abstract Result search​(Query query, Execution execution, T connection)
      Perform the search against the given connection.
      protected boolean shouldRetry​(Query query, Result result)
      Returns whether this query and result should be retried against another connection if possible.
      protected boolean timedOut​(Query query)
      Returns true if this query has timed out now
      void working​(T node)
      NodeManager method, called from ClusterMonitor

      • Methods inherited from class com.yahoo.component.chain.ChainedComponent

        getAnnotatedDependencies, getDefaultAnnotatedDependencies, getDependencies, initDependencies

      • Methods inherited from class com.yahoo.component.AbstractComponent

        clone, compareTo, getClassName, getId, getIdString, hasInitializedId, initId, isDeconstructable, setIsDeconstructable

      • Methods inherited from class java.lang.Object

        equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    • Constructor Detail

      • ClusterSearcher

        public ClusterSearcher​(com.yahoo.component.ComponentId id,
                       java.util.List<T> connections,
                       boolean internal)
        Creates a new cluster searcher
        Parameters:
        id - the id of this searcher
        connections - the connections of the cluster
        internal - whether or not this cluster is internal (part of the same installation)

      • ClusterSearcher

        public ClusterSearcher​(com.yahoo.component.ComponentId id,
                       java.util.List<T> connections,
                       Hasher<T> hasher,
                       boolean internal)

      • ClusterSearcher

        protected ClusterSearcher​(com.yahoo.component.ComponentId id,
                          java.util.List<T> connections,
                          Hasher<T> hasher,
                          boolean internal,
                          boolean startPingThread)

    • Method Detail

      • ping

        public final void ping​(ClusterMonitor<T> clusterMonitor,
                       T p,
                       java.util.concurrent.Executor executor)
        Pinging a node, called from ClusterMonitor
        Specified by:
        ping in interface NodeManager<T>

      • ping

        protected abstract Pong ping​(Ping ping,
                             T connection)
        Pings this connection. Pings may be sent "out of band" at any time by the monitoring subsystem to determine the status of this connection. If the ping fails, it is ok both to set an error in the pong or to throw an exception.

      • getFirstConnection

        protected T getFirstConnection​(Hasher.NodeList<T> nodes,
                               int code,
                               int trynum,
                               Query query)

      • search

        public final Result search​(Query query,
                           Execution execution)
        Description copied from class: Searcher
        Override this to implement your searcher.

        Searcher implementation subclasses will, depending on their type of logic, do one of the following:

        • Query processors: Access the query, then call execution.search and return the result
        • Result processors: Call execution.search to get the result, access it and return
        • Sources (which produces results): Create a result, add the desired hits and return it.
        • Federators (which forwards the search to multiple subchains): Call search on the desired subchains in parallel and get the results. Combine the results to one and return it.
        • Workflows: Call execution.search as many times as desired, using different queries. Eventually return a result.

        Hits come in two kinds - concrete hits are actual content of the kind requested by the user, meta hits are hits which provides information about the collection of hits, on the query, the service and so on.

        The query specifies a window into a larger result list that must be returned from the searcher through hits and offset; Searchers which returns list of hits in the top level in the result must return at least hits number of hits (or if impossible; all that are available), starting at the given offset. In addition, searchers are allowed to return any number of meta hits (although this number is expected to be low). For hits contained in nested hit groups, the concept of a window defined by hits and offset is not well defined and does not apply.

        Error handling in searchers:

        • Unexpected events: Throw any RuntimeException. This query will fail with the exception message, and the error will be logged
        • Expected events: Create (new Result(Query, ErrorMessage) or add result.setErrorIfNoOtherErrors(ErrorMessage) an error message to the Result.
        • Recoverable user errors: Add a FeedbackHit explaining the condition and how to correct it.
        Specified by:
        search in class Searcher
        Parameters:
        query - the query
        Returns:
        the result of making this query

      • shouldRetry

        protected boolean shouldRetry​(Query query,
                              Result result)
        Returns whether this query and result should be retried against another connection if possible. This default implementation returns true if the result contains some error.

      • search

        protected Result search​(Query query,
                        Execution execution,
                        ErrorMessage message)
        This is called (instead of search(query, execution, connection) to handle searches where no (suitable) backend was available. The default implementation returns an error result.

      • robustSearch

        protected Result robustSearch​(Query query,
                              Execution execution,
                              T connection)
        Call search(Query, Execution, T) and handle any exceptions returned which we do not want to propagate upwards By default this catches all runtime exceptions and puts them into the result

      • search

        protected abstract Result search​(Query query,
                                 Execution execution,
                                 T connection)
        Perform the search against the given connection. Return a result containing an error or throw an exception on failures.

      • fill

        public final void fill​(Result result,
                       java.lang.String summaryClass,
                       Execution execution)
        Description copied from class: Searcher
        Fill hit properties with data using the given summary class. Calling this on already filled results has no cost.

        This needs to be overridden by federating searchers to contact search sources again by propagating the fill call down through the search chain, and by source searchers which talks to fill capable backends to request the data to be filled. Other searchers do not need to override this.

        Overrides:
        fill in class Searcher
        Parameters:
        result - the result to fill
        summaryClass - the name of the collection of fields to fetch the values of, or null to use the default

      • fill

        protected abstract void fill​(Result result,
                             java.lang.String summaryClass,
                             Execution execution,
                             T connection)
        Perform the fill against the given connection. Add an error to the result or throw an exception on failures.

      • working

        public void working​(T node)
        NodeManager method, called from ClusterMonitor
        Specified by:
        working in interface NodeManager<T>

      • failed

        public void failed​(T node)
        NodeManager method, called from ClusterMonitor
        Specified by:
        failed in interface NodeManager<T>

      • getHasher

        public Hasher<T> getHasher()
        Returns the hasher used internally in this. Do not mutate this hasher while in use.

      • getMonitor

        public ClusterMonitor<T> getMonitor()
        Returns the monitor of these nodes

      • timedOut

        protected boolean timedOut​(Query query)
        Returns true if this query has timed out now

      • log

        protected void log​(java.util.logging.Level level,
                   java.lang.Object... objects)

      • deconstruct

        public void deconstruct()
        Overrides:
        deconstruct in class com.yahoo.component.AbstractComponent