com.yahoo.prelude.statistics

Class StatisticsSearcher

java.lang.Object

com.yahoo.component.AbstractComponent

com.yahoo.component.chain.ChainedComponent

com.yahoo.processing.Processor

com.yahoo.search.Searcher

com.yahoo.prelude.statistics.StatisticsSearcher

All Implemented Interfaces:

com.yahoo.component.Component, Comparable<com.yahoo.component.Component>

public class StatisticsSearcher
extends Searcher

A searcher to gather statistics such as queries completed and query latency. There may be more than 1 StatisticsSearcher in the Searcher chain, each identified by a Searcher ID. The statistics accumulated by all StatisticsSearchers are stored in the singleton StatisticsManager object.

TODO: Fix events to handle more than one of these searchers properly.

Author:

Gene Meyers, Steinar Knutsen, bergum

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private class	StatisticsSearcher.ActivitySampler
private static class	StatisticsSearcher.DegradedReason
private class	StatisticsSearcher.PeakQpsReporter

Field Summary

Fields

Modifier and Type	Field and Description
private static String	ACTIVE_QUERIES_METRIC
private com.yahoo.statistics.Value	activeQueries
private Map<String,com.yahoo.jdisc.Metric.Context>	chainContexts
private static String	DEGRADED_METRIC
private Map<String,Map<StatisticsSearcher.DegradedReason,com.yahoo.jdisc.Metric.Context>>	degradedReasonContexts
private static String	DOCS_COVERED_METRIC
private static String	DOCS_TOTAL_METRIC
private static String	EMPTY_RESULTS_METRIC
private com.yahoo.statistics.Counter	emptyResults
private static String	FAILED_QUERIES_METRIC
private com.yahoo.statistics.Counter	failedQueries
private static String	HITS_PER_QUERY_METRIC
private com.yahoo.statistics.Value	hitsPerQuery
private static com.yahoo.processing.request.CompoundName	IGNORE_QUERY
private com.yahoo.statistics.Counter	illegalQueries
private static String	MAX_QUERY_LATENCY_METRIC
private com.yahoo.statistics.Value	maxQueryLatency
private static String	MEAN_QUERY_LATENCY_METRIC
private com.yahoo.jdisc.Metric	metric
private com.yahoo.statistics.Counter	nullQueries
private static String	PEAK_QPS_METRIC
private com.yahoo.statistics.Value	peakQPS
private StatisticsSearcher.PeakQpsReporter	peakQpsReporter
private com.yahoo.statistics.Counter	queries
private static String	QUERIES_METRIC
private static String	QUERY_LATENCY_METRIC
private com.yahoo.statistics.Value	queryLatency
private com.yahoo.statistics.Value	queryLatencyBuckets
private Timer	scheduler
private Map<String,com.yahoo.jdisc.Metric.Context>	statePageOnlyContexts
private static String	TOTALHITS_PER_QUERY_METRIC

Fields inherited from class com.yahoo.component.AbstractComponent

isDeconstructable

Constructor Summary

Constructors

Constructor and Description
StatisticsSearcher(com.yahoo.statistics.Statistics manager, com.yahoo.jdisc.Metric metric, com.yahoo.metrics.simple.MetricReceiver metricReceiver)

Method Summary

Modifier and Type	Method and Description
private void	addLatency(long latency, com.yahoo.jdisc.Metric.Context metricContext)
void	deconstruct()
private com.yahoo.jdisc.Metric.Context	getChainMetricContext(String chainName)
private com.yahoo.jdisc.Metric.Context	getDegradedMetricContext(String chainName, Coverage coverage)
private com.yahoo.jdisc.Metric.Context	getDimensions(String source)
private StatisticsSearcher.DegradedReason	getMostImportantDegradeReason(Coverage coverage)
private void	incrementStatePageOnlyErrors(Result result) Creates error metric for StateHandler only.
private void	incrErrorCount(Result result, com.yahoo.jdisc.Metric.Context metricContext)
private void	incrQueryCount(com.yahoo.jdisc.Metric.Context metricContext)
private void	logQuery(Query query)
private void	qps(com.yahoo.jdisc.Metric.Context metricContext)
Result	search(Query query, Execution execution) Generate statistics for the query passing through this Searcher 1) Add 1 to total query count 2) Add response time to total response time (time from entry to return) 3) .....

Methods inherited from class com.yahoo.search.Searcher

ensureFilled, fill, getLogger, process, toString

Methods inherited from class com.yahoo.component.chain.ChainedComponent

getAnnotatedDependencies, getDefaultAnnotatedDependencies, getDependencies, initDependencies

Methods inherited from class com.yahoo.component.AbstractComponent

clone, compareTo, getClassName, getId, getIdString, hasInitializedId, initId, isDeconstructable, setIsDeconstructable

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

IGNORE_QUERY

private static final com.yahoo.processing.request.CompoundName IGNORE_QUERY

MAX_QUERY_LATENCY_METRIC

private static final String MAX_QUERY_LATENCY_METRIC

See Also:

Constant Field Values

EMPTY_RESULTS_METRIC

private static final String EMPTY_RESULTS_METRIC

See Also:

Constant Field Values

HITS_PER_QUERY_METRIC

private static final String HITS_PER_QUERY_METRIC

See Also:

Constant Field Values

TOTALHITS_PER_QUERY_METRIC

private static final String TOTALHITS_PER_QUERY_METRIC

See Also:

Constant Field Values

FAILED_QUERIES_METRIC

private static final String FAILED_QUERIES_METRIC

See Also:

Constant Field Values

MEAN_QUERY_LATENCY_METRIC

private static final String MEAN_QUERY_LATENCY_METRIC

See Also:

Constant Field Values

QUERY_LATENCY_METRIC

private static final String QUERY_LATENCY_METRIC

See Also:

Constant Field Values

QUERIES_METRIC

private static final String QUERIES_METRIC

See Also:

Constant Field Values

ACTIVE_QUERIES_METRIC

private static final String ACTIVE_QUERIES_METRIC

See Also:

Constant Field Values

PEAK_QPS_METRIC

private static final String PEAK_QPS_METRIC

See Also:

Constant Field Values

DOCS_COVERED_METRIC

private static final String DOCS_COVERED_METRIC

See Also:

Constant Field Values

DOCS_TOTAL_METRIC

private static final String DOCS_TOTAL_METRIC

See Also:

Constant Field Values

DEGRADED_METRIC

private static final String DEGRADED_METRIC

See Also:

Constant Field Values

queries

private final com.yahoo.statistics.Counter queries

failedQueries

private final com.yahoo.statistics.Counter failedQueries

nullQueries

private final com.yahoo.statistics.Counter nullQueries

illegalQueries

private final com.yahoo.statistics.Counter illegalQueries

queryLatency

private final com.yahoo.statistics.Value queryLatency

queryLatencyBuckets

private final com.yahoo.statistics.Value queryLatencyBuckets

maxQueryLatency

private final com.yahoo.statistics.Value maxQueryLatency

activeQueries

private final com.yahoo.statistics.Value activeQueries

peakQPS

private final com.yahoo.statistics.Value peakQPS

emptyResults

private final com.yahoo.statistics.Counter emptyResults

hitsPerQuery

private final com.yahoo.statistics.Value hitsPerQuery

peakQpsReporter

private final StatisticsSearcher.PeakQpsReporter peakQpsReporter

metric

private com.yahoo.jdisc.Metric metric

chainContexts

private Map<String,com.yahoo.jdisc.Metric.Context> chainContexts

statePageOnlyContexts

private Map<String,com.yahoo.jdisc.Metric.Context> statePageOnlyContexts

degradedReasonContexts

private Map<String,Map<StatisticsSearcher.DegradedReason,com.yahoo.jdisc.Metric.Context>> degradedReasonContexts

scheduler

private Timer scheduler

Constructor Detail

StatisticsSearcher

public StatisticsSearcher(com.yahoo.statistics.Statistics manager,
                          com.yahoo.jdisc.Metric metric,
                          com.yahoo.metrics.simple.MetricReceiver metricReceiver)

Method Detail

deconstruct

public void deconstruct()

Overrides:

deconstruct in class com.yahoo.component.AbstractComponent

qps

private void qps(com.yahoo.jdisc.Metric.Context metricContext)

getChainMetricContext

private com.yahoo.jdisc.Metric.Context getChainMetricContext(String chainName)

getDegradedMetricContext

private com.yahoo.jdisc.Metric.Context getDegradedMetricContext(String chainName,
                                                                Coverage coverage)

getMostImportantDegradeReason

private StatisticsSearcher.DegradedReason getMostImportantDegradeReason(Coverage coverage)

search

public Result search(Query query,
                     Execution execution)

Generate statistics for the query passing through this Searcher 1) Add 1 to total query count 2) Add response time to total response time (time from entry to return) 3) .....

Specified by:

search in class Searcher

Parameters:

query - the query

Returns:

the result of making this query

logQuery

private void logQuery(Query query)

addLatency

private void addLatency(long latency,
                        com.yahoo.jdisc.Metric.Context metricContext)

incrQueryCount

private void incrQueryCount(com.yahoo.jdisc.Metric.Context metricContext)

incrErrorCount

private void incrErrorCount(Result result,
                            com.yahoo.jdisc.Metric.Context metricContext)

incrementStatePageOnlyErrors

private void incrementStatePageOnlyErrors(Result result)

Creates error metric for StateHandler only. These metrics are only exposed on /state/v1/metrics page and not forwarded to the log file.

Parameters:

result - The result to check for errors

getDimensions

private com.yahoo.jdisc.Metric.Context getDimensions(String source)