com.yahoo.search.grouping

Class UniqueGroupingSearcher

java.lang.Object

com.yahoo.component.AbstractComponent

com.yahoo.component.chain.ChainedComponent

com.yahoo.processing.Processor

com.yahoo.search.Searcher

com.yahoo.search.grouping.UniqueGroupingSearcher

All Implemented Interfaces:

com.yahoo.component.Component, Comparable<com.yahoo.component.Component>

public class UniqueGroupingSearcher
extends Searcher

Implements 'unique' using a grouping expression. It doesn't work for multi-level sorting.

Author:

andreer

Field Summary

Fields

Modifier and Type	Field and Description
(package private) static String	LABEL_COUNT
(package private) static String	LABEL_GROUPS
(package private) static String	LABEL_HITS
private static Logger	log
private static HitOrderer	NOP_ORDERER
static com.yahoo.processing.request.CompoundName	PARAM_UNIQUE

Fields inherited from class com.yahoo.component.AbstractComponent

isDeconstructable

Constructor Summary

Constructors

Constructor and Description
UniqueGroupingSearcher()

Method Summary

Modifier and Type	Method and Description
(package private) static GroupingOperation	buildGroupingExpression(String dedupField, int groupingHits, String summaryClass, Sorting sortSpec)
private static GroupingOperation	buildGroupingExpressionWithRanking(String dedupField, int groupingHits, String summaryClass) Create the grouping expression when ranking is used for ordering (which is the default for grouping expressions, so ranking is not explicitly mentioned).
private static GroupingOperation	buildGroupingExpressionWithSorting(String dedupField, int groupingHits, String summaryClass, Sorting sortSpec) Create the grouping expression when sorting is used for ordering This grouping expression is more complicated and probably quite a bit heavier to execute.
private static GroupingExpression	createGroupOrderingClause(Sorting sortingSpec) Create a hit ordering clause based on the sorting spec.
private static List<GroupingExpression>	createHitOrderingClause(Sorting sortingSpec) Create a hit ordering clause based on the sorting spec.
private static Result	dedupe(Query query, Execution execution, String dedupField) Until we can use the grouping pagination features in 5.1, we'll have to support offset by simply requesting and discarding hit #0 up to hit #offset.
private static List<Hit>	getAllHitsFromGroupingResult(GroupList resultGroups) Get all the hits returned by the grouping request.
private static List<Hit>	getRequestedHits(GroupList resultGroups, int offset, int hits) Retrieve the actually unique hits from the grouping results.
Result	search(Query query, Execution execution) Implements the deprecated "unique" api for deduplication by using grouping.

Methods inherited from class com.yahoo.search.Searcher

ensureFilled, fill, getLogger, process, toString

Methods inherited from class com.yahoo.component.chain.ChainedComponent

getAnnotatedDependencies, getDefaultAnnotatedDependencies, getDependencies, initDependencies

Methods inherited from class com.yahoo.component.AbstractComponent

clone, compareTo, deconstruct, getClassName, getId, getIdString, hasInitializedId, initId, isDeconstructable, setIsDeconstructable

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

PARAM_UNIQUE

public static final com.yahoo.processing.request.CompoundName PARAM_UNIQUE

log

private static final Logger log

NOP_ORDERER

private static final HitOrderer NOP_ORDERER

LABEL_COUNT

static final String LABEL_COUNT

See Also:

Constant Field Values

LABEL_GROUPS

static final String LABEL_GROUPS

See Also:

Constant Field Values

LABEL_HITS

static final String LABEL_HITS

See Also:

Constant Field Values

Constructor Detail

UniqueGroupingSearcher

public UniqueGroupingSearcher()

Method Detail

search

public Result search(Query query,
                     Execution execution)

Implements the deprecated "unique" api for deduplication by using grouping. We create a grouping expression on the field we wish to dedup on (which must be an attribute). Total hits is calculated using the new count unique groups functionality.

Specified by:

search in class Searcher

Parameters:

query - the query

Returns:

the result of making this query

dedupe

private static Result dedupe(Query query,
                             Execution execution,
                             String dedupField)

Until we can use the grouping pagination features in 5.1, we'll have to support offset by simply requesting and discarding hit #0 up to hit #offset.

createHitOrderingClause

private static List<GroupingExpression> createHitOrderingClause(Sorting sortingSpec)

Create a hit ordering clause based on the sorting spec.

Parameters:

sortingSpec - A (single level!) sorting specification

Returns:

a grouping expression which produces a sortable value

createGroupOrderingClause

private static GroupingExpression createGroupOrderingClause(Sorting sortingSpec)

Create a hit ordering clause based on the sorting spec.

Parameters:

sortingSpec - A (single level!) sorting specification

Returns:

a grouping expression which produces a sortable value

getRequestedHits

private static List<Hit> getRequestedHits(GroupList resultGroups,
                                          int offset,
                                          int hits)

Retrieve the actually unique hits from the grouping results.

Parameters:

resultGroups - the results of the dedup grouping expression.

offset - the requested offset. Hits before this are discarded.

hits - the requested number of hits. Hits in excess of this are discarded.

Returns:

A list of the actually requested hits, sorted as by the grouping expression.

getAllHitsFromGroupingResult

private static List<Hit> getAllHitsFromGroupingResult(GroupList resultGroups)

Get all the hits returned by the grouping request. This might be more or less than the user requested. This method handles the results from two different types of grouping expression, depending on whether sorting was used for the query or not.

Parameters:

resultGroups - The result group of the dedup grouping request

Returns:

A (correctly sorted) list of all the hits returned by the grouping expression.

buildGroupingExpression

static GroupingOperation buildGroupingExpression(String dedupField,
                                                 int groupingHits,
                                                 String summaryClass,
                                                 Sorting sortSpec)

buildGroupingExpressionWithRanking

private static GroupingOperation buildGroupingExpressionWithRanking(String dedupField,
                                                                    int groupingHits,
                                                                    String summaryClass)

Create the grouping expression when ranking is used for ordering (which is the default for grouping expressions, so ranking is not explicitly mentioned). See unit test for examples

buildGroupingExpressionWithSorting

private static GroupingOperation buildGroupingExpressionWithSorting(String dedupField,
                                                                    int groupingHits,
                                                                    String summaryClass,
                                                                    Sorting sortSpec)

Create the grouping expression when sorting is used for ordering This grouping expression is more complicated and probably quite a bit heavier to execute. See unit test for examples