Package com.yahoo.prelude.query.parser

Class AbstractParser

java.lang.Object

com.yahoo.prelude.query.parser.AbstractParser

All Implemented Interfaces:

CustomParser, Parser

Direct Known Subclasses:

AdvancedParser, AllParser, AnyParser, PhraseParser

public abstract class AbstractParser
extends Object
implements CustomParser

The Vespa query parser.

Author:

bratseth, Steinar Knutsen

Field Summary

Fields

Modifier and Type	Field	Description
protected int	braceLevelURL	The counter for braces in URLs, braces in URLs are accepted so long as they are balanced.
protected ParserEnvironment	environment
protected IndexFacts.Session	indexFacts	The IndexFacts.Session of this query
protected com.yahoo.language.Language	language	The current language of this parser.
protected com.yahoo.prelude.query.parser.AbstractParser.Submodes	submodes	The current submodes of this parser
protected com.yahoo.prelude.query.parser.TokenPosition	tokens

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	AbstractParser(ParserEnvironment environment)	Creates a new instance of this class, storing the given ParserEnvironment for parse-time access to the environment.

Method Summary

Modifier and Type	Method	Description
protected String	normalize(String input)	Unicode normalizes some piece of natural language text.
QueryTree	parse(Parsable query)	Parser the given Parsable, and returns a corresponding QueryTree.
Item	parse(String queryToParse, String filterToParse, com.yahoo.language.Language parsingLanguage, IndexFacts.Session indexFacts, String defaultIndexName)
protected abstract Item	parseItems()
protected Item	segment(String indexName, Token token, boolean quoted)	Segments a token
protected void	setState(com.yahoo.language.Language queryLanguage, IndexFacts.Session indexFacts)
static Item	simplifyPhrases(Item unwashed)	Collapses single item phrases in the tree to the contained item.
protected void	tokenize(String query, String defaultIndexName, IndexFacts.Session indexFacts, com.yahoo.language.Language language)	Tokenizes the given string and initializes tokens with the found tokens.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.yahoo.prelude.query.parser.CustomParser

parse

Field Detail

submodes

protected com.yahoo.prelude.query.parser.AbstractParser.Submodes submodes

The current submodes of this parser

language

protected com.yahoo.language.Language language

The current language of this parser. Used to decide whether and how to use the CJKSegmenter

indexFacts

protected IndexFacts.Session indexFacts

The IndexFacts.Session of this query

braceLevelURL

protected int braceLevelURL

The counter for braces in URLs, braces in URLs are accepted so long as they are balanced.

environment

protected final ParserEnvironment environment

tokens

protected final com.yahoo.prelude.query.parser.TokenPosition tokens

Constructor Detail

AbstractParser

protected AbstractParser(ParserEnvironment environment)

Creates a new instance of this class, storing the given ParserEnvironment for parse-time access to the environment.

Parameters:

environment - the environment settings to attach to the Parser

Method Detail

parse

public final QueryTree parse(Parsable query)

Description copied from interface: Parser

Parser the given Parsable, and returns a corresponding QueryTree. If parsing fails without an exception, the contained root will be an instance of NullItem.

Specified by:

parse in interface Parser

Parameters:

query - the Parsable to parse

Returns:

the parsed QueryTree, never null

parse

public final Item parse(String queryToParse,
                        String filterToParse,
                        com.yahoo.language.Language parsingLanguage,
                        IndexFacts.Session indexFacts,
                        String defaultIndexName)

Specified by:

parse in interface CustomParser

parseItems

protected abstract Item parseItems()

normalize

protected String normalize(String input)

Unicode normalizes some piece of natural language text. The chosen form is compatibility decomposition, canonical composition (NFKC).

setState

protected void setState(com.yahoo.language.Language queryLanguage,
                        IndexFacts.Session indexFacts)

tokenize

protected void tokenize(String query,
                        String defaultIndexName,
                        IndexFacts.Session indexFacts,
                        com.yahoo.language.Language language)

Tokenizes the given string and initializes tokens with the found tokens.

Parameters:

query - the string to tokenize

defaultIndexName - the name of the index to use as default

indexFacts - resolved information about the index we are searching

language - the language set for this query, or null if none

simplifyPhrases

public static Item simplifyPhrases(Item unwashed)

Collapses single item phrases in the tree to the contained item.

Parameters:

unwashed - The item whose phrases to simplify.

Returns:

The simplified item.

segment

protected Item segment(String indexName,
                       Token token,
                       boolean quoted)

Segments a token

Parameters:

indexName - the index name which preceeded this token, or null if none

token - the token to segment

quoted - whether this segment is within quoted text

Returns:

the resulting item