Package com.yahoo.searchdefinition.parser

Class SDParser

java.lang.Object

com.yahoo.searchdefinition.parser.SDParser

All Implemented Interfaces:

SDParserConstants

public class SDParser
extends Object
implements SDParserConstants

The schema parser

Author:

bratseth

Field Summary

Fields

Modifier and Type	Field	Description
Token	jj_nt	Next token.
Token	token	Current token.
SDParserTokenManager	token_source	Generated Token Manager.

Fields inherited from interface com.yahoo.searchdefinition.parser.SDParserConstants

ALIAS, ALWAYS, ANNOTATION, ANNOTATIONREFERENCE, ARITY, ARRAY, AS, ASCENDING, ATTRIBUTE, BODY, BOLDING, BRACE_ML_CONTENT, BRACE_ML_LEVEL_1, BRACE_ML_LEVEL_2, BRACE_ML_LEVEL_3, BRACE_SL_CONTENT, BRACE_SL_LEVEL_1, BRACE_SL_LEVEL_2, BRACE_SL_LEVEL_3, BTREE, CASED, COLON, COMMA, COMPRESSION, COMPRESSIONLEVEL, COMPRESSIONTHRESHOLD, CONSTANT, CONSTANTS, CONTEXT, CREATEIFNONEXISTENT, CUTOFF_FACTOR, CUTOFF_STRATEGY, DEFAULT, DENSEPOSTINGLISTTHRESHOLD, DESCENDING, DICTIONARY, DIRECT, DISTANCEMETRIC, DIVERSITY, DOCUMENT, DOCUMENTSUMMARY, DOT, DOUBLE, DYNAMIC, ENABLE_BM25, ENABLEBITVECTORS, ENABLEONLYBITVECTOR, EOF, EVALUATION_POINT, EXACT, EXACTTERMINATOR, EXPRESSION_ML, EXPRESSION_SL, FALSE, FASTACCESS, FASTSEARCH, FIELD, FIELDS, FIELDSET, FILE, FILE_PATH, FILTER, FIRSTPHASE, FROMDISK, FULL, FUNCTION, GRAM, GRAMSIZE, GREATERTHAN, HASH, HEADER, HNSW, HTTP, HUGE, ID, IDENTICAL, IDENTIFIER, IDENTIFIER_WITH_DASH, IGNOREDEFAULTRANKFEATURES, IMPORT, INDEX, INDEXING, INDEXINGREWRITE, INHERITS, INLINE, INTEGER, KEEPRANKCOUNT, LBRACE, LESSTHAN, LITERAL, LOCALE, LONG, LOOSE, LOWERBOUND, LOWERCASE, LZ4, MACRO, MAP, MATCH, MATCHEDELEMENTSONLY, MATCHFEATURES_ML, MATCHFEATURES_ML_INHERITS, MATCHFEATURES_SL, MATCHPHASE, MAXFILTERCOVERAGE, MAXHITS, MAXLENGTH, MAXLINKSPERNODE, MIN_GROUPS, MINHITSPERTHREAD, MODEL, MTOKEN, MULTITHREADEDINDEXING, MUTABLE, MUTATE, NEIGHBORSTOEXPLOREATINSERT, NEVER, NL, NONE, NORMAL, NORMALIZING, NUMSEARCHPARTITIONS, NUMTHREADSPERSEARCH, OFF, OMITSUMMARYFEATURES, ON, ON_FIRST_PHASE, ON_MATCH, ON_SECOND_PHASE, ON_SUMMARY, ONDEMAND, ONNX_INPUT_SL, ONNX_OUTPUT_SL, ONNXMODEL, OPERATION, ORDER, PAGED, PRE_POST_FILTER_TIPPING_POINT, PREFIX, PRIMARY, PROPERTIES, QUATERNARY, QUERYCOMMAND, QUESTIONMARK, QUOTEDSTRING, RANK, RANKDEGRADATION, RANKDEGRADATIONFREQ, RANKFEATURES_ML, RANKFEATURES_SL, RANKPROFILE, RANKPROPERTIES, RANKSCOREDROPLIMIT, RANKTYPE, RAW, RAW_AS_BASE64_IN_SUMMARY, RBRACE, REFERENCE, REMOVEIFZERO, RERANKCOUNT, RPBINLOW, RPBINSIZE, RPPOSBINSIZE, SCHEMA, SEARCH, SEARCHLIB_SKIP, SECONDARY, SECONDPHASE, SINGLE_LINE_COMMENT, SORTING, SOURCE, SSCONTEXTUAL, SSOVERRIDE, SSTITLE, SSURL, STATIC, STEMMING, STRENGTH, STRICT, STRING, STRUCT, STRUCTFIELD, SUBSTRING, SUFFIX, SUMMARY, SUMMARYFEATURES_ML, SUMMARYFEATURES_ML_INHERITS, SUMMARYFEATURES_SL, SUMMARYTO, SYMMETRIC, TENSOR_TYPE, TENSOR_VALUE_ML, TENSOR_VALUE_SL, TERMWISELIMIT, TERTIARY, TEXT, TO, tokenImage, TRUE, TYPE, UCA, UNCASED, UPPERBOUND, URI, URI_PATH, USEDOCUMENT, VARIABLE, WEIGHT, WEIGHTEDSET, WORD

Constructor Summary

Constructors

Constructor	Description
SDParser(CharStream stream)	Constructor with user supplied CharStream.
SDParser(SDParserTokenManager tm)	Constructor with generated Token Manager.
SDParser(SimpleCharStream stream, com.yahoo.config.application.api.ApplicationPackage applicationPackage, com.yahoo.config.application.api.FileRegistry fileRegistry, com.yahoo.config.application.api.DeployLogger deployLogger, com.yahoo.config.model.api.ModelContext.Properties properties, RankProfileRegistry rankProfileRegistry, boolean documentsOnly)	Creates a parser

Method Summary

Modifier and Type	Method	Description
final void	alias(FieldOperationContainer container)
final void	annotation(Schema schema, SDDocumentType document)	Consumes a annotation block from within either a document element.
final SDAnnotationType	annotationBody(Schema schema, SDDocumentType repo, SDAnnotationType type)	Consumes a single element of an annotation body block.
final void	annotationOutside(Schema schema)	This rule consumes a annotation block from within either a document element or a schema element.
final com.yahoo.document.DataType	annotationRefDataType()
final void	attribute(FieldOperationContainer field)	This rule consumes an attribute statement of a field element.
final void	attributeSetting(FieldOperationContainer field, AttributeOperation attribute, String attributeName)	This rule consumes a single attribute setting statement of an attribute element.
final void	attributeTensorType(AttributeOperation attribute)	This rule consumes a tensor type statement for an attribute element.
final void	body(SDField field)	This rule consumes a body statement of a field element.
final void	bodycfg(SDDocumentType document)	Consumes a document body block.
final void	bolding(FieldOperationContainer field)	This rule consumes a bolding statement of a field element.
final Boolean	bool()	This rule consumes a boolean value.
final void	compression(SDDocumentType document)	Consumes a compression block.
final void	compressionItem()	Consumes the body of a compression block.
final void	constants(RankProfile profile)	Consumes a set of constants available in ranking expressions in the enclosing profile.
final void	constantTensor(RankProfile profile, String name)
final String	constantTensorErrorMessage(String rankProfileName, String constantTensorName)
final void	constantValue(RankProfile profile, String name)
final double	consumeFloat()	This rule consumes a floating-point token and returns its numeric value.
final long	consumeLong()	This rule consumes a long or integer token and returns its numeric value.
final Number	consumeNumber()
final com.yahoo.document.DataType	dataType()	This rule consumes a data type block from within a field element.
final void	dictionary(FieldOperationContainer field)	This rule consumes a dictionary statement of a field element.
final void	dictionarySetting(FieldOperationContainer field)
final void	disable_tracing()	Disable tracing.
final void	diversity(RankProfile.MatchPhaseSettings profile)	This rule consumes a diversity block of a rank profile.
final void	diversityItem(RankProfile.DiversitySettings settings)
final void	document(Schema schema)	Consumes a document element.
final void	documentBody(SDDocumentType document, Schema schema)	Consumes a document body block
final void	documentSummary(Schema schema)	Consumes a document-summary block from within a schema block.
final void	documentSummaryItem(DocumentSummary summary)	Consumes a single document-summary item.
final void	enable_tracing()	Enable tracing.
final void	exactTerminator(FieldOperationContainer container)
final String	expression()	This rule consumes an expression token and returns its image.
final void	field(SDDocumentType document, Schema schema)	Consumes a field block from within a document element.
final void	fieldBody(SDField field, Schema schema, SDDocumentType document)	This rule consumes a single element of a field body block.
final void	fieldRankFilter(RankProfile profile)	This rule consumes a rank filter statement of a rank profile.
final void	fieldRankType(RankProfile profile)	This rule consumes a rank-type statement of a rank profile.
final void	fieldSet(Schema schema)
final void	fieldStemming(FieldOperationContainer field)	This rule consumes a stemming block of a field element.
final void	fieldWeight(RankProfile profile)	This rule consumes a field-weight statement of a rank profile.
final String	fileItem()
final void	firstPhase(RankProfile profile)	Consumes the first-phase block of a rank profile.
final void	firstPhaseItem(RankProfile profile)
final void	function(RankProfile profile)	This rule consumes a function statement of a rank-profile.
ParseException	generateParseException()	Generate ParseException.
final Token	getNextToken()
final Token	getToken(int index)
final void	gramSize(FieldOperationContainer container)
final void	header(SDField field)	This rule consumes a header statement of a field element.
final void	headercfg(SDDocumentType document)	Consumes a document head block.
final void	hnswIndex(IndexOperation index)
final void	hnswIndexBody(HnswIndexParams.Builder params)
final void	id(FieldOperationContainer field, SDDocumentType document)	Consumes an id statement of a field body block.
final String	identifier()	Consumes an identifier.
final String	identifierWithDash()
final void	ignoreRankFeatures(RankProfile profile)	This rule consumes a ignore-default-rank-features statement for a rank profile.
final void	importField(Schema schema)
final void	index(Schema schema, FieldOperationContainer field)	Consumes an index block for a field element.
final void	indexBody(IndexOperation index)	Consumes a single index statement for an index block.
final void	indexing(FieldOperationContainer field)	This rule consumes an indexing block of a field element.
final void	indexingOperation(FieldOperationContainer field, boolean multiLine)	This rule consumes an IL script block.
final void	indexingRewrite(FieldOperationContainer field)	Consumes an indexing-rewrite statement of a field body block.
final void	inheritsAnnotation(SDAnnotationType annotation)
final void	inheritsDocument(SDDocumentType document)	Consumes a document inheritance statement.
final void	inheritsDocumentSummary(DocumentSummary documentSummary, Schema schema)	This rule consumes an inherits statement of a document summary.
final void	inheritsRankProfile(RankProfile profile)	This rule consumes an inherits statement of a rank-profile.
final boolean	inline()
final int	integer()	This rule consumes an integer token and returns its numeric value.
final void	lbrace()	This rule consumes an opening brace with leading and trailing newline tokens.
final com.yahoo.document.DataType	mapDataType()
final FieldOperationContainer	match(FieldOperationContainer field)
final void	matchFeatures(RankProfile profile)	This rule consumes a match-features block of a rank profile.
final void	matchItem(FieldOperationContainer field)	This rule consumes a single match item for a match block.
final void	matchPhase(RankProfile profile)	This rule consumes a match-phase block of a rank profile.
final void	matchPhaseItem(RankProfile.MatchPhaseSettings settings)
final void	matchSize(FieldOperationContainer container)
final void	matchType(FieldOperationContainer container)
final void	minHitsPerThread(RankProfile profile)	This rule consumes a min-hits-per-thread statement for a rank profile.
final void	mutate(RankProfile profile)	This rule consumes an mutate statement of a rank-profile.
final String	mutate_expr()
final void	mutate_operation(RankProfile profile)
final void	namedDocument(Schema schema)	Consumes a document element, explicitly named
final void	normalizing(FieldOperationContainer field)	This rule consumes a normalizing statement of a field element.
final void	numSearchPartitions(RankProfile profile)	This rule consumes a num-search-partitions statement for a rank profile.
final void	numThreadsPerSearch(RankProfile profile)	This rule consumes a num-threads-per-search statement for a rank profile.
final void	onnxModel(Schema schema)	Consumes a onnx-model block of a schema element.
final void	onnxModelItem(OnnxModel onnxModel)	This rule consumes an onnx-model block.
final void	queryCommand(FieldOperationContainer container)
final String	quotedString()	Consumes a quoted string token and returns the token image minus the quotes.
final void	rank(FieldOperationContainer field)	Consumes a rank statement of a field element.
final void	rankDegradation(RankProfile profile)	This rule consumes a rank-degradation statement of a rank profile.
final void	rankDegradationBinLow()	This rule consumes part of a rank-degradation statement of a rank profile.
final void	rankDegradationBinSize()	This rule consumes part of a rank-degradation statement of a rank profile.
final void	rankDegradationItem()	This rule consumes part of a rank-degradation statement of a rank profile.
final void	rankDegradationPosbinSize()	This rule consumes part of a rank-degradation statement of a rank profile.
final void	rankFeatures(RankProfile profile)	Consumes a rank-features block of a rank profile
final void	rankingConstant(Schema schema)	Consumes a constant block of a schema element.
final String	rankingConstantErrorMessage(String name)
final void	rankProfile(Schema schema)	Consumes a rank-profile block of a schema element.
final void	rankProfileItem(RankProfile profile)	This rule consumes a single statement for a rank-profile block.
final void	rankProperties(RankProfile profile)	This rule consumes a rank-properties block of a rank profile.
final void	rankProperty(RankProfile profile)	This rule consumes a single rank property pair for a rank profile.
final String	rankPropertyItem()	This rule consumes a single rank property for a rank-properties block.
final void	rankSetting(RankOperation field)	Consumes a single rank setting of a rank statement.
final void	rankType(FieldOperationContainer field)	This rule consumes a rank-type statement of a field element.
final void	rawAsBase64(Schema schema)
void	ReInit(CharStream stream)	Reinitialise.
void	ReInit(SDParserTokenManager tm)	Reinitialise
final Schema	rootDocument()	Consumes a schema definition that contains only documents to be used for inheritance, etc.
final void	rootDocumentItem(Schema schema)	Consumes a single item from within a root document node.
final Schema	rootSchema()	This rule consumes a proper schema block.
final void	rootSchemaItem(Schema schema)	Consumes an element of a schema block.
final Schema	schema(com.yahoo.document.DocumentTypeManager docMan)	The rule consumes any schema and returns the corresponding object.
final void	searchStemming(Schema schema)	This rule consumes a stemming statement for a schema element.
final void	secondPhase(RankProfile profile)	Consumes the second-phase block of a rank profile.
final void	secondPhaseItem(RankProfile profile)	Consumes a statement for a second-phase block.
final void	sorting(FieldOperationContainer field, String name)
final void	sortingSetting(SortingOperation sorting, String attributeName)
final void	strict(RankProfile profile)
final String	string()	Consumes a string token and returns the token image.
final SDDocumentType	structDefinition(Schema schema, SDDocumentType repo)	This rule consumes a struct block from within a document element.
final void	structField(FieldOperationContainer field, Schema schema, SDDocumentType document)	This rule consumes a struct subfield from a document field body.
final void	structFieldBody(FieldOperationContainer field, Schema schema, SDDocumentType document)	This rule consumes a single element of a struct subfield body block.
final void	structFieldDefinition(SDDocumentType document, SDDocumentType struct)	This rule consumes a field block of a struct body.
final void	structInside(SDDocumentType document, Schema schema)	This rule consumes a struct block from within a document element.
final void	structOutside(Schema schema)	This rule consumes a struct block from within a document element.
final void	summaryBolding(SummaryInFieldLongOperation field)	This rule consumes a bolding statement for a summary field element.
final void	summaryDestinationList(SummaryInFieldLongOperation field)	This rule consumes a destination-list statement for a summary field element.
final void	summaryFeatures(RankProfile profile)	This rule consumes a summary-features block of a rank profile.
final void	summaryInDocument(DocumentSummary document)	This rule consumes a summary statement defined inside a document-summary block.
final void	summaryInField(FieldOperationContainer field)	The rule consumes a summary statement defined inside a field.
final SummaryInFieldOperation	summaryInFieldLong(FieldOperationContainer field)	This rule consumes a multi-line summary field.
final SummaryInFieldOperation	summaryInFieldShort(FieldOperationContainer field)	This rule consumes a single-line summary field.
final void	summaryItem(SummaryInFieldLongOperation field)	This rule consumes an item of a summary field block.
final void	summaryProperties(SummaryInFieldLongOperation field)	This rule consumes properties for a summary field element.
final void	summaryProperty(SummaryInFieldLongOperation field)	This rule consumes a single summary property pair for a summary field element.
final void	summarySourceList(SummaryInFieldOperation field)	This rule consumes a source-list statement for a summary field element.
final void	summaryTo(FieldOperationContainer field)	This rule consumes a summary-to statement of a field element.
final void	summaryTransform(SummaryInFieldOperation field)	This rule consumes a transform statement for a summary field element.
final com.yahoo.tensor.TensorType	tensorType(String errorMessage)
final com.yahoo.tensor.TensorType	tensorTypeWithPrefix(String errorMessage)
final String	tensorValue()
final void	termwiseLimit(RankProfile profile)	This rule consumes a num-threads-per-search statement for a rank profile.
final boolean	trace_enabled()
final String	uriItem()
final void	useDocument(Schema schema)	Consumes a use-document statement.
final void	weight(FieldOperationContainer field)	This rule consumes a weight statement of a field element.
final void	weightedset(FieldOperationContainer field)	This rule consumes a weighted set statement of a field element.
final void	weightedsetBody(WeightedSetOperation field)	This rule consumes one body item of a weighted set block.
final com.yahoo.document.DataType	wildCardType()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

token_source

public SDParserTokenManager token_source

Generated Token Manager.

token

public Token token

Current token.

jj_nt

public Token jj_nt

Next token.

Constructor Details

SDParser

public SDParser(SimpleCharStream stream,
 com.yahoo.config.application.api.ApplicationPackage applicationPackage,
 com.yahoo.config.application.api.FileRegistry fileRegistry,
 com.yahoo.config.application.api.DeployLogger deployLogger,
 com.yahoo.config.model.api.ModelContext.Properties properties,
 RankProfileRegistry rankProfileRegistry,
 boolean documentsOnly)

Creates a parser

Parameters:

documentsOnly - true to only parse the document aspect of a schema (e.g skip rank profiles)

SDParser

public SDParser(CharStream stream)

Constructor with user supplied CharStream.

Parameters:

stream - stream to init with

SDParser

public SDParser(SDParserTokenManager tm)

Constructor with generated Token Manager.

Parameters:

tm - Token manager to use

Method Details

schema

public final Schema schema(com.yahoo.document.DocumentTypeManager docMan)
                    throws ParseException

The rule consumes any schema and returns the corresponding object. This is the only production that should ever consume leading newlines.

Returns:

the schema object

Throws:

ParseException

rootSchema

public final Schema rootSchema()
                        throws ParseException

This rule consumes a proper schema block. This and rootDocument() are the only rules that should ever consume trailing newline tokens.

Returns:

the schema definition object.

Throws:

ParseException

rootSchemaItem

public final void rootSchemaItem(Schema schema)
                          throws ParseException

Consumes an element of a schema block. This and rootSearch() are the only rules that should ever consume trailing newline tokens.

Parameters:

schema - the schema object to modify.

Throws:

ParseException

rootDocument

public final Schema rootDocument()
                          throws ParseException

Consumes a schema definition that contains only documents to be used for inheritance, etc.

Returns:

the schema definition object.

Throws:

ParseException

rootDocumentItem

public final void rootDocumentItem(Schema schema)
                            throws ParseException

Consumes a single item from within a root document node.

Parameters:

schema - the schema object to modify.

Throws:

ParseException

useDocument

public final void useDocument(Schema schema)
                       throws ParseException

Consumes a use-document statement. This currently does nothing.

Parameters:

schema - the schema object to modify.

Throws:

ParseException

document

public final void document(Schema schema)
                    throws ParseException

Consumes a document element. The name defaults to the schema's name, but may be set.

Parameters:

schema - the schema object to add content to.

Throws:

ParseException

namedDocument

public final void namedDocument(Schema schema)
                         throws ParseException

Consumes a document element, explicitly named

Parameters:

schema - the schema object to add content to

Throws:

ParseException

documentBody

public final void documentBody(SDDocumentType document,
 Schema schema)
                        throws ParseException

Consumes a document body block

Parameters:

document - the document type to modify.

schema - the schema object to add content to

Throws:

ParseException

rawAsBase64

public final void rawAsBase64(Schema schema)
                       throws ParseException

Throws:

ParseException

headercfg

public final void headercfg(SDDocumentType document)
                     throws ParseException

Consumes a document head block.

Parameters:

document - The document type to modify.

Throws:

ParseException

bodycfg

public final void bodycfg(SDDocumentType document)
                   throws ParseException

Consumes a document body block.

Parameters:

document - The document type to modify.

Throws:

ParseException

compression

public final void compression(SDDocumentType document)
                       throws ParseException

Consumes a compression block. This can be set in both document header and -body block.

Parameters:

document - The document type to modify.

Throws:

ParseException

compressionItem

public final void compressionItem()
                           throws ParseException

Consumes the body of a compression block.

Throws:

ParseException

inheritsDocument

public final void inheritsDocument(SDDocumentType document)
                            throws ParseException

Consumes a document inheritance statement.

Parameters:

document - The document type to modify.

Throws:

ParseException

field

public final void field(SDDocumentType document,
 Schema schema)
                 throws ParseException

Consumes a field block from within a document element.

Parameters:

document - the document type to modify

schema - the schema object to add content to

Throws:

ParseException

fieldSet

public final void fieldSet(Schema schema)
                    throws ParseException

Throws:

ParseException

annotationOutside

public final void annotationOutside(Schema schema)
                             throws ParseException

This rule consumes a annotation block from within either a document element or a schema element.

Parameters:

schema - the schema object to add content to

Throws:

ParseException

annotation

public final void annotation(Schema schema,
 SDDocumentType document)
                      throws ParseException

Consumes a annotation block from within either a document element.

Parameters:

document - the document object to add content to

Throws:

ParseException

annotationBody

public final SDAnnotationType annotationBody(Schema schema,
 SDDocumentType repo,
 SDAnnotationType type)
                                      throws ParseException

Consumes a single element of an annotation body block.

Parameters:

schema - the schema object to add content to

type - the type being built

Returns:

a modified or new AnnotationType instance

Throws:

ParseException

inheritsAnnotation

public final void inheritsAnnotation(SDAnnotationType annotation)
                              throws ParseException

Throws:

ParseException

structInside

public final void structInside(SDDocumentType document,
 Schema schema)
                        throws ParseException

This rule consumes a struct block from within a document element.

Parameters:

schema - the schema object to add content to

Throws:

ParseException

structOutside

public final void structOutside(Schema schema)
                         throws ParseException

This rule consumes a struct block from within a document element.

Parameters:

schema - the schema object to add content to

Throws:

ParseException

structDefinition

public final SDDocumentType structDefinition(Schema schema,
 SDDocumentType repo)
                                      throws ParseException

This rule consumes a struct block from within a document element.

Parameters:

schema - the schema object to add content to

Throws:

ParseException

dataType

public final com.yahoo.document.DataType dataType()
                                           throws ParseException

This rule consumes a data type block from within a field element.

Returns:

the consumed data type

Throws:

ParseException

annotationRefDataType

public final com.yahoo.document.DataType annotationRefDataType()
                                                        throws ParseException

Throws:

ParseException

mapDataType

public final com.yahoo.document.DataType mapDataType()
                                              throws ParseException

Throws:

ParseException

wildCardType

public final com.yahoo.document.DataType wildCardType()
                                               throws ParseException

Throws:

ParseException

structFieldDefinition

public final void structFieldDefinition(SDDocumentType document,
 SDDocumentType struct)
                                 throws ParseException

This rule consumes a field block of a struct body.

Parameters:

struct - The struct to modify.

Throws:

ParseException

structField

public final void structField(FieldOperationContainer field,
 Schema schema,
 SDDocumentType document)
                       throws ParseException

This rule consumes a struct subfield from a document field body. This is not to be confused with a document struct's fields, but rather this is a subfield of a document field of type struct.

Parameters:

field - the field to modify

schema - the schema object to add content to

document - the document type to modify

Throws:

ParseException

fieldBody

public final void fieldBody(SDField field,
 Schema schema,
 SDDocumentType document)
                     throws ParseException

This rule consumes a single element of a field body block.

Parameters:

field - the field being built

schema - the schema object to add content to

document - the owning document, or null if this is a search field

Throws:

ParseException

structFieldBody

public final void structFieldBody(FieldOperationContainer field,
 Schema schema,
 SDDocumentType document)
                           throws ParseException

This rule consumes a single element of a struct subfield body block. Only elements that are supported in streaming schema and indexed schema (with complex attributes) are allowed.

Parameters:

field - the field being built

schema - the schema object to add content to

document - the owning document, or null if this is a schema field

Throws:

ParseException

indexing

public final void indexing(FieldOperationContainer field)
                    throws ParseException

This rule consumes an indexing block of a field element.

Parameters:

field - The field to modify.

Throws:

ParseException

indexingOperation

public final void indexingOperation(FieldOperationContainer field,
 boolean multiLine)
                             throws ParseException

This rule consumes an IL script block. This is expected to consume trailing newlines.

Parameters:

field - The field to modify.

Throws:

ParseException

summaryTo

public final void summaryTo(FieldOperationContainer field)
                     throws ParseException

This rule consumes a summary-to statement of a field element.

Parameters:

field - The field to modify.

Throws:

ParseException

weight

public final void weight(FieldOperationContainer field)
                  throws ParseException

This rule consumes a weight statement of a field element.

Parameters:

field - The field to modify.

Throws:

ParseException

weightedset

public final void weightedset(FieldOperationContainer field)
                       throws ParseException

This rule consumes a weighted set statement of a field element.

Parameters:

field - The field to modify.

Throws:

ParseException

weightedsetBody

public final void weightedsetBody(WeightedSetOperation field)
                           throws ParseException

This rule consumes one body item of a weighted set block.

Parameters:

field - The field to modify.

Throws:

ParseException

rankType

public final void rankType(FieldOperationContainer field)
                    throws ParseException

This rule consumes a rank-type statement of a field element.

Parameters:

field - The field to modify.

Throws:

ParseException

attribute

public final void attribute(FieldOperationContainer field)
                     throws ParseException

This rule consumes an attribute statement of a field element.

Parameters:

field - The field to modify.

Throws:

ParseException

sorting

public final void sorting(FieldOperationContainer field,
 String name)
                   throws ParseException

Throws:

ParseException

sortingSetting

public final void sortingSetting(SortingOperation sorting,
 String attributeName)
                          throws ParseException

Throws:

ParseException

attributeSetting

public final void attributeSetting(FieldOperationContainer field,
 AttributeOperation attribute,
 String attributeName)
                            throws ParseException

This rule consumes a single attribute setting statement of an attribute element.

Parameters:

field - The field to modify.

attributeName - The name of the attribute to change.

Throws:

ParseException

attributeTensorType

public final void attributeTensorType(AttributeOperation attribute)
                               throws ParseException

This rule consumes a tensor type statement for an attribute element.

Parameters:

attribute - The attribute to modify.

Throws:

ParseException

summaryInDocument

public final void summaryInDocument(DocumentSummary document)
                             throws ParseException

This rule consumes a summary statement defined inside a document-summary block.

Parameters:

document - The document summary to modify.

Throws:

ParseException

summaryInField

public final void summaryInField(FieldOperationContainer field)
                          throws ParseException

The rule consumes a summary statement defined inside a field.

Parameters:

field - The field to modify.

Throws:

ParseException

summaryInFieldShort

public final SummaryInFieldOperation summaryInFieldShort(FieldOperationContainer field)
                                                  throws ParseException

This rule consumes a single-line summary field.

Parameters:

field - The field to modify.

Returns:

The consumed summary field.

Throws:

ParseException

summaryInFieldLong

public final SummaryInFieldOperation summaryInFieldLong(FieldOperationContainer field)
                                                 throws ParseException

This rule consumes a multi-line summary field.

Returns:

The consumed summary field.

Throws:

ParseException

summaryItem

public final void summaryItem(SummaryInFieldLongOperation field)
                       throws ParseException

This rule consumes an item of a summary field block.

Parameters:

field - The field to modify.

Throws:

ParseException

summaryTransform

public final void summaryTransform(SummaryInFieldOperation field)
                            throws ParseException

This rule consumes a transform statement for a summary field element.

Parameters:

field - The field to modify.

Throws:

ParseException

summaryBolding

public final void summaryBolding(SummaryInFieldLongOperation field)
                          throws ParseException

This rule consumes a bolding statement for a summary field element.

Parameters:

field - The summary field to modify.

Throws:

ParseException

summarySourceList

public final void summarySourceList(SummaryInFieldOperation field)
                             throws ParseException

This rule consumes a source-list statement for a summary field element.

Parameters:

field - The summary field to modify.

Throws:

ParseException

summaryDestinationList

public final void summaryDestinationList(SummaryInFieldLongOperation field)
                                  throws ParseException

This rule consumes a destination-list statement for a summary field element.

Parameters:

field - The summary field to modify.

Throws:

ParseException

summaryProperties

public final void summaryProperties(SummaryInFieldLongOperation field)
                             throws ParseException

This rule consumes properties for a summary field element.

Parameters:

field - The summary field to modify.

Throws:

ParseException

summaryProperty

public final void summaryProperty(SummaryInFieldLongOperation field)
                           throws ParseException

This rule consumes a single summary property pair for a summary field element.

Parameters:

field - The summary field to modify.

Throws:

ParseException

fieldStemming

public final void fieldStemming(FieldOperationContainer field)
                         throws ParseException

This rule consumes a stemming block of a field element.

Parameters:

field - The field to modify.

Throws:

ParseException

searchStemming

public final void searchStemming(Schema schema)
                          throws ParseException

This rule consumes a stemming statement for a schema element.

Parameters:

schema - the schema to modify

Throws:

ParseException

normalizing

public final void normalizing(FieldOperationContainer field)
                       throws ParseException

This rule consumes a normalizing statement of a field element. At the moment, this can only be used to turn off normalizing.

Parameters:

field - The field to modify.

Throws:

ParseException

bolding

public final void bolding(FieldOperationContainer field)
                   throws ParseException

This rule consumes a bolding statement of a field element.

Parameters:

field - The field to modify.

Throws:

ParseException

dictionary

public final void dictionary(FieldOperationContainer field)
                      throws ParseException

This rule consumes a dictionary statement of a field element.

Parameters:

field - The field to modify.

Throws:

ParseException

dictionarySetting

public final void dictionarySetting(FieldOperationContainer field)
                             throws ParseException

Throws:

ParseException

body

public final void body(SDField field)
                throws ParseException

This rule consumes a body statement of a field element.

Parameters:

field - The field to modify.

Throws:

ParseException

header

public final void header(SDField field)
                  throws ParseException

This rule consumes a header statement of a field element.

Parameters:

field - The field to modify.

Throws:

ParseException

queryCommand

public final void queryCommand(FieldOperationContainer container)
                        throws ParseException

Throws:

ParseException

alias

public final void alias(FieldOperationContainer container)
                 throws ParseException

Throws:

ParseException

match

public final FieldOperationContainer match(FieldOperationContainer field)
                                    throws ParseException

Throws:

ParseException

matchItem

public final void matchItem(FieldOperationContainer field)
                     throws ParseException

This rule consumes a single match item for a match block.

Parameters:

field - The field to modify.

Throws:

ParseException

matchType

public final void matchType(FieldOperationContainer container)
                     throws ParseException

Throws:

ParseException

exactTerminator

public final void exactTerminator(FieldOperationContainer container)
                           throws ParseException

Throws:

ParseException

gramSize

public final void gramSize(FieldOperationContainer container)
                    throws ParseException

Throws:

ParseException

matchSize

public final void matchSize(FieldOperationContainer container)
                     throws ParseException

Throws:

ParseException

rank

public final void rank(FieldOperationContainer field)
                throws ParseException

Consumes a rank statement of a field element.

Parameters:

field - The field to modify.

Throws:

ParseException

rankSetting

public final void rankSetting(RankOperation field)
                       throws ParseException

Consumes a single rank setting of a rank statement.

Parameters:

field - The field to modify.

Throws:

ParseException

id

public final void id(FieldOperationContainer field,
 SDDocumentType document)
              throws ParseException

Consumes an id statement of a field body block.

Parameters:

field - The field to modify.

document - The document type to modify.

Throws:

ParseException

indexingRewrite

public final void indexingRewrite(FieldOperationContainer field)
                           throws ParseException

Consumes an indexing-rewrite statement of a field body block.

Parameters:

field - The field to modify.

Throws:

ParseException

documentSummary

public final void documentSummary(Schema schema)
                           throws ParseException

Consumes a document-summary block from within a schema block.

Parameters:

schema - the schema object to add content to

Throws:

ParseException

inheritsDocumentSummary

public final void inheritsDocumentSummary(DocumentSummary documentSummary,
 Schema schema)
                                   throws ParseException

This rule consumes an inherits statement of a document summary.

Parameters:

documentSummary - the document summary to modify

schema - the schema object documentSummary is being added to

Throws:

ParseException

documentSummaryItem

public final void documentSummaryItem(DocumentSummary summary)
                               throws ParseException

Consumes a single document-summary item.

Parameters:

summary - The document summary to modify.

Throws:

ParseException

index

public final void index(Schema schema,
 FieldOperationContainer field)
                 throws ParseException

Consumes an index block for a field element.

Parameters:

schema - the schema object to add content to

field - the field to modify

Throws:

ParseException

indexBody

public final void indexBody(IndexOperation index)
                     throws ParseException

Consumes a single index statement for an index block.

Parameters:

index - The index to modify.

Throws:

ParseException

hnswIndex

public final void hnswIndex(IndexOperation index)
                     throws ParseException

Throws:

ParseException

hnswIndexBody

public final void hnswIndexBody(HnswIndexParams.Builder params)
                         throws ParseException

Throws:

ParseException

onnxModel

public final void onnxModel(Schema schema)
                     throws ParseException

Consumes a onnx-model block of a schema element.

Parameters:

schema - the schema object to add content to.

Throws:

ParseException

onnxModelItem

public final void onnxModelItem(OnnxModel onnxModel)
                         throws ParseException

This rule consumes an onnx-model block.

Parameters:

onnxModel - The onnxModel to modify.

Throws:

ParseException

rankingConstant

public final void rankingConstant(Schema schema)
                           throws ParseException

Consumes a constant block of a schema element.

Parameters:

schema - the schema object to add content to

Throws:

ParseException

fileItem

public final String fileItem()
                      throws ParseException

Throws:

ParseException

uriItem

public final String uriItem()
                     throws ParseException

Throws:

ParseException

rankingConstantErrorMessage

public final String rankingConstantErrorMessage(String name)
                                         throws ParseException

Throws:

ParseException

rankProfile

public final void rankProfile(Schema schema)
                       throws ParseException

Consumes a rank-profile block of a schema element.

Parameters:

schema - the schema object to add content to

Throws:

ParseException

rankProfileItem

public final void rankProfileItem(RankProfile profile)
                           throws ParseException

This rule consumes a single statement for a rank-profile block.

Parameters:

profile - The rank profile to modify.

Throws:

ParseException

inheritsRankProfile

public final void inheritsRankProfile(RankProfile profile)
                               throws ParseException

This rule consumes an inherits statement of a rank-profile.

Parameters:

profile - the profile to modify

Throws:

ParseException

mutate

public final void mutate(RankProfile profile)
                  throws ParseException

This rule consumes an mutate statement of a rank-profile.

Parameters:

profile - The profile to modify.

Throws:

ParseException

mutate_operation

public final void mutate_operation(RankProfile profile)
                            throws ParseException

Throws:

ParseException

mutate_expr

public final String mutate_expr()
                         throws ParseException

Throws:

ParseException

function

public final void function(RankProfile profile)
                    throws ParseException

This rule consumes a function statement of a rank-profile.

Parameters:

profile - The profile to modify.

Throws:

ParseException

inline

public final boolean inline()
                     throws ParseException

Throws:

ParseException

matchPhase

public final void matchPhase(RankProfile profile)
                      throws ParseException

This rule consumes a match-phase block of a rank profile.

Parameters:

profile - The rank profile to modify.

Throws:

ParseException

matchPhaseItem

public final void matchPhaseItem(RankProfile.MatchPhaseSettings settings)
                          throws ParseException

Throws:

ParseException

diversity

public final void diversity(RankProfile.MatchPhaseSettings profile)
                     throws ParseException

This rule consumes a diversity block of a rank profile.

Parameters:

profile - The rank profile to modify.

Throws:

ParseException

diversityItem

public final void diversityItem(RankProfile.DiversitySettings settings)
                         throws ParseException

Throws:

ParseException

firstPhase

public final void firstPhase(RankProfile profile)
                      throws ParseException

Consumes the first-phase block of a rank profile.

Parameters:

profile - The rank profile to modify.

Throws:

ParseException

firstPhaseItem

public final void firstPhaseItem(RankProfile profile)
                          throws ParseException

Throws:

ParseException

secondPhase

public final void secondPhase(RankProfile profile)
                       throws ParseException

Consumes the second-phase block of a rank profile.

Parameters:

profile - The rank profile to modify.

Throws:

ParseException

secondPhaseItem

public final void secondPhaseItem(RankProfile profile)
                           throws ParseException

Consumes a statement for a second-phase block.

Parameters:

profile - The rank profile to modify.

Throws:

ParseException

summaryFeatures

public final void summaryFeatures(RankProfile profile)
                           throws ParseException

This rule consumes a summary-features block of a rank profile.

Parameters:

profile - The rank profile to modify.

Throws:

ParseException

strict

public final void strict(RankProfile profile)
                  throws ParseException

Throws:

ParseException

matchFeatures

public final void matchFeatures(RankProfile profile)
                         throws ParseException

This rule consumes a match-features block of a rank profile.

Parameters:

profile - The rank profile to modify.

Throws:

ParseException

rankFeatures

public final void rankFeatures(RankProfile profile)
                        throws ParseException

Consumes a rank-features block of a rank profile

Throws:

ParseException

ignoreRankFeatures

public final void ignoreRankFeatures(RankProfile profile)
                              throws ParseException

This rule consumes a ignore-default-rank-features statement for a rank profile.

Parameters:

profile - The rank profile to modify.

Throws:

ParseException

numThreadsPerSearch

public final void numThreadsPerSearch(RankProfile profile)
                               throws ParseException

This rule consumes a num-threads-per-search statement for a rank profile.

Parameters:

profile - The rank profile to modify.

Throws:

ParseException

minHitsPerThread

public final void minHitsPerThread(RankProfile profile)
                            throws ParseException

This rule consumes a min-hits-per-thread statement for a rank profile.

Parameters:

profile - The rank profile to modify.

Throws:

ParseException

numSearchPartitions

public final void numSearchPartitions(RankProfile profile)
                               throws ParseException

This rule consumes a num-search-partitions statement for a rank profile.

Parameters:

profile - the rank profile to modify

Throws:

ParseException

termwiseLimit

public final void termwiseLimit(RankProfile profile)
                         throws ParseException

This rule consumes a num-threads-per-search statement for a rank profile.

Parameters:

profile - the rank profile to modify

Throws:

ParseException

rankProperties

public final void rankProperties(RankProfile profile)
                          throws ParseException

This rule consumes a rank-properties block of a rank profile. There is a little trick within this rule to allow the final rank property to skip the terminating newline token.

Parameters:

profile - the rank profile to modify

Throws:

ParseException

rankProperty

public final void rankProperty(RankProfile profile)
                        throws ParseException

This rule consumes a single rank property pair for a rank profile.

Parameters:

profile - the rank profile to modify

Throws:

ParseException

rankPropertyItem

public final String rankPropertyItem()
                              throws ParseException

This rule consumes a single rank property for a rank-properties block.

Returns:

The token image of the consumed item.

Throws:

ParseException

fieldWeight

public final void fieldWeight(RankProfile profile)
                       throws ParseException

This rule consumes a field-weight statement of a rank profile.

Parameters:

profile - The rank profile to modify.

Throws:

ParseException

fieldRankType

public final void fieldRankType(RankProfile profile)
                         throws ParseException

This rule consumes a rank-type statement of a rank profile.

Parameters:

profile - The rank profile to modify.

Throws:

ParseException

fieldRankFilter

public final void fieldRankFilter(RankProfile profile)
                           throws ParseException

This rule consumes a rank filter statement of a rank profile.

Parameters:

profile - The rank profile to modify.

Throws:

ParseException

rankDegradationBinSize

public final void rankDegradationBinSize()
                                  throws ParseException

This rule consumes part of a rank-degradation statement of a rank profile.

Throws:

ParseException

rankDegradationBinLow

public final void rankDegradationBinLow()
                                 throws ParseException

This rule consumes part of a rank-degradation statement of a rank profile.

Throws:

ParseException

rankDegradationPosbinSize

public final void rankDegradationPosbinSize()
                                     throws ParseException

This rule consumes part of a rank-degradation statement of a rank profile.

Throws:

ParseException

rankDegradationItem

public final void rankDegradationItem()
                               throws ParseException

This rule consumes part of a rank-degradation statement of a rank profile.

Throws:

ParseException

rankDegradation

public final void rankDegradation(RankProfile profile)
                           throws ParseException

This rule consumes a rank-degradation statement of a rank profile.

Parameters:

profile - The rank profile to modify.

Throws:

ParseException

constants

public final void constants(RankProfile profile)
                     throws ParseException

Consumes a set of constants available in ranking expressions in the enclosing profile.

Throws:

ParseException

constantValue

public final void constantValue(RankProfile profile,
 String name)
                         throws ParseException

Throws:

ParseException

constantTensor

public final void constantTensor(RankProfile profile,
 String name)
                          throws ParseException

Throws:

ParseException

constantTensorErrorMessage

public final String constantTensorErrorMessage(String rankProfileName,
 String constantTensorName)
                                        throws ParseException

Throws:

ParseException

tensorValue

public final String tensorValue()
                         throws ParseException

Throws:

ParseException

tensorTypeWithPrefix

public final com.yahoo.tensor.TensorType tensorTypeWithPrefix(String errorMessage)
                                                       throws ParseException

Throws:

ParseException

tensorType

public final com.yahoo.tensor.TensorType tensorType(String errorMessage)
                                             throws ParseException

Throws:

ParseException

importField

public final void importField(Schema schema)
                       throws ParseException

Throws:

ParseException

expression

public final String expression()
                        throws ParseException

This rule consumes an expression token and returns its image.

Returns:

The consumed token image.

Throws:

ParseException

identifierWithDash

public final String identifierWithDash()
                                throws ParseException

Throws:

ParseException

identifier

public final String identifier()
                        throws ParseException

Consumes an identifier. This must be kept in sync with all word tokens that should be parseable as identifiers.

Returns:

the identifier string

Throws:

ParseException

string

public final String string()
                    throws ParseException

Consumes a string token and returns the token image.

Returns:

The consumed token image.

Throws:

ParseException

quotedString

public final String quotedString()
                          throws ParseException

Consumes a quoted string token and returns the token image minus the quotes. This does not perform unescaping of the content, it simply removes the first and last character of the image. However, the token itself can contain anything but a double quote.

Returns:

The unquoted token image.

Throws:

ParseException

bool

public final Boolean bool()
                   throws ParseException

This rule consumes a boolean value.

Returns:

The consumed boolean value.

Throws:

ParseException

integer

public final int integer()
                  throws ParseException

This rule consumes an integer token and returns its numeric value.

Returns:

The consumed integer value.

Throws:

ParseException

consumeLong

public final long consumeLong()
                       throws ParseException

This rule consumes a long or integer token and returns its numeric value.

Returns:

The consumed long value.

Throws:

ParseException

consumeFloat

public final double consumeFloat()
                          throws ParseException

This rule consumes a floating-point token and returns its numeric value.

Returns:

The consumed value.

Throws:

ParseException

consumeNumber

public final Number consumeNumber()
                           throws ParseException

Throws:

ParseException

lbrace

public final void lbrace()
                  throws ParseException

This rule consumes an opening brace with leading and trailing newline tokens.

Throws:

ParseException

ReInit

public void ReInit(CharStream stream)

Reinitialise.

Parameters:

stream - stream to init with

ReInit

public void ReInit(SDParserTokenManager tm)

Reinitialise

Parameters:

tm - Token manager to use

getNextToken

public final Token getNextToken()

Returns:

the next Token.

getToken

public final Token getToken(int index)

Parameters:

index - index to be retrieved

Returns:

the specific Token.

generateParseException

public ParseException generateParseException()

Generate ParseException.

Returns:

new Exception object. Never null

trace_enabled

public final boolean trace_enabled()

Returns:

Always false.

enable_tracing

public final void enable_tracing()

Enable tracing.

disable_tracing

public final void disable_tracing()

Disable tracing.