Package org.antlr.tool
Class Grammar
- java.lang.Object
-
- org.antlr.tool.Grammar
-
public class Grammar extends java.lang.Object
Represents a grammar in memory.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
Grammar.Decision
class
Grammar.LabelElementPair
-
Field Summary
Fields Modifier and Type Field Description protected boolean
allDecisionDFACreated
static java.lang.String[]
ANTLRLiteralCharValueEscape
Given a char, we need to be able to show as an ANTLR literal.static int[]
ANTLRLiteralEscapedCharValue
When converting ANTLR char and string literals, here is the value set of escape chars.static java.lang.String
ARTIFICIAL_TOKENS_RULENAME
boolean
atLeastOneBacktrackOption
At least one backtrack=true in rule or decision or grammar.boolean
atLeastOneRuleMemoizes
At least one rule has memoize=truestatic java.lang.String
AUTO_GENERATED_TOKEN_NAME_PREFIX
java.util.Set<GrammarAST>
blocksWithSemPreds
Track decisions with syn preds specified for reporting.java.util.Set<GrammarAST>
blocksWithSynPreds
Track decisions with syn preds specified for reporting.protected boolean
builtFromString
We need a way to detect when a lexer grammar is autogenerated from another grammar or we are just sending in a string representing a grammar.static int
CHAR_LABEL
protected IntSet
charVocabulary
TODO: hook this to the charVocabulary optionstatic int
COMBINED
CompositeGrammar
composite
If this grammar is part of a larger composite grammar via delegate statement, then this points at the composite.CompositeGrammarTree
compositeTreeNode
A pointer back into grammar tree.protected int
decisionCount
Be able to assign a number to every decision in grammar; decisions in 1..njava.util.Set<DFA>
decisionsWhoseDFAsUsesSemPreds
Track decisions that actually use the syn preds in the DFA.java.util.Set<DFA>
decisionsWhoseDFAsUsesSynPreds
Track decisions that actually use the syn preds in the DFA.static java.util.Map<java.lang.String,java.lang.String>
defaultBlockOptions
What are the default options for a subrule?static java.util.Map<java.lang.String,java.lang.String>
defaultLexerBlockOptions
static java.util.Map<java.lang.String,java.lang.String>
defaultOptions
static java.lang.String
defaultTokenOption
protected java.util.Set<Rule>
delegatedRuleReferences
The list of all rules referenced in this grammar, not defined here, and defined in a delegate grammar.long
DFACreationWallClockTimeInMS
How long in ms did it take to build DFAs for this grammar? If this grammar is a combined grammar, it only records time for the parser grammar component.static java.util.Set<java.lang.String>
doNotCopyOptionsToLexer
protected boolean
externalAnalysisAbort
An external tool requests that DFA analysis abort prematurely.protected NFAFactory
factory
protected java.lang.String
fileName
What file name holds this grammar?static java.lang.String
FRAGMENT_RULE_MODIFIER
protected CodeGenerator
generator
If non-null, this is the code generator we will use to generate recognizers in the target language.protected int
global_k
Is there a global fixed lookahead set for this grammar? If 0, nothing specified.static java.lang.String
GRAMMAR_FILE_EXTENSION
protected GrammarAST
grammarTree
An AST that records entire input grammar with all rules.static java.lang.String[]
grammarTypeToFileNameSuffix
static java.lang.String[]
grammarTypeToString
static java.lang.String
IGNORE_STRING_IN_GRAMMAR_FILE_NAME
boolean
implicitLexer
Was this created from a COMBINED grammar?protected java.util.Vector<Grammar.Decision>
indexToDecision
Each subrule/rule is a decision point and we must track them so we can go back later and build DFA predictors for them.static int
INITIAL_DECISION_LIST_SIZE
static int
INVALID_RULE_INDEX
java.lang.String
label
If this is a delegate of another grammar, this is the label used as an instance var by that grammar to point at this grammar.static java.lang.String[]
LabelTypeToString
protected java.util.Set<Rule>
leftRecursiveRules
A list of all rules that are in any left-recursive cycle.static java.util.Set<java.lang.String>
legalBlockOptions
static java.util.Set<java.lang.String>
legalLexerOptions
static java.util.Set<java.lang.String>
legalParserOptions
static java.util.Set<java.lang.String>
legalTokenOptions
Legal options for terminal refs like ID<node=MyVarNode>static java.util.Set<java.lang.String>
legalTreeParserOptions
static int
LEXER
static java.lang.String
LEXER_GRAMMAR_FILE_EXTENSION
used for generating lexer temp filesprotected org.stringtemplate.v4.ST
lexerGrammarST
protected java.lang.String
lexerGrammarTemplate
For merged lexer/parsers, we must construct a separate lexer spec.java.util.List<java.lang.String>
lexerRuleNamesInCombined
The ANTLRParser tracks lexer rules when reading combined grammars so we can build the Tokens rule.LL1Analyzer
ll1Analyzer
java.lang.String
name
What name did the user provide for this grammar?NameSpaceChecker
nameSpaceChecker
protected java.util.LinkedHashMap<java.lang.String,Rule>
nameToRuleMap
Map a rule to it's Rule objectprotected java.util.LinkedHashMap<java.lang.String,GrammarAST>
nameToSynpredASTMap
When we read in a grammar, we track the list of syntactic predicates and build faux rules for them later.NFA
nfa
The NFA that represents the grammar with edges labelled with tokens or epsilon.int
numberOfManualLookaheadOptions
int
numberOfSemanticPredicates
int
numNonLLStar
protected java.util.Map<java.lang.String,java.lang.Object>
options
A list of options specified at the grammar level such as language=Java.java.util.Set<java.lang.String>
overriddenRules
If this rule is a delegate, some rules might be overridden; don't want to gen code for them.static int
PARSER
java.util.List<java.lang.String>
precRuleInitCodeBlocks
Each left-recursive precedence rule must define precedence array for binary operators like: static int[] e_prec = new int[tokenNames.length]; static { e_prec[75] = 1; } Track and we push into parser later; this is computed early when we look for prec rules.static int
RULE_LABEL
static int
RULE_LIST_LABEL
protected java.util.Set<GrammarAST>
ruleRefs
The unique set of all rule references in any rule; set of tree node objects so two refs to same rule can exist but at different line/position.protected java.util.Set<GrammarAST>
scopedRuleRefs
protected java.util.Map<java.lang.String,AttributeScope>
scopes
Track the scopes defined outside of rules and the scopes associated with all rules (even if empty).java.util.Set<java.lang.Integer>
setOfNondeterministicDecisionNumbers
java.util.Set<java.lang.Integer>
setOfNondeterministicDecisionNumbersResolvedWithPredicates
static java.lang.String
SYNPRED_RULE_PREFIX
static java.lang.String
SYNPREDGATE_ACTION_NAME
java.util.Set<java.lang.String>
synPredNamesUsedInDFA
Track names of preds so we can avoid generating preds that aren't used Computed during NFA to DFA conversion.static int
TOKEN_LABEL
static int
TOKEN_LIST_LABEL
org.antlr.runtime.CommonTokenStream
tokenBuffer
This is the buffer of *all* tokens found in the grammar file including whitespace tokens etc...protected java.util.Set<org.antlr.runtime.Token>
tokenIDRefs
The unique set of all token ID references in any ruleTool
tool
static int
TREE_PARSER
int
type
What type of grammar is this: lexer, parser, tree walkerstatic MultiMap<java.lang.Integer,java.lang.Integer>
validDelegations
Set of valid imports.static int
WILDCARD_TREE_LABEL
static int
WILDCARD_TREE_LIST_LABEL
-
Constructor Summary
Constructors Constructor Description Grammar()
Useful for when you are sure that you are not part of a composite already.Grammar(java.lang.String grammarString)
Used for testing; only useful on noncomposite grammars.Grammar(Tool tool)
Grammar(Tool tool, java.lang.String grammarString)
Used for testing and Interp/RandomPhrase.Grammar(Tool tool, java.lang.String fileName, CompositeGrammar composite)
Create a grammar from file name.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description GrammarAST
addArtificialMatchTokensRule(GrammarAST grammarAST, java.util.List<java.lang.String> ruleNames, java.util.List<java.lang.String> delegateNames, boolean filterMode)
Parse a rule we add artificially that is a list of the other lexer rules like this: "Tokens : ID | INT | SEMI ;" nextToken() will invoke this to set the current token.protected void
addDelegateGrammar(Grammar delegateGrammar)
add new delegate to composite treevoid
addRule(GrammarAST grammarTree, GrammarAST t)
void
addRulesForSyntacticPredicates()
boolean
allDecisionDFAHaveBeenCreated()
void
altReferencesRule(java.lang.String enclosingRuleName, GrammarAST refScopeAST, GrammarAST refAST, int outerAltNum)
Track a rule reference within an outermost alt of a rule.void
altReferencesTokenID(java.lang.String ruleName, GrammarAST refAST, int outerAltNum)
Track a token reference within an outermost alt of a rule.int
assignDecisionNumber(NFAState state)
boolean
buildAST()
void
buildNFA()
boolean
buildTemplate()
java.util.List<? extends java.util.Collection<? extends Rule>>
checkAllRulesForLeftRecursion()
void
checkAllRulesForUselessLabels()
Remove all labels on rule refs whose target rules have no return value.void
checkNameSpaceAndActions()
ANALYZE ACTIONS, LOOKING FOR LABEL AND ATTR REFS, sanity checkvoid
checkRuleReference(GrammarAST scopeAST, GrammarAST refAST, GrammarAST argsAST, java.lang.String currentRuleName)
IntSet
complement(int atom)
IntSet
complement(IntSet set)
For lexer grammars, return everything in unicode not in set.java.lang.String
computeTokenNameFromLiteral(int tokenType, java.lang.String literal)
given a token type and the text of the literal, come up with a decent token type label.protected Grammar.Decision
createDecision(int decision)
DFA
createLL_1_LookaheadDFA(int decision)
DFA
createLookaheadDFA(int decision, boolean wackTempStructures)
void
createLookaheadDFAs()
For each decision in this grammar, compute a single DFA using the NFA states associated with the decision.void
createLookaheadDFAs(boolean wackTempStructures)
AttributeScope
createParameterScope(java.lang.String ruleName, org.antlr.runtime.Token argAction)
AttributeScope
createReturnScope(java.lang.String ruleName, org.antlr.runtime.Token retAction)
AttributeScope
createRuleScope(java.lang.String ruleName, org.antlr.runtime.Token scopeAction)
void
createRuleStartAndStopNFAStates()
Define all the rule begin/end NFAStates to solve forward reference issues.protected void
dealWithTreeFilterMode()
AttributeScope
defineGlobalScope(java.lang.String name, org.antlr.runtime.Token scopeAction)
void
defineGrammarSymbols()
protected void
defineLabel(Rule r, org.antlr.runtime.Token label, GrammarAST element, int type)
Define a label defined in a rule r; check the validity then ask the Rule object to actually define it.void
defineLexerRuleForAliasedStringLiteral(java.lang.String tokenID, java.lang.String literal, int tokenType)
If someone does PLUS='+' in the parser, must make sure we get "PLUS : '+' ;" in lexer not "T73 : '+';"void
defineLexerRuleForStringLiteral(java.lang.String literal, int tokenType)
void
defineLexerRuleFoundInParser(org.antlr.runtime.Token ruleToken, GrammarAST ruleAST)
void
defineNamedAction(GrammarAST ampersandAST, java.lang.String scope, GrammarAST nameAST, GrammarAST actionAST)
Given @scope::name {action} define it for this grammar.void
defineRule(org.antlr.runtime.Token ruleToken, java.lang.String modifier, java.util.Map<java.lang.String,java.lang.Object> options, GrammarAST tree, GrammarAST argActionAST, int numAlts)
Define a new rule.void
defineRuleListLabel(java.lang.String ruleName, org.antlr.runtime.Token label, GrammarAST element)
void
defineRuleRefLabel(java.lang.String ruleName, org.antlr.runtime.Token label, GrammarAST ruleRef)
java.lang.String
defineSyntacticPredicate(GrammarAST blockAST, java.lang.String currentRuleName)
Define a new predicate and get back its name for use in building a semantic predicate reference to the syn pred.void
defineToken(java.lang.String text, int tokenType)
Define a token at a particular token type value.void
defineTokenListLabel(java.lang.String ruleName, org.antlr.runtime.Token label, GrammarAST element)
void
defineTokenRefLabel(java.lang.String ruleName, org.antlr.runtime.Token label, GrammarAST tokenRef)
void
defineWildcardTreeLabel(java.lang.String ruleName, org.antlr.runtime.Token label, GrammarAST tokenRef)
void
defineWildcardTreeListLabel(java.lang.String ruleName, org.antlr.runtime.Token label, GrammarAST tokenRef)
protected void
examineAllExecutableActions()
Before generating code, we examine all actions that can have $x.y and $y stuff in them because some code generation depends on Rule.referencedPredefinedRuleAttributes.void
externallyAbortNFAToDFAConversion()
Terminate DFA creation (grammar analysis).protected GrammarAST
findFirstTypeOutsideRewrite(GrammarAST block, java.util.BitSet types)
LookaheadSet
FIRST(NFAState s)
boolean
generateMethodForRule(java.lang.String ruleName)
Should codegen.g gen rule for ruleName? If synpred, only gen if used in a DFA.java.util.Map<java.lang.String,java.util.Map<java.lang.String,java.lang.Object>>
getActions()
IntSet
getAllCharValues()
If there is a char vocabulary, use it; else return min to max char as defined by the target.java.util.Set<? extends Rule>
getAllImportedRules()
Get set of all rules imported from all delegate grammars even if indirectly delegated.static java.lang.String
getANTLRCharLiteralForChar(int c)
Return a string representing the escaped char for code c.protected java.util.List<? extends GrammarAST>
getArtificialRulesForSyntacticPredicates(java.util.LinkedHashMap<java.lang.String,GrammarAST> nameToSynpredASTMap)
for any syntactic predicates, we need to define rules for them; they will get defined automatically like any other rule.boolean
getAutoBacktrackMode(int decision)
java.lang.Object
getBlockOption(GrammarAST blockAST, java.lang.String key)
static int
getCharValueFromGrammarCharLiteral(java.lang.String literal)
Given a literal like (the 3 char sequence with single quotes) 'a', return the int value of 'a'.CodeGenerator
getCodeGenerator()
protected Grammar.Decision
getDecision(int decision)
GrammarAST
getDecisionBlockAST(int decision)
NFAState
getDecisionNFAStartState(int decision)
java.util.List<NFAState>
getDecisionNFAStartStateList()
java.util.List<Grammar.Decision>
getDecisions()
java.lang.String
getDefaultActionScope(int grammarType)
Given a grammar type, what should be the default action scope? If I say @members in a COMBINED grammar, for example, the default scope should be "parser".java.util.Set<Rule>
getDelegatedRuleReferences()
java.util.Set<? extends Rule>
getDelegatedRules()
Get the set of Rules that need to have manual delegations like "void rule() { importedGrammar.rule(); }" If this grammar is master, get list of all rule definitions from all delegate grammars.java.util.List<java.lang.String>
getDelegateNames()
java.util.List<Grammar>
getDelegates()
Get list of all delegates from all grammars directly or indirectly imported into this grammar.Grammar
getDelegator()
Who's my direct parent grammar?java.util.List<Grammar>
getDelegators()
Get list of all delegators.java.util.List<Grammar>
getDirectDelegates()
java.lang.String
getFileName()
AttributeScope
getGlobalScope(java.lang.String name)
Get a global scopejava.util.Map<java.lang.String,AttributeScope>
getGlobalScopes()
boolean
getGrammarIsRoot()
int
getGrammarMaxLookahead()
GrammarAST
getGrammarTree()
java.lang.String
getGrammarTypeString()
boolean
getHasDelegates()
java.lang.String
getImplicitlyGeneratedLexerFileName()
java.util.List<Grammar>
getIndirectDelegates()
Get delegates below direct delegatesjava.util.Set<java.lang.String>
getLabels(java.util.Set<GrammarAST> rewriteElements, int labelType)
Given a set of all rewrite elements on right of ->, filter for label types such as Grammar.TOKEN_LABEL, Grammar.TOKEN_LIST_LABEL, ...java.util.Set<Rule>
getLeftRecursiveRules()
Return a list of left-recursive rules; no analysis can be done successfully on these.java.lang.String
getLexerGrammar()
If the grammar is a combined grammar, return the text of the implicit lexer grammar.java.util.Map<java.lang.String,DFA>
getLineColumnToLookaheadDFAMap()
java.lang.Object
getLocallyDefinedOption(java.lang.String key)
Rule
getLocallyDefinedRule(java.lang.String ruleName)
DFA
getLookaheadDFA(int decision)
java.util.List<java.lang.Integer>
getLookaheadDFAColumnsForLineInFile(int line)
returns a list of column numbers for all decisions on a particular line so ANTLRWorks choose the decision depending on the location of the cursor (otherwise, ANTLRWorks has to give the *exact* location which is not easy from the user point of view).DFA
getLookaheadDFAFromPositionInFile(int line, int col)
Useful for ANTLRWorks to map position in file to the DFA for displayint
getMaxCharValue()
What is the max char value possible for this grammar's target? Use unicode max if no target defined.int
getMaxTokenType()
How many token types have been allocated so far?int
getNewTokenType()
Return a new unique integer in the token type spaceNFAState
getNFAStateForAltOfDecision(NFAState decisionState, int alt)
Get the ith alternative (1..n) from a decision; return null when an invalid alt is requested.int
getNumberOfAltsForDecisionNFA(NFAState decisionState)
Decisions are linked together with transition(1).int
getNumberOfCyclicDecisions()
int
getNumberOfDecisions()
java.lang.Object
getOption(java.lang.String key)
java.lang.String
getRecognizerName()
Get the name of the generated recognizer; may or may not be same as grammar name.Rule
getRule(java.lang.String ruleName)
Rule
getRule(java.lang.String scopeName, java.lang.String ruleName)
int
getRuleIndex(java.lang.String ruleName)
int
getRuleIndex(java.lang.String scopeName, java.lang.String ruleName)
java.lang.String
getRuleModifier(java.lang.String ruleName)
java.lang.String
getRuleName(int ruleIndex)
java.util.Collection<Rule>
getRules()
NFAState
getRuleStartState(java.lang.String ruleName)
NFAState
getRuleStartState(java.lang.String scopeName, java.lang.String ruleName)
NFAState
getRuleStopState(java.lang.String ruleName)
IntSet
getSetFromRule(TreeToNFAConverter nfabuilder, java.lang.String ruleName)
Get the set equivalent (if any) of the indicated rule from this grammar.java.util.Set<java.lang.String>
getStringLiterals()
Get the list of ANTLR String literalsGrammarAST
getSyntacticPredicate(java.lang.String name)
java.util.LinkedHashMap<java.lang.String,GrammarAST>
getSyntacticPredicates()
java.lang.String
getTokenDisplayName(int ttype)
Given a token type, get a meaningful name for it such as the ID or string literal.java.util.Set<java.lang.String>
getTokenDisplayNames()
Get a list of all token IDs and literals that have an associated token type.java.util.Set<java.lang.String>
getTokenIDs()
Get the list of tokens that are IDs like BLOCK and LPARENint
getTokenType(java.lang.String tokenName)
IntSet
getTokenTypes()
Return a set of all possible token or char types for this grammarjava.util.Collection<java.lang.Integer>
getTokenTypesWithoutID()
Return an ordered integer list of token types that have no corresponding token ID like INT or KEYWORD_BEGIN; for stuff like 'begin'.Tool
getTool()
static java.lang.StringBuffer
getUnescapedStringFromGrammarStringLiteral(java.lang.String literal)
ANTLR does not convert escape sequences during the parse phase because it could not know how to print String/char literals back out when printing grammars etc...int
getUserMaxLookahead(int decision)
java.lang.String
grammarTreeToString(GrammarAST t)
java.lang.String
grammarTreeToString(GrammarAST t, boolean showActions)
void
importGrammar(GrammarAST grammarNameAST, java.lang.String label)
Import the rules/tokens of a delegate grammar.int
importTokenVocabulary(Grammar importFromGr)
Pull your token definitions from an existing grammar in memory.int
importTokenVocabulary(GrammarAST tokenVocabOptionAST, java.lang.String vocabName)
Load a vocab file <vocabName>.tokens and return max token type found.boolean
isAtomTokenType(int ttype)
boolean
isBuiltFromString()
boolean
isEmptyRule(GrammarAST block)
Rules like "a : ;" and "a : {...} ;" should not generate try/catch blocks for RecognitionException.boolean
isValidSet(TreeToNFAConverter nfabuilder, GrammarAST t)
Given set tree like ( SET A B ), check that A and B are both valid sets themselves, else we must tree like a BLOCKboolean
legalOption(java.lang.String key)
LookaheadSet
LOOK(NFAState s)
protected java.util.List<IntervalSet>
makeEdgeSetsDisjoint(java.util.List<IntervalSet> edges)
boolean
NFAToDFAConversionExternallyAborted()
boolean
optionIsValid(java.lang.String key, java.lang.Object value)
void
parseAndBuildAST()
void
parseAndBuildAST(java.io.Reader r)
GrammarAST
parseArtificialRule(java.lang.String ruleText)
void
printGrammar(java.io.PrintStream output)
void
referenceRuleLabelPredefinedAttribute(java.lang.String ruleName)
To yield smaller, more readable code, track which rules have their predefined attributes accessed.protected void
removeUselessLabels(java.util.Map<java.lang.String,Grammar.LabelElementPair> ruleToElementLabelPairMap)
A label on a rule is useless if the rule has no return value, no tree or template output, and it is not referenced in an action.boolean
rewriteMode()
void
setCodeGenerator(CodeGenerator generator)
void
setDecisionBlockAST(int decision, GrammarAST blockAST)
void
setDecisionNFA(int decision, NFAState state)
void
setFileName(java.lang.String fileName)
void
setGrammarContent(java.lang.String grammarString)
void
setGrammarTree(GrammarAST value)
void
setLookaheadDFA(int decision, DFA lookaheadDFA)
Set the lookahead DFA for a particular decision.void
setName(java.lang.String name)
java.lang.String
setOption(java.lang.String key, java.lang.Object value, org.antlr.runtime.Token optionsStartToken)
Save the option key/value pair and process it; return the key or null if invalid option.void
setOptions(java.util.Map<java.lang.String,java.lang.Object> options, org.antlr.runtime.Token optionsStartToken)
void
setRuleAST(java.lang.String ruleName, GrammarAST t)
void
setSynPredGateIfNotAlready(org.stringtemplate.v4.ST gateST)
void
setTool(Tool tool)
void
synPredUsedInDFA(DFA dfa, SemanticContext semCtx)
java.lang.String
toString()
void
translateLeftRecursiveRule(GrammarAST ruleAST)
boolean
validImport(Grammar delegate)
Many imports are illegal such as lexer into a tree grammar
-
-
-
Field Detail
-
SYNPRED_RULE_PREFIX
public static final java.lang.String SYNPRED_RULE_PREFIX
- See Also:
- Constant Field Values
-
GRAMMAR_FILE_EXTENSION
public static final java.lang.String GRAMMAR_FILE_EXTENSION
- See Also:
- Constant Field Values
-
LEXER_GRAMMAR_FILE_EXTENSION
public static final java.lang.String LEXER_GRAMMAR_FILE_EXTENSION
used for generating lexer temp files- See Also:
- Constant Field Values
-
INITIAL_DECISION_LIST_SIZE
public static final int INITIAL_DECISION_LIST_SIZE
- See Also:
- Constant Field Values
-
INVALID_RULE_INDEX
public static final int INVALID_RULE_INDEX
- See Also:
- Constant Field Values
-
RULE_LABEL
public static final int RULE_LABEL
- See Also:
- Constant Field Values
-
TOKEN_LABEL
public static final int TOKEN_LABEL
- See Also:
- Constant Field Values
-
RULE_LIST_LABEL
public static final int RULE_LIST_LABEL
- See Also:
- Constant Field Values
-
TOKEN_LIST_LABEL
public static final int TOKEN_LIST_LABEL
- See Also:
- Constant Field Values
-
CHAR_LABEL
public static final int CHAR_LABEL
- See Also:
- Constant Field Values
-
WILDCARD_TREE_LABEL
public static final int WILDCARD_TREE_LABEL
- See Also:
- Constant Field Values
-
WILDCARD_TREE_LIST_LABEL
public static final int WILDCARD_TREE_LIST_LABEL
- See Also:
- Constant Field Values
-
LabelTypeToString
public static java.lang.String[] LabelTypeToString
-
ARTIFICIAL_TOKENS_RULENAME
public static final java.lang.String ARTIFICIAL_TOKENS_RULENAME
- See Also:
- Constant Field Values
-
FRAGMENT_RULE_MODIFIER
public static final java.lang.String FRAGMENT_RULE_MODIFIER
- See Also:
- Constant Field Values
-
SYNPREDGATE_ACTION_NAME
public static final java.lang.String SYNPREDGATE_ACTION_NAME
- See Also:
- Constant Field Values
-
ANTLRLiteralEscapedCharValue
public static int[] ANTLRLiteralEscapedCharValue
When converting ANTLR char and string literals, here is the value set of escape chars.
-
ANTLRLiteralCharValueEscape
public static java.lang.String[] ANTLRLiteralCharValueEscape
Given a char, we need to be able to show as an ANTLR literal.
-
LEXER
public static final int LEXER
- See Also:
- Constant Field Values
-
PARSER
public static final int PARSER
- See Also:
- Constant Field Values
-
TREE_PARSER
public static final int TREE_PARSER
- See Also:
- Constant Field Values
-
COMBINED
public static final int COMBINED
- See Also:
- Constant Field Values
-
grammarTypeToString
public static final java.lang.String[] grammarTypeToString
-
grammarTypeToFileNameSuffix
public static final java.lang.String[] grammarTypeToFileNameSuffix
-
validDelegations
public static MultiMap<java.lang.Integer,java.lang.Integer> validDelegations
Set of valid imports. E.g., can only import a tree parser into another tree parser. Maps delegate to set of delegator grammar types. validDelegations.get(LEXER) gives list of the kinds of delegators that can import lexers.
-
tokenBuffer
public org.antlr.runtime.CommonTokenStream tokenBuffer
This is the buffer of *all* tokens found in the grammar file including whitespace tokens etc... I use this to extract lexer rules from combined grammars.
-
IGNORE_STRING_IN_GRAMMAR_FILE_NAME
public static final java.lang.String IGNORE_STRING_IN_GRAMMAR_FILE_NAME
- See Also:
- Constant Field Values
-
AUTO_GENERATED_TOKEN_NAME_PREFIX
public static final java.lang.String AUTO_GENERATED_TOKEN_NAME_PREFIX
- See Also:
- Constant Field Values
-
name
public java.lang.String name
What name did the user provide for this grammar?
-
type
public int type
What type of grammar is this: lexer, parser, tree walker
-
options
protected java.util.Map<java.lang.String,java.lang.Object> options
A list of options specified at the grammar level such as language=Java. The value can be an AST for complicated values such as character sets. There may be code generator specific options in here. I do no interpretation of the key/value pairs...they are simply available for who wants them.
-
legalLexerOptions
public static final java.util.Set<java.lang.String> legalLexerOptions
-
legalParserOptions
public static final java.util.Set<java.lang.String> legalParserOptions
-
legalTreeParserOptions
public static final java.util.Set<java.lang.String> legalTreeParserOptions
-
doNotCopyOptionsToLexer
public static final java.util.Set<java.lang.String> doNotCopyOptionsToLexer
-
defaultOptions
public static final java.util.Map<java.lang.String,java.lang.String> defaultOptions
-
legalBlockOptions
public static final java.util.Set<java.lang.String> legalBlockOptions
-
defaultBlockOptions
public static final java.util.Map<java.lang.String,java.lang.String> defaultBlockOptions
What are the default options for a subrule?
-
defaultLexerBlockOptions
public static final java.util.Map<java.lang.String,java.lang.String> defaultLexerBlockOptions
-
legalTokenOptions
public static final java.util.Set<java.lang.String> legalTokenOptions
Legal options for terminal refs like ID<node=MyVarNode>
-
defaultTokenOption
public static final java.lang.String defaultTokenOption
- See Also:
- Constant Field Values
-
global_k
protected int global_k
Is there a global fixed lookahead set for this grammar? If 0, nothing specified. -1 implies we have not looked at the options table yet to set k.
-
nfa
public NFA nfa
The NFA that represents the grammar with edges labelled with tokens or epsilon. It is more suitable to analysis than an AST representation.
-
factory
protected NFAFactory factory
-
composite
public CompositeGrammar composite
If this grammar is part of a larger composite grammar via delegate statement, then this points at the composite. The composite holds a global list of rules, token types, decision numbers, etc...
-
compositeTreeNode
public CompositeGrammarTree compositeTreeNode
A pointer back into grammar tree. Needed so we can add delegates.
-
label
public java.lang.String label
If this is a delegate of another grammar, this is the label used as an instance var by that grammar to point at this grammar. null if no label was specified in the delegate statement.
-
charVocabulary
protected IntSet charVocabulary
TODO: hook this to the charVocabulary option
-
tool
public Tool tool
-
ruleRefs
protected java.util.Set<GrammarAST> ruleRefs
The unique set of all rule references in any rule; set of tree node objects so two refs to same rule can exist but at different line/position.
-
scopedRuleRefs
protected java.util.Set<GrammarAST> scopedRuleRefs
-
tokenIDRefs
protected java.util.Set<org.antlr.runtime.Token> tokenIDRefs
The unique set of all token ID references in any rule
-
decisionCount
protected int decisionCount
Be able to assign a number to every decision in grammar; decisions in 1..n
-
leftRecursiveRules
protected java.util.Set<Rule> leftRecursiveRules
A list of all rules that are in any left-recursive cycle. There could be multiple cycles, but this is a flat list of all problematic rules. This is stuff we couldn't refactor to precedence rule.
-
externalAnalysisAbort
protected boolean externalAnalysisAbort
An external tool requests that DFA analysis abort prematurely. Stops at DFA granularity, which are limited to a DFA size and time computation as failsafe.
-
numNonLLStar
public int numNonLLStar
-
nameToSynpredASTMap
protected java.util.LinkedHashMap<java.lang.String,GrammarAST> nameToSynpredASTMap
When we read in a grammar, we track the list of syntactic predicates and build faux rules for them later. See my blog entry Dec 2, 2005: http://www.antlr.org/blog/antlr3/lookahead.tml This maps the name (we make up) for a pred to the AST grammar fragment.
-
precRuleInitCodeBlocks
public java.util.List<java.lang.String> precRuleInitCodeBlocks
Each left-recursive precedence rule must define precedence array for binary operators like: static int[] e_prec = new int[tokenNames.length]; static { e_prec[75] = 1; } Track and we push into parser later; this is computed early when we look for prec rules.
-
atLeastOneRuleMemoizes
public boolean atLeastOneRuleMemoizes
At least one rule has memoize=true
-
atLeastOneBacktrackOption
public boolean atLeastOneBacktrackOption
At least one backtrack=true in rule or decision or grammar.
-
implicitLexer
public boolean implicitLexer
Was this created from a COMBINED grammar?
-
nameToRuleMap
protected java.util.LinkedHashMap<java.lang.String,Rule> nameToRuleMap
Map a rule to it's Rule object
-
overriddenRules
public java.util.Set<java.lang.String> overriddenRules
If this rule is a delegate, some rules might be overridden; don't want to gen code for them.
-
delegatedRuleReferences
protected java.util.Set<Rule> delegatedRuleReferences
The list of all rules referenced in this grammar, not defined here, and defined in a delegate grammar. Not all of these will be generated in the recognizer for this file; only those that are affected by rule definitions in this grammar. I am not sure the Java target will need this but I'm leaving in case other targets need it. see NameSpaceChecker.lookForReferencesToUndefinedSymbols()
-
lexerRuleNamesInCombined
public java.util.List<java.lang.String> lexerRuleNamesInCombined
The ANTLRParser tracks lexer rules when reading combined grammars so we can build the Tokens rule.
-
scopes
protected java.util.Map<java.lang.String,AttributeScope> scopes
Track the scopes defined outside of rules and the scopes associated with all rules (even if empty).
-
grammarTree
protected GrammarAST grammarTree
An AST that records entire input grammar with all rules. A simple grammar with one rule, "grammar t; a : A | B ;", looks like: ( grammar t ( rule a ( BLOCK ( ALT A ) ( ALT B ) ) <end-of-rule> ) )
-
indexToDecision
protected java.util.Vector<Grammar.Decision> indexToDecision
Each subrule/rule is a decision point and we must track them so we can go back later and build DFA predictors for them. This includes all the rules, subrules, optional blocks, ()+, ()* etc...
-
generator
protected CodeGenerator generator
If non-null, this is the code generator we will use to generate recognizers in the target language.
-
nameSpaceChecker
public NameSpaceChecker nameSpaceChecker
-
ll1Analyzer
public LL1Analyzer ll1Analyzer
-
lexerGrammarTemplate
protected java.lang.String lexerGrammarTemplate
For merged lexer/parsers, we must construct a separate lexer spec. This is the template for lexer; put the literals first then the regular rules. We don't need to specify a token vocab import as I make the new grammar import from the old all in memory; don't want to force it to read from the disk. Lexer grammar will have same name as original grammar but will be in different filename. Foo.g with combined grammar will have FooParser.java generated and Foo__.g with again Foo inside. It will however generate FooLexer.java as it's a lexer grammar. A bit odd, but autogenerated. Can tweak later if we want.
-
lexerGrammarST
protected org.stringtemplate.v4.ST lexerGrammarST
-
fileName
protected java.lang.String fileName
What file name holds this grammar?
-
DFACreationWallClockTimeInMS
public long DFACreationWallClockTimeInMS
How long in ms did it take to build DFAs for this grammar? If this grammar is a combined grammar, it only records time for the parser grammar component. This only records the time to do the LL(*) work; NFA→DFA conversion.
-
numberOfSemanticPredicates
public int numberOfSemanticPredicates
-
numberOfManualLookaheadOptions
public int numberOfManualLookaheadOptions
-
setOfNondeterministicDecisionNumbers
public java.util.Set<java.lang.Integer> setOfNondeterministicDecisionNumbers
-
setOfNondeterministicDecisionNumbersResolvedWithPredicates
public java.util.Set<java.lang.Integer> setOfNondeterministicDecisionNumbersResolvedWithPredicates
-
blocksWithSynPreds
public java.util.Set<GrammarAST> blocksWithSynPreds
Track decisions with syn preds specified for reporting. This is the a set of BLOCK type AST nodes.
-
decisionsWhoseDFAsUsesSynPreds
public java.util.Set<DFA> decisionsWhoseDFAsUsesSynPreds
Track decisions that actually use the syn preds in the DFA. Computed during NFA to DFA conversion.
-
synPredNamesUsedInDFA
public java.util.Set<java.lang.String> synPredNamesUsedInDFA
Track names of preds so we can avoid generating preds that aren't used Computed during NFA to DFA conversion. Just walk accept states and look for synpreds because that is the only state target whose incident edges can have synpreds. Same is try for decisionsWhoseDFAsUsesSynPreds.
-
blocksWithSemPreds
public java.util.Set<GrammarAST> blocksWithSemPreds
Track decisions with syn preds specified for reporting. This is the a set of BLOCK type AST nodes.
-
decisionsWhoseDFAsUsesSemPreds
public java.util.Set<DFA> decisionsWhoseDFAsUsesSemPreds
Track decisions that actually use the syn preds in the DFA.
-
allDecisionDFACreated
protected boolean allDecisionDFACreated
-
builtFromString
protected boolean builtFromString
We need a way to detect when a lexer grammar is autogenerated from another grammar or we are just sending in a string representing a grammar. We don't want to generate a .tokens file, for example, in such cases.
-
-
Constructor Detail
-
Grammar
public Grammar(Tool tool, java.lang.String fileName, CompositeGrammar composite)
Create a grammar from file name.
-
Grammar
public Grammar()
Useful for when you are sure that you are not part of a composite already. Used in Interp/RandomPhrase and testing.
-
Grammar
public Grammar(Tool tool)
-
Grammar
public Grammar(java.lang.String grammarString) throws org.antlr.runtime.RecognitionException
Used for testing; only useful on noncomposite grammars.- Throws:
org.antlr.runtime.RecognitionException
-
Grammar
public Grammar(Tool tool, java.lang.String grammarString) throws org.antlr.runtime.RecognitionException
Used for testing and Interp/RandomPhrase. Only useful on noncomposite grammars.- Throws:
org.antlr.runtime.RecognitionException
-
-
Method Detail
-
setFileName
public void setFileName(java.lang.String fileName)
-
getFileName
public java.lang.String getFileName()
-
setName
public void setName(java.lang.String name)
-
setGrammarContent
public void setGrammarContent(java.lang.String grammarString) throws org.antlr.runtime.RecognitionException
- Throws:
org.antlr.runtime.RecognitionException
-
parseAndBuildAST
public void parseAndBuildAST() throws java.io.IOException
- Throws:
java.io.IOException
-
parseAndBuildAST
public void parseAndBuildAST(java.io.Reader r)
-
dealWithTreeFilterMode
protected void dealWithTreeFilterMode()
-
translateLeftRecursiveRule
public void translateLeftRecursiveRule(GrammarAST ruleAST)
-
defineGrammarSymbols
public void defineGrammarSymbols()
-
checkNameSpaceAndActions
public void checkNameSpaceAndActions()
ANALYZE ACTIONS, LOOKING FOR LABEL AND ATTR REFS, sanity check
-
validImport
public boolean validImport(Grammar delegate)
Many imports are illegal such as lexer into a tree grammar
-
getLexerGrammar
public java.lang.String getLexerGrammar()
If the grammar is a combined grammar, return the text of the implicit lexer grammar.
-
getImplicitlyGeneratedLexerFileName
public java.lang.String getImplicitlyGeneratedLexerFileName()
-
getRecognizerName
public java.lang.String getRecognizerName()
Get the name of the generated recognizer; may or may not be same as grammar name. Recognizer is TParser and TLexer from T if combined, else just use T regardless of grammar type.
-
addArtificialMatchTokensRule
public GrammarAST addArtificialMatchTokensRule(GrammarAST grammarAST, java.util.List<java.lang.String> ruleNames, java.util.List<java.lang.String> delegateNames, boolean filterMode)
Parse a rule we add artificially that is a list of the other lexer rules like this: "Tokens : ID | INT | SEMI ;" nextToken() will invoke this to set the current token. Add char literals before the rule references. If in filter mode, we want every alt to backtrack and we need to do k=1 to force the "first token def wins" rule. Otherwise, the longest-match rule comes into play with LL(*). The ANTLRParser antlr.g file now invokes this when parsing a lexer grammar, which I think is proper even though it peeks at the info that later phases will (re)compute. It gets a list of lexer rules and builds a string representing the rule; then it creates a parser and adds the resulting tree to the grammar's tree.
-
parseArtificialRule
public GrammarAST parseArtificialRule(java.lang.String ruleText)
-
addRule
public void addRule(GrammarAST grammarTree, GrammarAST t)
-
getArtificialRulesForSyntacticPredicates
protected java.util.List<? extends GrammarAST> getArtificialRulesForSyntacticPredicates(java.util.LinkedHashMap<java.lang.String,GrammarAST> nameToSynpredASTMap)
for any syntactic predicates, we need to define rules for them; they will get defined automatically like any other rule. :)
-
addRulesForSyntacticPredicates
public void addRulesForSyntacticPredicates()
-
createRuleStartAndStopNFAStates
public void createRuleStartAndStopNFAStates()
Define all the rule begin/end NFAStates to solve forward reference issues. Critical for composite grammars too. This is normally called on all root/delegates manually and then buildNFA() is called afterwards because the NFA construction needs to see rule start/stop states from potentially every grammar. Has to be have these created a priori. Testing routines will often just call buildNFA(), which forces a call to this method if not done already. Works ONLY for single noncomposite grammars.
-
buildNFA
public void buildNFA()
-
createLookaheadDFAs
public void createLookaheadDFAs()
For each decision in this grammar, compute a single DFA using the NFA states associated with the decision. The DFA construction determines whether or not the alternatives in the decision are separable using a regular lookahead language. Store the lookahead DFAs in the AST created from the user's grammar so the code generator or whoever can easily access it. This is a separate method because you might want to create a Grammar without doing the expensive analysis.
-
createLookaheadDFAs
public void createLookaheadDFAs(boolean wackTempStructures)
-
createLL_1_LookaheadDFA
public DFA createLL_1_LookaheadDFA(int decision)
-
makeEdgeSetsDisjoint
protected java.util.List<IntervalSet> makeEdgeSetsDisjoint(java.util.List<IntervalSet> edges)
-
createLookaheadDFA
public DFA createLookaheadDFA(int decision, boolean wackTempStructures)
-
externallyAbortNFAToDFAConversion
public void externallyAbortNFAToDFAConversion()
Terminate DFA creation (grammar analysis).
-
NFAToDFAConversionExternallyAborted
public boolean NFAToDFAConversionExternallyAborted()
-
getNewTokenType
public int getNewTokenType()
Return a new unique integer in the token type space
-
defineToken
public void defineToken(java.lang.String text, int tokenType)
Define a token at a particular token type value. Blast an old value with a new one. This is called normal grammar processsing and during import vocab operations to set tokens with specific values.
-
defineRule
public void defineRule(org.antlr.runtime.Token ruleToken, java.lang.String modifier, java.util.Map<java.lang.String,java.lang.Object> options, GrammarAST tree, GrammarAST argActionAST, int numAlts)
Define a new rule. A new rule index is created by incrementing ruleIndex.
-
defineSyntacticPredicate
public java.lang.String defineSyntacticPredicate(GrammarAST blockAST, java.lang.String currentRuleName)
Define a new predicate and get back its name for use in building a semantic predicate reference to the syn pred.
-
getSyntacticPredicates
public java.util.LinkedHashMap<java.lang.String,GrammarAST> getSyntacticPredicates()
-
getSyntacticPredicate
public GrammarAST getSyntacticPredicate(java.lang.String name)
-
synPredUsedInDFA
public void synPredUsedInDFA(DFA dfa, SemanticContext semCtx)
-
defineNamedAction
public void defineNamedAction(GrammarAST ampersandAST, java.lang.String scope, GrammarAST nameAST, GrammarAST actionAST)
Given @scope::name {action} define it for this grammar. Later, the code generator will ask for the actions table. For composite grammars, make sure header action propogates down to all delegates.
-
setSynPredGateIfNotAlready
public void setSynPredGateIfNotAlready(org.stringtemplate.v4.ST gateST)
-
getActions
public java.util.Map<java.lang.String,java.util.Map<java.lang.String,java.lang.Object>> getActions()
-
getDefaultActionScope
public java.lang.String getDefaultActionScope(int grammarType)
Given a grammar type, what should be the default action scope? If I say @members in a COMBINED grammar, for example, the default scope should be "parser".
-
defineLexerRuleFoundInParser
public void defineLexerRuleFoundInParser(org.antlr.runtime.Token ruleToken, GrammarAST ruleAST)
-
defineLexerRuleForAliasedStringLiteral
public void defineLexerRuleForAliasedStringLiteral(java.lang.String tokenID, java.lang.String literal, int tokenType)
If someone does PLUS='+' in the parser, must make sure we get "PLUS : '+' ;" in lexer not "T73 : '+';"
-
defineLexerRuleForStringLiteral
public void defineLexerRuleForStringLiteral(java.lang.String literal, int tokenType)
-
getLocallyDefinedRule
public Rule getLocallyDefinedRule(java.lang.String ruleName)
-
getRule
public Rule getRule(java.lang.String ruleName)
-
getRule
public Rule getRule(java.lang.String scopeName, java.lang.String ruleName)
-
getRuleIndex
public int getRuleIndex(java.lang.String scopeName, java.lang.String ruleName)
-
getRuleIndex
public int getRuleIndex(java.lang.String ruleName)
-
getRuleName
public java.lang.String getRuleName(int ruleIndex)
-
generateMethodForRule
public boolean generateMethodForRule(java.lang.String ruleName)
Should codegen.g gen rule for ruleName? If synpred, only gen if used in a DFA. If regular rule, only gen if not overridden in delegator Always gen Tokens rule though.
-
defineGlobalScope
public AttributeScope defineGlobalScope(java.lang.String name, org.antlr.runtime.Token scopeAction)
-
createReturnScope
public AttributeScope createReturnScope(java.lang.String ruleName, org.antlr.runtime.Token retAction)
-
createRuleScope
public AttributeScope createRuleScope(java.lang.String ruleName, org.antlr.runtime.Token scopeAction)
-
createParameterScope
public AttributeScope createParameterScope(java.lang.String ruleName, org.antlr.runtime.Token argAction)
-
getGlobalScope
public AttributeScope getGlobalScope(java.lang.String name)
Get a global scope
-
getGlobalScopes
public java.util.Map<java.lang.String,AttributeScope> getGlobalScopes()
-
defineLabel
protected void defineLabel(Rule r, org.antlr.runtime.Token label, GrammarAST element, int type)
Define a label defined in a rule r; check the validity then ask the Rule object to actually define it.
-
defineTokenRefLabel
public void defineTokenRefLabel(java.lang.String ruleName, org.antlr.runtime.Token label, GrammarAST tokenRef)
-
defineWildcardTreeLabel
public void defineWildcardTreeLabel(java.lang.String ruleName, org.antlr.runtime.Token label, GrammarAST tokenRef)
-
defineWildcardTreeListLabel
public void defineWildcardTreeListLabel(java.lang.String ruleName, org.antlr.runtime.Token label, GrammarAST tokenRef)
-
defineRuleRefLabel
public void defineRuleRefLabel(java.lang.String ruleName, org.antlr.runtime.Token label, GrammarAST ruleRef)
-
defineTokenListLabel
public void defineTokenListLabel(java.lang.String ruleName, org.antlr.runtime.Token label, GrammarAST element)
-
defineRuleListLabel
public void defineRuleListLabel(java.lang.String ruleName, org.antlr.runtime.Token label, GrammarAST element)
-
getLabels
public java.util.Set<java.lang.String> getLabels(java.util.Set<GrammarAST> rewriteElements, int labelType)
Given a set of all rewrite elements on right of ->, filter for label types such as Grammar.TOKEN_LABEL, Grammar.TOKEN_LIST_LABEL, ... Return a displayable token type name computed from the GrammarAST.
-
examineAllExecutableActions
protected void examineAllExecutableActions()
Before generating code, we examine all actions that can have $x.y and $y stuff in them because some code generation depends on Rule.referencedPredefinedRuleAttributes. I need to remove unused rule labels for example.
-
checkAllRulesForUselessLabels
public void checkAllRulesForUselessLabels()
Remove all labels on rule refs whose target rules have no return value. Do this for all rules in grammar.
-
removeUselessLabels
protected void removeUselessLabels(java.util.Map<java.lang.String,Grammar.LabelElementPair> ruleToElementLabelPairMap)
A label on a rule is useless if the rule has no return value, no tree or template output, and it is not referenced in an action.
-
altReferencesRule
public void altReferencesRule(java.lang.String enclosingRuleName, GrammarAST refScopeAST, GrammarAST refAST, int outerAltNum)
Track a rule reference within an outermost alt of a rule. Used at the moment to decide if $ruleref refers to a unique rule ref in the alt. Rewrite rules force tracking of all rule AST results. This data is also used to verify that all rules have been defined.
-
altReferencesTokenID
public void altReferencesTokenID(java.lang.String ruleName, GrammarAST refAST, int outerAltNum)
Track a token reference within an outermost alt of a rule. Used to decide if $tokenref refers to a unique token ref in the alt. Does not track literals! Rewrite rules force tracking of all tokens.
-
referenceRuleLabelPredefinedAttribute
public void referenceRuleLabelPredefinedAttribute(java.lang.String ruleName)
To yield smaller, more readable code, track which rules have their predefined attributes accessed. If the rule has no user-defined return values, then don't generate the return value scope classes etc... Make the rule have void return value. Don't track for lexer rules.
-
checkAllRulesForLeftRecursion
public java.util.List<? extends java.util.Collection<? extends Rule>> checkAllRulesForLeftRecursion()
-
getLeftRecursiveRules
public java.util.Set<Rule> getLeftRecursiveRules()
Return a list of left-recursive rules; no analysis can be done successfully on these. Useful to skip these rules then and also for ANTLRWorks to highlight them.
-
checkRuleReference
public void checkRuleReference(GrammarAST scopeAST, GrammarAST refAST, GrammarAST argsAST, java.lang.String currentRuleName)
-
isEmptyRule
public boolean isEmptyRule(GrammarAST block)
Rules like "a : ;" and "a : {...} ;" should not generate try/catch blocks for RecognitionException. To detect this it's probably ok to just look for any reference to an atom that can match some input. W/o that, the rule is unlikey to have any else.
-
findFirstTypeOutsideRewrite
protected GrammarAST findFirstTypeOutsideRewrite(GrammarAST block, java.util.BitSet types)
-
isAtomTokenType
public boolean isAtomTokenType(int ttype)
-
getTokenType
public int getTokenType(java.lang.String tokenName)
-
getTokenIDs
public java.util.Set<java.lang.String> getTokenIDs()
Get the list of tokens that are IDs like BLOCK and LPAREN
-
getTokenTypesWithoutID
public java.util.Collection<java.lang.Integer> getTokenTypesWithoutID()
Return an ordered integer list of token types that have no corresponding token ID like INT or KEYWORD_BEGIN; for stuff like 'begin'.
-
getTokenDisplayNames
public java.util.Set<java.lang.String> getTokenDisplayNames()
Get a list of all token IDs and literals that have an associated token type.
-
getCharValueFromGrammarCharLiteral
public static int getCharValueFromGrammarCharLiteral(java.lang.String literal)
Given a literal like (the 3 char sequence with single quotes) 'a', return the int value of 'a'. Convert escape sequences here also. ANTLR's antlr.g parser does not convert escape sequences. 11/26/2005: I changed literals to always be '...' even for strings. This routine still works though.
-
getUnescapedStringFromGrammarStringLiteral
public static java.lang.StringBuffer getUnescapedStringFromGrammarStringLiteral(java.lang.String literal)
ANTLR does not convert escape sequences during the parse phase because it could not know how to print String/char literals back out when printing grammars etc... Someone in China might use the real unicode char in a literal as it will display on their screen; when printing back out, I could not know whether to display or use a unicode escape. This routine converts a string literal with possible escape sequences into a pure string of 16-bit char values. Escapes and unicode specs are converted to pure chars. return in a buffer; people may want to walk/manipulate further. The NFA construction routine must know the actual char values.
-
importTokenVocabulary
public int importTokenVocabulary(Grammar importFromGr)
Pull your token definitions from an existing grammar in memory. You must use Grammar() ctor then this method then setGrammarContent() to make this work. This was useful primarily for testing and interpreting grammars until I added import grammar functionality. When you import a grammar you implicitly import its vocabulary as well and keep the same token type values. Returns the max token type found.
-
importGrammar
public void importGrammar(GrammarAST grammarNameAST, java.lang.String label)
Import the rules/tokens of a delegate grammar. All delegate grammars are read during the ctor of first Grammar created. Do not create NFA here because NFA construction needs to hook up with overridden rules in delegation root grammar.
-
addDelegateGrammar
protected void addDelegateGrammar(Grammar delegateGrammar)
add new delegate to composite tree
-
importTokenVocabulary
public int importTokenVocabulary(GrammarAST tokenVocabOptionAST, java.lang.String vocabName)
Load a vocab file <vocabName>.tokens and return max token type found.
-
getTokenDisplayName
public java.lang.String getTokenDisplayName(int ttype)
Given a token type, get a meaningful name for it such as the ID or string literal. If this is a lexer and the ttype is in the char vocabulary, compute an ANTLR-valid (possibly escaped) char literal.
-
getStringLiterals
public java.util.Set<java.lang.String> getStringLiterals()
Get the list of ANTLR String literals
-
getGrammarTypeString
public java.lang.String getGrammarTypeString()
-
getGrammarMaxLookahead
public int getGrammarMaxLookahead()
-
setOption
public java.lang.String setOption(java.lang.String key, java.lang.Object value, org.antlr.runtime.Token optionsStartToken)
Save the option key/value pair and process it; return the key or null if invalid option.
-
legalOption
public boolean legalOption(java.lang.String key)
-
setOptions
public void setOptions(java.util.Map<java.lang.String,java.lang.Object> options, org.antlr.runtime.Token optionsStartToken)
-
getOption
public java.lang.Object getOption(java.lang.String key)
-
getLocallyDefinedOption
public java.lang.Object getLocallyDefinedOption(java.lang.String key)
-
getBlockOption
public java.lang.Object getBlockOption(GrammarAST blockAST, java.lang.String key)
-
getUserMaxLookahead
public int getUserMaxLookahead(int decision)
-
getAutoBacktrackMode
public boolean getAutoBacktrackMode(int decision)
-
optionIsValid
public boolean optionIsValid(java.lang.String key, java.lang.Object value)
-
buildAST
public boolean buildAST()
-
rewriteMode
public boolean rewriteMode()
-
isBuiltFromString
public boolean isBuiltFromString()
-
buildTemplate
public boolean buildTemplate()
-
getRules
public java.util.Collection<Rule> getRules()
-
getDelegatedRules
public java.util.Set<? extends Rule> getDelegatedRules()
Get the set of Rules that need to have manual delegations like "void rule() { importedGrammar.rule(); }" If this grammar is master, get list of all rule definitions from all delegate grammars. Only master has complete interface from combined grammars...we will generated delegates as helper objects. Composite grammars that are not the root/master do not have complete interfaces. It is not my intention that people use subcomposites. Only the outermost grammar should be used from outside code. The other grammar components are specifically generated to work only with the master/root. delegatedRules = imported - overridden
-
getAllImportedRules
public java.util.Set<? extends Rule> getAllImportedRules()
Get set of all rules imported from all delegate grammars even if indirectly delegated.
-
getDelegates
public java.util.List<Grammar> getDelegates()
Get list of all delegates from all grammars directly or indirectly imported into this grammar.
-
getHasDelegates
public boolean getHasDelegates()
-
getDelegateNames
public java.util.List<java.lang.String> getDelegateNames()
-
getDirectDelegates
public java.util.List<Grammar> getDirectDelegates()
-
getIndirectDelegates
public java.util.List<Grammar> getIndirectDelegates()
Get delegates below direct delegates
-
getDelegators
public java.util.List<Grammar> getDelegators()
Get list of all delegators. This amounts to the grammars on the path to the root of the delegation tree.
-
getDelegator
public Grammar getDelegator()
Who's my direct parent grammar?
-
getDelegatedRuleReferences
public java.util.Set<Rule> getDelegatedRuleReferences()
-
getGrammarIsRoot
public boolean getGrammarIsRoot()
-
setRuleAST
public void setRuleAST(java.lang.String ruleName, GrammarAST t)
-
getRuleStartState
public NFAState getRuleStartState(java.lang.String ruleName)
-
getRuleStartState
public NFAState getRuleStartState(java.lang.String scopeName, java.lang.String ruleName)
-
getRuleModifier
public java.lang.String getRuleModifier(java.lang.String ruleName)
-
getRuleStopState
public NFAState getRuleStopState(java.lang.String ruleName)
-
assignDecisionNumber
public int assignDecisionNumber(NFAState state)
-
getDecision
protected Grammar.Decision getDecision(int decision)
-
getDecisions
public java.util.List<Grammar.Decision> getDecisions()
-
createDecision
protected Grammar.Decision createDecision(int decision)
-
getDecisionNFAStartStateList
public java.util.List<NFAState> getDecisionNFAStartStateList()
-
getDecisionNFAStartState
public NFAState getDecisionNFAStartState(int decision)
-
getLookaheadDFA
public DFA getLookaheadDFA(int decision)
-
getDecisionBlockAST
public GrammarAST getDecisionBlockAST(int decision)
-
getLookaheadDFAColumnsForLineInFile
public java.util.List<java.lang.Integer> getLookaheadDFAColumnsForLineInFile(int line)
returns a list of column numbers for all decisions on a particular line so ANTLRWorks choose the decision depending on the location of the cursor (otherwise, ANTLRWorks has to give the *exact* location which is not easy from the user point of view). This is not particularly fast as it walks entire line:col→DFA map looking for a prefix of "line:".
-
getLookaheadDFAFromPositionInFile
public DFA getLookaheadDFAFromPositionInFile(int line, int col)
Useful for ANTLRWorks to map position in file to the DFA for display
-
getLineColumnToLookaheadDFAMap
public java.util.Map<java.lang.String,DFA> getLineColumnToLookaheadDFAMap()
-
getNumberOfDecisions
public int getNumberOfDecisions()
-
getNumberOfCyclicDecisions
public int getNumberOfCyclicDecisions()
-
setLookaheadDFA
public void setLookaheadDFA(int decision, DFA lookaheadDFA)
Set the lookahead DFA for a particular decision. This means that the appropriate AST node must updated to have the new lookahead DFA. This method could be used to properly set the DFAs without using the createLookaheadDFAs() method. You could do this Grammar g = new Grammar("..."); g.setLookahead(1, dfa1); g.setLookahead(2, dfa2); ...
-
setDecisionNFA
public void setDecisionNFA(int decision, NFAState state)
-
setDecisionBlockAST
public void setDecisionBlockAST(int decision, GrammarAST blockAST)
-
allDecisionDFAHaveBeenCreated
public boolean allDecisionDFAHaveBeenCreated()
-
getMaxTokenType
public int getMaxTokenType()
How many token types have been allocated so far?
-
getMaxCharValue
public int getMaxCharValue()
What is the max char value possible for this grammar's target? Use unicode max if no target defined.
-
getTokenTypes
public IntSet getTokenTypes()
Return a set of all possible token or char types for this grammar
-
getAllCharValues
public IntSet getAllCharValues()
If there is a char vocabulary, use it; else return min to max char as defined by the target. If no target, use max unicode char value.
-
getANTLRCharLiteralForChar
public static java.lang.String getANTLRCharLiteralForChar(int c)
Return a string representing the escaped char for code c. E.g., If c has value 0x100, you will get "Ā". ASCII gets the usual char (non-hex) representation. Control characters are spit out as unicode. While this is specially set up for returning Java strings, it can be used by any language target that has the same syntax. :) 11/26/2005: I changed this to use double quotes, consistent with antlr.g 12/09/2005: I changed so everything is single quotes
-
complement
public IntSet complement(IntSet set)
For lexer grammars, return everything in unicode not in set. For parser and tree grammars, return everything in token space from MIN_TOKEN_TYPE to last valid token type or char value.
-
complement
public IntSet complement(int atom)
-
isValidSet
public boolean isValidSet(TreeToNFAConverter nfabuilder, GrammarAST t)
Given set tree like ( SET A B ), check that A and B are both valid sets themselves, else we must tree like a BLOCK
-
getSetFromRule
public IntSet getSetFromRule(TreeToNFAConverter nfabuilder, java.lang.String ruleName) throws org.antlr.runtime.RecognitionException
Get the set equivalent (if any) of the indicated rule from this grammar. Mostly used in the lexer to do ~T for some fragment rule T. If the rule AST has a SET use that. If the rule is a single char convert it to a set and return. If rule is not a simple set (w/o actions) then return null. Rules have AST form: ^( RULE ID modifier ARG RET SCOPE block EOR )- Throws:
org.antlr.runtime.RecognitionException
-
getNumberOfAltsForDecisionNFA
public int getNumberOfAltsForDecisionNFA(NFAState decisionState)
Decisions are linked together with transition(1). Count how many there are. This is here rather than in NFAState because a grammar decides how NFAs are put together to form a decision.
-
getNFAStateForAltOfDecision
public NFAState getNFAStateForAltOfDecision(NFAState decisionState, int alt)
Get the ith alternative (1..n) from a decision; return null when an invalid alt is requested. I must count in to find the right alternative number. For (A|B), you get NFA structure (roughly): o->o-A->o | o->o-B->o This routine returns the leftmost state for each alt. So alt=1, returns the upperleft most state in this structure.
-
FIRST
public LookaheadSet FIRST(NFAState s)
-
LOOK
public LookaheadSet LOOK(NFAState s)
-
setCodeGenerator
public void setCodeGenerator(CodeGenerator generator)
-
getCodeGenerator
public CodeGenerator getCodeGenerator()
-
getGrammarTree
public GrammarAST getGrammarTree()
-
setGrammarTree
public void setGrammarTree(GrammarAST value)
-
getTool
public Tool getTool()
-
setTool
public void setTool(Tool tool)
-
computeTokenNameFromLiteral
public java.lang.String computeTokenNameFromLiteral(int tokenType, java.lang.String literal)
given a token type and the text of the literal, come up with a decent token type label. For now it's just T<type>. Actually, if there is an aliased name from tokens like PLUS='+', use it.
-
toString
public java.lang.String toString()
- Overrides:
toString
in classjava.lang.Object
-
grammarTreeToString
public java.lang.String grammarTreeToString(GrammarAST t)
-
grammarTreeToString
public java.lang.String grammarTreeToString(GrammarAST t, boolean showActions)
-
printGrammar
public void printGrammar(java.io.PrintStream output)
-
-