A TokenTransform is a function that maps a token to a set of symbols.
The token is described using a TransitionParserState and a StateRef (see the definition of
StateRef for details). For instance, using StackRef(0) will cause the TokenTransform to
operate on the token at the top of the stack in the current parser state.
The purpose of a TokenTransform is primarily to facilitate feature creation (e.g. see
StackRefFeature) by allowing us, say for instance, to map the token at top of the state's stack
to its word representation. This would be achieved with:
A TokenTransform is a function that maps a token to a set of symbols.
The token is described using a TransitionParserState and a StateRef (see the definition of StateRef for details). For instance, using StackRef(0) will cause the TokenTransform to operate on the token at the top of the stack in the current parser state.
The purpose of a TokenTransform is primarily to facilitate feature creation (e.g. see StackRefFeature) by allowing us, say for instance, to map the token at top of the state's stack to its word representation. This would be achieved with:
WordTransform(state, StackRef(0))