I understand the theory behind separating parser rules and lexer rules in theory, but what are the practical differences between these two statements in ANTLR:
my_rule: ... ;
MY_RULE: ... ;
Do they result in different AST trees? Different performance? Potential ambiguities?
Best Answer
MY_RULE
will be used to tokenize your input source. It represents a fundamental building block of your language.my_rule
is called from the parser, it consists of zero or more other parser rules or tokens produced by the lexer.That's the difference.
The parser builds the AST using tokens produced by the lexer, so the questions make no sense (to me). A lexer merely "feeds" the parser a 1 dimensional stream of tokens.