核磁共振Ruby 1.9.2中的词汇分析

在ruby源代码中有 parse.y 包含语法的文件。我相对确定ruby使用单独的lexer(就像大多数LR解析器一样)。同时,似乎雷克萨斯是有状态的:

enum lex_state_e {
EXPR_BEG,           /* ignore newline, +/- is a sign. */
EXPR_END,           /* newline significant, +/- is an operator. */
EXPR_ENDARG,        /* ditto, and unbound braces. */
EXPR_ARG,           /* newline significant, +/- is an operator. */
EXPR_CMDARG,        /* newline significant, +/- is an operator. */
EXPR_MID,           /* newline significant, +/- is an operator. */
EXPR_FNAME,         /* ignore newline, no reserved words. */
EXPR_DOT,           /* right after `.' or `::', no reserved words. */
EXPR_CLASS,         /* immediate after `class', no here document. */
EXPR_VALUE          /* alike EXPR_BEG but label is disallowed. */
};

但我不是专家。

编辑:在parse.y文件中深入查看,lexer并没有与解析器完全分离:

superclass  : //[...]
    | '<'
        {
        lex_state = EXPR_BEG;
        }