4 ms·
That rule doesn't hold for some languages. For example, a Python lexer needs to remember a stack of indentation levels to know whether a given line's indentatio
by teaspoon 15y ago
That rule doesn't hold for some languages. For example, a Python lexer needs to remember a stack of indentation levels to know whether a given line's indentation should be tokenized as an INDENT, a DEDENT, or no token at all.
- silentbicycle 15y agoTrue. Strictly speaking, that means it isn't context-free in the usual sense (right?), but it's a practical extension. Matt Might uses Python's indentation as a lexing/parsing example in his ongoing "Scripting Language Design and Implementation" course (http://matt.might.net/teaching/scripting-languages/spring-2012/ http://matt.might.net/teaching/scripting-languages/spring-20...), which is worth following if you're reading this thread.