I have similar compiler history and employ similar practice. Keep things where they logically belong - the parser is for mapping tokens onto syntax, for that i

I have similar compiler history and employ similar practice. Keep things where they logically belong - the parser is for mapping tokens onto syntax, for that it needs tokens! It doesn't matter if lexing is done in a separate pass, or "interleaved" in time. What matters is that the each "pass" is logically distinct. Most languages are designed so that it's easy to produce tokens from text, without knowing where you are in the syntax tree - that makes it easy to read code fragments. (There are languages where the same string can have very distinct structure & meaning depending on context. They tend to cause much swearing and confusion.)