6 ms·

gracefully handling errors and giving useful output is the hardest part of writing parsers, why does everybody skip over that part?? :)

yellowapple 6y ago
Agreed, and it's disappointing that you got downvoted to oblivion for raising a valid point. There is an abundance of articles on parser combinators and grammars and other theoretical parsing concepts, and a dearth of articles (AFAICT) on actual practical applications or graceful error handling. I suspect the difficulty of error handling or parsing a non-trivial language are exactly why few articles cover them. Crafting Interpreters¹ seems to be a notable exception here. ---- ¹: https://www.craftinginterpreters.com/parsing-expressions.html#syntax-errors https://www.craftinginterpreters.com/parsing-expressions.htm...
BoiledCabbage 6y ago
Anyone know how Elm does it? Elm is incredible. It should be the gold standard any compiler strives for is quality of messages.
- saagarjha 6y ago
  I don’t know much about Elm, but one of the few things I have heard of is this GitHub issue: https://github.com/elm/compiler/issues/1773 https://github.com/elm/compiler/issues/1773. Am I putting too much emphasis on this particular picked cherry or does the compiler do this in general?
  - hawski 6y ago
    It seems that instead of fixing it it is an error now [0]. This and the comment from the issue [1] do not inspire confidence in me. [0] https://github.com/elm/compiler/commit/0e669b8ad076f64e450f2fa9b29ce93791466667 https://github.com/elm/compiler/commit/0e669b8ad076f64e450f2... [1] https://github.com/elm/compiler/issues/1773#issuecomment-418478847 https://github.com/elm/compiler/issues/1773#issuecomment-418... the same on master
lalaithion 6y ago
You can get parsing stacktraces by changing the definition of Parser to data Parser a c = Parser { runParser :: String -> Either [c] (a, String) } and adding a combinator withContext c p = Parser \s -> case runParser p of Left stackTrace = Left (c:stackTrace) Right x = Right x This allows you to write stuff like number = withContext "parsing a number" $ ... addition = withContext "parsing addition expression" $ ... expr = withContext "parsing a mathematical expression" $ ... and combine that with a technique that keeps track of where you are in the string when failure occurs, you can pretty print that to something like: Failed to parse! 2 + 34.0O4 ^ While: parsing a number While: parsing addition expression While: parsing a mathematical expression
- mathgladiator 6y ago
  So, one of the key things that I see is that the constructed abstract syntax tree must have the raw tokens within it so all other layers have full awareness. I feel the criticism is valid because most parsers in production are done via hand without tooling or fancy techniques.
  - Quekid5 6y ago
    > most parsers in production are done via hand without tooling or fancy techniques. What? Do you have any data you'd like to share? I'm given to understand that e.g. the C++ compilers usually have a hand-coded, but AFAIUI that's mostly due to the complexity of actually parsing it (and fitting that into anything other than just raw code).
    - mathgladiator 6y ago
      only anecdotal and observations over the years https://www.drdobbs.com/architecture-and-design/so-you-want-to-write-your-own-language/240165488 https://www.drdobbs.com/architecture-and-design/so-you-want-... A common theme is that the parser generator does not provide you the tools to write the high quality error messages. Having used ANTLR and other tools, I believe it now that I'm trying to ship a real language.
madushan1000 6y ago
Checkout Munch[1], it has full error handling and fully functioning, the tutorial is long and sometimes hard to follow though. [1] http://nbloomf.blog/munch/munch.html http://nbloomf.blog/munch/munch.html
whateveracct 6y ago
Mega parsec is a parser combinator library that definitely satisfies these requirements. In fact, I would say Haskell-style parser combinators are the best way to satisfy those requirements in general.

yellowapple 6y ago

Agreed, and it's disappointing that you got downvoted to oblivion for raising a valid point. There is an abundance of articles on parser combinators and grammars and other theoretical parsing concepts, and a dearth of articles (AFAICT) on actual practical applications or graceful error handling. I suspect the difficulty of error handling or parsing a non-trivial language are exactly why few articles cover them. Crafting Interpreters¹ seems to be a notable exception here. ---- ¹: https://www.craftinginterpreters.com/parsing-expressions.html#syntax-errors https://www.craftinginterpreters.com/parsing-expressions.htm...

BoiledCabbage 6y ago

Anyone know how Elm does it? Elm is incredible. It should be the gold standard any compiler strives for is quality of messages.

saagarjha 6y ago
I don’t know much about Elm, but one of the few things I have heard of is this GitHub issue: https://github.com/elm/compiler/issues/1773 https://github.com/elm/compiler/issues/1773. Am I putting too much emphasis on this particular picked cherry or does the compiler do this in general?
- hawski 6y ago
  It seems that instead of fixing it it is an error now [0]. This and the comment from the issue [1] do not inspire confidence in me. [0] https://github.com/elm/compiler/commit/0e669b8ad076f64e450f2fa9b29ce93791466667 https://github.com/elm/compiler/commit/0e669b8ad076f64e450f2... [1] https://github.com/elm/compiler/issues/1773#issuecomment-418478847 https://github.com/elm/compiler/issues/1773#issuecomment-418... the same on master

lalaithion 6y ago

You can get parsing stacktraces by changing the definition of Parser to data Parser a c = Parser { runParser :: String -> Either [c] (a, String) } and adding a combinator withContext c p = Parser \s -> case runParser p of Left stackTrace = Left (c:stackTrace) Right x = Right x This allows you to write stuff like number = withContext "parsing a number" $ ... addition = withContext "parsing addition expression" $ ... expr = withContext "parsing a mathematical expression" $ ... and combine that with a technique that keeps track of where you are in the string when failure occurs, you can pretty print that to something like: Failed to parse! 2 + 34.0O4 ^ While: parsing a number While: parsing addition expression While: parsing a mathematical expression

mathgladiator 6y ago
So, one of the key things that I see is that the constructed abstract syntax tree must have the raw tokens within it so all other layers have full awareness. I feel the criticism is valid because most parsers in production are done via hand without tooling or fancy techniques.
- Quekid5 6y ago
  > most parsers in production are done via hand without tooling or fancy techniques. What? Do you have any data you'd like to share? I'm given to understand that e.g. the C++ compilers usually have a hand-coded, but AFAIUI that's mostly due to the complexity of actually parsing it (and fitting that into anything other than just raw code).
  - mathgladiator 6y ago
    only anecdotal and observations over the years https://www.drdobbs.com/architecture-and-design/so-you-want-to-write-your-own-language/240165488 https://www.drdobbs.com/architecture-and-design/so-you-want-... A common theme is that the parser generator does not provide you the tools to write the high quality error messages. Having used ANTLR and other tools, I believe it now that I'm trying to ship a real language.

madushan1000 6y ago

Checkout Munch[1], it has full error handling and fully functioning, the tutorial is long and sometimes hard to follow though. [1] http://nbloomf.blog/munch/munch.html http://nbloomf.blog/munch/munch.html

whateveracct 6y ago

Mega parsec is a parser combinator library that definitely satisfies these requirements. In fact, I would say Haskell-style parser combinators are the best way to satisfy those requirements in general.