6 ms·
My first reaction: wow, incredible. My second reaction: still incredible, but noting that a C compiler is one of the most rigorously specified pieces of softwa
by itay-maman 8mo ago
My first reaction: wow, incredible.
My second reaction: still incredible, but noting that a C compiler is one of the most rigorously specified pieces of software out there. The spec is precise, the expected behavior is well-defined, and test cases are unambiguous.
I'm curious how well this translates to the kind of work most of us do day-to-day where requirements are fuzzy, many edge cases are discovered on the go, and what we want to build is a moving target.
- ndesaulniers 8mo ago> C compiler is one of the most rigorously specified pieces of software out there /me Laughs in "unspecified behavior."
- ori_b 8mo agoThere's undefined behavior, which is quite well specified. What do you mean by unspecified behavior? Do you have an example?
- ndesaulniers 7mo agohttps://www.open-std.org/jtc1/sc22/wg14/www/docs/n3685.pdf https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3685.pdf Read section J.1.
- irishcoffee 8mo agoUndefined is absolutely clear in the spec. Unspecified is whatever you want it to mean. I am also laughing, having never heard "unspecified" before.
- LiamPowell 7mo agoUnspecified behaviour is defined in the glossary at the start of the spec and the term "unspecified" appears over a hundred times...
- softwaredoug 8mo agoYes I think any codegen with a lot of tests and verification is more about “fitting” to the tests. Like fitting an ML model. It’s model training, not coding. But a lot of programming we discover correctness as we go, one reason humans don’t completely exit the loop. We need to see and build tests as we go, giving them particular care and attention to ensure they test what matters.
- uywykjdskn 7mo agoThe agent can obviously do that
- cryptonector 7mo ago> My second reaction: This is the key: the more you constrain the LLM, the better it will perform. At least that's my experience with Claude. When working with existing code, the better the code to begin with, the better Claude performs, while if the code has issues then Claude can end up spinning its wheels.
- astrange 7mo agoThe C spec is certainly not formal or precise. https://www.ralfj.de/blog/2020/12/14/provenance.html https://www.ralfj.de/blog/2020/12/14/provenance.html Another example is that it's unclear from the standard if you can write malloc() in C.