I don't think these comparisons between AI generated code and an idealised version of the code are productive. We have to compare with code that exists in the r

I don't think these comparisons between AI generated code and an idealised version of the code are productive. We have to compare with code that exists in the real world, so AI against human, and not only that, good AI generated code with good human written code, as well as AI slop with human slop. I think if we did that we'd see that on average the code written by humans are less legible, parts of the intention will live forever in the mind of the developer at time of writing, and more prone to complexity build-up over time, simply because there wasn't time and incentive to go back and refactor code that works, apparently. Now with AI sure you produce a lot more complexity, more than the human prompter could write by himself, but complexity can be managed with the same workflow that created it, by analysing and removing code paths, changing code architecture, replacing reimplementation with consolidated libs etc... It's a matter of knowing how to use the tool and not creating a false sense of nostalgia where we feel like we had it better in the old days, which is not true at all. > How do you reward human-understandability in benchmarks and unsupervised training? And to answer this, there is no replacement for humans immersed in their world yet, so we need developers with with good understanding of the domain and that are able to write good descriptive prose in order to steer agents into producing acceptable code.