6 ms·
At work I recently credited Claude with writing some of the tests in my PR. Why? Partly to apologize for the verbosity of the tests making them harder to revi
by comex 2mo ago
At work I recently credited Claude with writing some of the tests in my PR. Why? Partly to apologize for the verbosity of the tests making them harder to review. Partly to set expectations: the tests are more likely to be buggy, low-quality, or incomplete, or misunderstand the code being tested, than if I had written them by hand.
I did in fact "bother to review" the tests, and I thought they were good enough to merge. Indeed, they were better than what I would have written on my own: they were lower-quality on a line-by-line basis, but for tests, quantity is a quality of its own, and I would not have bothered to write nearly as many tests. Even so, it was worth flagging that Claude wrote them.
If I trusted LLMs more, would I still feel the need to credit them? Maybe. They are improving very quickly, and I think they will eventually improve to the point where a mostly-LLM-written PR will be as good as mine in every way. Then it will no longer be necessary to apologize for submitting LLM code. But if and when that point is reached, I wouldn’t get used to taking credit for said code! Because it probably won’t be much longer until the human in the loop becomes redundant.
Going back to today, I don’t feel the need to credit LLMs when I use them for tasks that don’t involve submitting their output. In particular, they have recently become quite good at finding bugs in my code.