5 ms·
One difference is that clichéd prose is bad and clichéd code is generally good.
by lukasb 9mo ago
One difference is that clichéd prose is bad and clichéd code is generally good.
- joshka 9mo agoDepends on what your prose is for. If it's for documentation, then prose which matches the expected tone and form of other similar docs would be clichéd in this perspective. I think this is a really good use of LLMs - making docs consistent across a large library / codebase.
- minimaxir 9mo agoI have been testing agentic coding with Claude 4.5 Opus and the problem is that it's too good at documentation and test cases. It's thorough in a way that it goes out of scope, so I have to edit it down to increase the signal-to-noise.
- girvo 9mo agoThe “change capture”/straight jacket style tests LLMs like to output drive me nuts. But humans write those all the time too so I shouldn’t be that surprised either!
- mulmboy 9mo agoWhat do these look like?
- pmg101 9mo ago1. Take every single function, even private ones. 2. Mock every argument and collaborator. 3. Call the function. 4. Assert the mocks were called in the expected way. These tests help you find inadvertent changes, yes, but they also create constant noise about changes you intend.
- ornornor 9mo agoJuniors on one of the teams I work with only write this kind of tests. It’s tiring, and I have to tell them to test the behaviour, not the implementation. And yet every time they do the same thing. Or rather their AI IDE spits these out.
- senbrow 9mo agoThese tests also break encapsulation in many cases because they're not testing the interface contract, they're testing the implementation.
- girvo 9mo agoYou beat me to it, and yep these are exactly it. “Mock the world then test your mocks”, I’m simply not convinced these have any value at all after my nearly two decades of doing this professionally
- diamond559 9mo agoIf the goal is to document the code and it gets sidetracked and focuses on only certain parts it failed the test. It just further proves llm's are incapable of grasping meaning and context.
- dcre 9mo agoDocs also often don’t have anyone’s name on them, in which case they’re already attributed to an unknown composite author.
- danenania 9mo agoA problem I’ve found with LLMs for docs is that they are like ten times too wordy. They want to document every path and edge case rather focusing on what really matters. It can be addressed with prompting, but you have to fight this constantly.
- bigiain 9mo agoI think probably my most common prompt is "Make it shorter. No more than ($x) (words|sentences|paragraphs)."
- pxc 9mo agoI've never been able to get that to work. LLMs can't count; they don't actually know how long their output is.
- pxc 9mo ago> A problem I’ve found with LLMs for docs is that they are like ten times too wordy This is one of the problems I feel with LLM-generated code, as well. It's almost always between 5x and long and 20x (!) as long as it needs to be. Though in the case of code verbosity, it's usually not because of thoroughness so much as extremely bad style.