4 ms·
If the LLM hallucinates a Python package, someone else might notice that and sneak in a malicious version of that hallucinated package. Or if it hallucinates a
by lambda 2y ago
If the LLM hallucinates a Python package, someone else might notice that and sneak in a malicious version of that hallucinated package.
Or if it hallucinates a method name, it might be in a code path that goes untested. How often are people using these tools to also write comprehensive test suites?
- simonw 2y agoLike I said, you have to put the work in if you're going to use these tools effectively. I write a LOT of tests both with and without the help of LLMs. I try hard never to leave any code path from an LLM (or from code I write myself) untested - always manually and usually through automated tests too. If you don't do that, the chances LLM-generated code will introduce weird bugs are high. But the chances you yourself (or your coworkers) will introduce weird bugs are high as well - QA, code reviews and testing are important no matter what helped write the code.