7 ms·
Why?
by eclipxe 6mo ago
Why?
- fl4regun 6mo agoIs this a serious question? If you are handling sensitive information how do you confirm your application is secure and won't leak or expose information to people who shouldn't know it?
- lijok 6mo agoHow do you with classic code?
- hallway_monitor 6mo agoExactly.... -> Unit tests. Integration tests. UI tests. This is how code should be verified no matter the author. Just today I told my team we should not be reading every line of LLM code. Understand the pattern. Read the interesting / complex parts. Read the tests.
- GrinningFool 6mo agoBut unit and integration tests generally only catch the things you can think of. That leaves a lot of unexplored space in which things can go wrong. Separately, but related - if you offload writing of the tests and writing of the code, how does anybody know what they have other than green tests and coverage numbers?
- dntrkv 6mo agoI have been seeing this problem building over the last year. LLM generated logic being tested by massive LLM generated tests. Everyone just goes overboard with the tests since you can easily just tell the LLM to expand on the suite. So you end up with a massive test suite that looks very thorough and is less likely to be scrutinized.
- fl4regun 6mo agoif you are asking me how you *guarantee* there is not a single possible exploit in your code, you can't do that. But you can do your best and learn about common pitfalls and be reasonably competent. Just because you can't do the former doesn't mean the latter is useless.