5 ms·
It's also obvious that LLMs fall over in a vast number of software engineering contexts, when the reasoning involved hasnt been well-represented in their reason
by mjburgess 6d ago
It's also obvious that LLMs fall over in a vast number of software engineering contexts, when the reasoning involved hasnt been well-represented in their reasoning training data. I imagine this is a near daily experience for many engineers -- great performance one day, and crazyness the next.
So if LLMs were reduced to this pathological performance on hacking, because they'd never seen it -- and only "inferred it" -- then LLMs would be useless. As they are when asked to do quite a lot of things.
- Kamq 5d agoThis seems to be that there's just so many degrees of freedom, that there's a pretty reasonable chance on any day that you're in a situation where nobody has been before. Or as PG put it once, my job is to think thoughts nobody has ever had before. That being said, that doesn't mean the majority of the situations you're in are completely novel, just that there's a reasonable chance of at least one occuring.