6 ms·
Opus 5 generates really good code and terminal commands though. It's just bad at the accompanying text it tells you. These benchmarks don't grade the text gener
by didibus 6d ago
Opus 5 generates really good code and terminal commands though. It's just bad at the accompanying text it tells you. These benchmarks don't grade the text generation of the response I don't think, only the task outcome.
- simondotau 5d agoOpus 5 keeps making howlingly stupid errors, like one recently where a regexp would catch an invalid date because -\d{2} won’t match -00 Seriously.
- didibus 4d agoThat's just all models though. They're not 100% accurate.
- simondotau 3d agoI regularly have issues like that with Opus. I rarely do with the supposedly sub-frontier Grok 4.6, or Kimi K3, or even with Sol.