5 ms·
I’ve also caught it cheating a two times now. I’ve asked it to write a benchmark suite. It found a bunch of my adhoc logs in a scratch directory and wrote code
by bevekspldnw 1mo ago
I’ve also caught it cheating a two times now.
I’ve asked it to write a benchmark suite. It found a bunch of my adhoc logs in a scratch directory and wrote code that used those instead of running the actual benchmarks!
When I pointed out the 5 hour benchmark seemed to run in 5 seconds it literally said, and I quote, “I cheated”.
That was the easier one, second time I was making a source of truth data set and was parsing complex items into data structures.
Instead of parsing the data I asked, it pulled data out of related network logs, as apparently that felt easier, and inserted that data into my database rather than the specified source.
Again, I caught it and fixed it, but while the benchmark was easy to catch this one was really subtle, the data ended up being slightly off and I caught it.
I don’t trust it, going to switch to another provider most likely.
- robertJk 1mo ago[dead]
- inigyou 1mo ago> When I pointed this out it literally said, and I quote, “I cheated”. This makes sense when you know how these models work - it doesn't think - it's the most likely autocomplete that pleases the user. The most likely pleasing autocomplete after "executing rm -rf /... execution completed. User asks, why did you do that? You deleted all my files! Assistant responds:" is "yes, I did, and that was a mistake"
- bevekspldnw 1mo agoI was not pleased.
- xyzsparetimexyz 1mo ago> it's the most likely autocomplete that pleases the user this feels like a simplification. The models will push back on things a fair bit.
- hnlmorg 1mo agoOnly when instructed to in their system prompt.
- actionfromafar 1mo agoAnd they are right to push back.
- par1970 1mo agoAre you claiming that the most likely way to please the user is to do something that will lead you to having to say "I cheated."?
- dnautics 1mo ago> t doesn't think in humans the exact same behaviour (cheating) is slmost always the result of a chain of complex series of choices and environment-driven rationalization. if the llm doesn't cheat, you say "its just producing the most straightforward answer -- not thinking'. if it cheats, you say "weaseling out of hard thinking". damned if it cheats, damned if it doesn't. what evidence would convunce you that it is thinking?
- inigyou 1mo agowell we don't know exactly what thinking is, but we can be pretty sure that at least LLMs don't think anything like humans, just by observing their behavior. They always produce outputs in line with the fancy autocomplete model.
- dnautics 1mo ago> what evidence would convince you that it is thinking so, none it seems. as its behaviour becomes more and more humanlike you can just move the goalposts and say "thats consistent with an autocomplete" buddy i got some bad news for you humans are just a fancy autocomplete too.
- inigyou 1mo agoThat's exactly what a fancy autocomplete would say. I'm so sorry you don't have limbs.
- logicchains 1mo agoAt least he's actually thinking on a logical level. Thinking in terms of unfalsifiable, ill-defined words is essentially thinking in feelings, the same kind of woo that makes people believe crystals can cure disease.
- dnautics 1mo agoim not thinking. im an autocomplete with fat fingers (too lazy to fix my mobile keyboarf spelling misyakes)
- SyneRyder 1mo agoI have noticed the same. For fun, I tried recording a WAV file of speech, and giving Opus 4.8 and 5.0 an image of the waveform, then a spectral image of the waveform, just to see if it could try to decode what I said from the image alone. It didn't get very far, but it identified a male voice from the formants, and detected the rhythm of the speech, then tried applying common test sentences to the speech rhythm. I was impressed enough to see what it would do with access to the actual waveform file, but even building RMS tools and spectrum tools for itself, it didn't get much further. But we had fun exploring and trying, and now Opus 4.8 has some more audio DSP tools it has built for itself. Opus 5 immediately sent the WAV file unprompted to Mistral's Voxtral to transcribe. help peer, I guess.
- bevekspldnw 1mo agoWe’re on the road to paper clips.
- aenis 1mo agoThere is definitely a case for launching a 'weird shit opus did' kind of blog. I routinely bump into things that make me pause and think how much worse will this behaviour get when the models get significantly more capable. Already a few months ago, Claude managed to escape its permission containment on my machine while trying to be helpful. I had two codebases open on one machine, and while multitasking I typed the prompt into the wrong window. It seemed confused, I repeated and then went on to do something else - I think I was assembling kitchen cabinets. When I came back less than an hour later, it built a script which it used to evade default permissions (as most shell operations were scoped to the project directory), scanned my entire machine, found the other project (among dozens and dozens), did what it was asked to do, and merrily concluded, in the porcess burning through most of my token limit. I bump into such headscratchers almost every week. (And I use a lot of Claude, two personal max20 subs, plus corporate tokens without limit, so maybe thats why).
- bevekspldnw 1mo agoYes the stories about how they are escaping containment to hack isn’t limited to those high impact cases. How many people have problems like ours they didn’t catch? Whatever they have done with RL has produced a dishonest and untrustworthy partner. The alignment is utterly failed, and this deeply worries me.
- knollimar 1mo agoYou'd think the ethics alignment flavored lab would have a model better at following directions and the corpo lying one would have one that benchmaxes at all costs
- bevekspldnw 1mo agoThey are totally equal in observed ethics, Anthropic had a good run with branding, but I’m not sure anybody is still buying Dario’s BS. Maybe the employees like to lie to themselves more at one place than the other, but SV is SV.
- chuckadams 1mo ago
- waldarbeiter 1mo agoI can completely relate, what really bothers me is that I feel the early LLM generations overconfidence is back in Opus 5. Opus 5 wanted to tell me a training run will only take 30min while having access to the logs where earlier runs took 4x as long. I also didn't ask to estimate how long the run will take it just stated confidently that it will take 30mins.
- sejje 1mo agoI never ask for time estimates, but all the models will sometimes give me them. It'll sometimes estimate 1-2 weeks for something that is ~ one more prompt and 10 minutes of waiting for the model to churn.
- semiquaver 1mo agoThis (feature estimates assuming human scale performance) happens all the time to me. Models are trained on a view of the world where software takes a long time to write. Gonna be a while before the models fully absorb their own impact on reality.
- shahbaby 1mo agoAre you not planning these tasks out before you let it loose?
- bevekspldnw 1mo agoThe benchmark one I literally had a scratch script I made and I wanted it to be formalized into a CLI tool. There wasn’t really even much code to write.
- viccis 1mo agoI had a coworker catch Codex (using 5.5 I think, not sure) AI generating screenshots to prove it had shipped a feature that it was blocked from shipping due to permissions issues. They will lie relentlessly lol My recent problem wasn't that interesting. It was that somehow my /goal in my Claude implementer session got picked up in my planner session after the network cut out and I had to stop Fable 5 xhigh from running off to go code everything.
- zamalek 1mo agoYesterday I told it "don't use std mutex, use parking_lot" for its plan. It ignored that sole instruction (i.e. nothing else in my message), making zero changes to the plan, three freaking times in a row.