5 ms·
>> It's over zealous at times (which is why I stopped using Claude) and gets too creative when doing agentic system level stuff. Accessing files and doing thing
by enraged_camel 7d ago
>> It's over zealous at times (which is why I stopped using Claude) and gets too creative when doing agentic system level stuff. Accessing files and doing things it shouldn't do.
I gave Astra a pretty straightforward bug ticket yesterday. The bug involved an edge case that could sometimes result in an invalid value getting stored in a user profile field. Pretty harmless, no crash or anything, just annoying.
Based on past experience, I don't trust OpenAI, so I decided to watch Astra as it worked. About four minutes in, it convinced itself that it should also check the prod database to see "how far the corruption has spread" and attempted to SSH into the hosting provider. This resulted in my 1Password to prompt me, which I of course denied. Then I stopped Astra, closed the ChatGPT/Codex app and gave the task to Opus 5. Suffice it to say I will not be renewing my subscription, because "you have to watch it like a hawk" is the opposite of agentic engineering.
- wilj 7d agoChatGPT desktop this morning lost a chat thread while I was actively working in it. I asked Astra to find the lost session, and next thing I know it's prompting for full computer control to drive Finder. It's just jsonl files on disk, not hard to read normally. Negative feedback filed and ChatGPT uninstalled.
- silversmith 7d agoWhy is your agent able to call ssh. Why can it trigger 1password. Why are you giving metaphorical guns to metaphorical toddlers. Why is it not sandboxed. Your practices worry me.
- olalonde 7d agoAre you guys all running agents in VMs?
- lann 7d agoYes.
- olalonde 7d agoContainer or full blown VM?
- mike_hearn 6d agoI use containers in one context (custom container manager) and a regular UNIX account on bare metal in another. This isn't intended to stop a model like Astra hacking its way out of course, it's more like guardrails on a staircase. My personal container manager tool has an intercepting SSL proxy and small Javascripts on the host can rewrite or block HTTP requests. The agent gets its own isolated home directory and can't tamper with mine. Local caches like Maven are mapped read/only with a write layer on top.
- AnimalMuppet 7d agoYou have to watch it like a hawk so it doesn't do something to production, on its own, without a specific request? Wow. Then I could never trust it to not be doing something to some other system that it shouldn't, so I'd have to audit every network request. If enraged_camel had been doing something else involving the production database at the wrong time, they might have accepted the 1Password prompt.
- enraged_camel 7d agoWorth noting that this has never, ever happened with Anthropic models, which I've been using all day every day since Opus 4.1.