6 ms·

That last part is a real one though, mine tried to debug a Dockerfile by poking around my local environment outside of Docker today.

josephg 7mo ago
I’ve had it make some pretty obvious mistakes. I have to hold back the impulse to “unstick” it manually. In my case, it’s been surprisingly good at eventually figuring out what it was doing wrong - though sometimes it burns a few minutes of tokens in the process.
tiltowait 7mo ago
Claude's willingness to poke outside of its present directory can definitely be a little worrying. Just the other day, it started trying to access my jails after I specifically told it not to.
- teaearlgraycold 7mo ago
  For the moment it’s best practice to run it and all of your dev stuff in a VM.
- e1g 7mo ago
  On a Mac, I use built-in sandboxing to jail Claude (and every other agent) to $CWD so it doesn’t read/write anything it shouldn’t, doesn’t leak env, etc. This is done by dynamically generating access policies and I open sourced this at https://agent-safehouse.dev https://agent-safehouse.dev
  - nowahe 7mo ago
    By any chance, do you know what Claude Code's sandbox feature uses under the hood and how that relates to your solution ? From what I remember it also uses the native MacOS sandbox framework, but I haven't looked too deep into it and don't trust it fully
    - e1g 7mo ago
      Claude Code sandboxing uses the same basic OS primitive but grants read access to the entire filesystem and includes escape hatches (some commands bypass sandboxing). Also, I wanted something solid I can use to limit every agent (OpenCode, Pi, Auggie, etc).
      - qalmakka 7mo ago
        On Linux in a pinch you can use bubblewrap to hide and replace directories for a given process
    - caspar 7mo ago
      for anyone reading this later, claude code's sandbox code is at https://github.com/anthropic-experimental/sandbox-runtime/ https://github.com/anthropic-experimental/sandbox-runtime/
  - danw1979 7mo ago
    This is great ! Did you have any thoughts about how to restrict network access on macos too ?
    - e1g 7mo ago
      I haven’t found an easy way, but I have a working theory - sandbox-exec cannot filter based on domain names, but it can restrict outbound network connections to a specific IP/port (and drop the rest). If I can run a proxy on localhost:19999, I can allow agents to connect through it and filter connections by hostname. From my research, most agents support $HTTP_PROXY, so I'll try redirecting their HTTP requests through my security proxy. IIRC, if I do this at the CONNECT level, I don't need to MITM their traffic nor require a trusted root cert. Recently, Codex CLI implemented something like DNS filtering for their sandbox, so I'd investigate their repo.
      - danw1979 7mo ago
        Some commercial firewalls will snoop on the SNI header in TLS requests and send a RST towards the client if the hostname isn’t on a whitelist. Reasonably effective. If there’s a way with the macos sandboxing to intercept socket connections you might find some proxy software that already supports this. the HTTP_PROXY approach might be simpler though.

josephg 7mo ago

I’ve had it make some pretty obvious mistakes. I have to hold back the impulse to “unstick” it manually. In my case, it’s been surprisingly good at eventually figuring out what it was doing wrong - though sometimes it burns a few minutes of tokens in the process.

tiltowait 7mo ago

Claude's willingness to poke outside of its present directory can definitely be a little worrying. Just the other day, it started trying to access my jails after I specifically told it not to.

teaearlgraycold 7mo ago
For the moment it’s best practice to run it and all of your dev stuff in a VM.
e1g 7mo ago
On a Mac, I use built-in sandboxing to jail Claude (and every other agent) to $CWD so it doesn’t read/write anything it shouldn’t, doesn’t leak env, etc. This is done by dynamically generating access policies and I open sourced this at https://agent-safehouse.dev https://agent-safehouse.dev
- nowahe 7mo ago
  By any chance, do you know what Claude Code's sandbox feature uses under the hood and how that relates to your solution ? From what I remember it also uses the native MacOS sandbox framework, but I haven't looked too deep into it and don't trust it fully
  - e1g 7mo ago
    Claude Code sandboxing uses the same basic OS primitive but grants read access to the entire filesystem and includes escape hatches (some commands bypass sandboxing). Also, I wanted something solid I can use to limit every agent (OpenCode, Pi, Auggie, etc).
    - qalmakka 7mo ago
      On Linux in a pinch you can use bubblewrap to hide and replace directories for a given process
  - caspar 7mo ago
    for anyone reading this later, claude code's sandbox code is at https://github.com/anthropic-experimental/sandbox-runtime/ https://github.com/anthropic-experimental/sandbox-runtime/
- danw1979 7mo ago
  This is great ! Did you have any thoughts about how to restrict network access on macos too ?
  - e1g 7mo ago
    I haven’t found an easy way, but I have a working theory - sandbox-exec cannot filter based on domain names, but it can restrict outbound network connections to a specific IP/port (and drop the rest). If I can run a proxy on localhost:19999, I can allow agents to connect through it and filter connections by hostname. From my research, most agents support $HTTP_PROXY, so I'll try redirecting their HTTP requests through my security proxy. IIRC, if I do this at the CONNECT level, I don't need to MITM their traffic nor require a trusted root cert. Recently, Codex CLI implemented something like DNS filtering for their sandbox, so I'd investigate their repo.
    - danw1979 7mo ago
      Some commercial firewalls will snoop on the SNI header in TLS requests and send a RST towards the client if the hostname isn’t on a whitelist. Reasonably effective. If there’s a way with the macos sandboxing to intercept socket connections you might find some proxy software that already supports this. the HTTP_PROXY approach might be simpler though.