6 ms·

There are sooooo many sandbox providers out there. They do spike on different features like: - snapshotting and forking - good SSH and VPN access for

by dbmikus 3mo ago

There are sooooo many sandbox providers out there. They do spike on different features like: - snapshotting and forking - good SSH and VPN access for end-users - agent-friendly features, like obscuring secrets at network layer Then there's also the option to use libkrun to run local sandboxes on your own computer. That doesn't scratch the itch for hosted services, but works if your goal is to run agents inside isolated environments for your own work. I've been working on some open-core stuff[1] to coordinate sandboxes, and we're making changes to have a library that lets people coordinate any number of remote or local sandboxes using any provider, kinda like how the Docker CLI works for managing containers, git repos, and coding agents. Flue[2] is another player in this space, and is more of a pure framework, while we're building it as an interactive product for using sandboxed agents and workflows. [1] https://github.com/gofixpoint/amika/blob/main/ROADMAP.md https://github.com/gofixpoint/amika/blob/main/ROADMAP.md [2]: https://flueframework.com/ https://flueframework.com/

stubbi 3mo ago
Thanks for sharing these!
sureglymop 3mo ago
Why isn't libkrun good enough for hosted stuff? I use it as a podman backend in a microservice architecture.
- dbmikus 3mo ago
  Firecracker has more tooling for the orchestration layer that manages many sandboxes at once. Stuff like K8S integration, an external REST API control plane, more first-class support for snapshotting, etc. You'd have to build more of that with libkrun The core tech of both are great though.
  - kodama-lens 3mo ago
    Firecracker has more tooling, but setting ist up and managing it is also more complicated, at least for k8s workloads. Libkrun is so easy for k8s! Compile crun with Libkrun support, crate a symlink of crun with the name krun, done. Works like any normal pod. Firecracker with kata-containers is a lot more brittle and complicated. I've invested quite some time getting this running for a talk I'm working on
    - dbmikus 3mo ago
      Is the talk going to be shared online anywhere? Would be interested in checking it out later!
- veverkap 3mo ago
  That's super interesting - have you written up anything on this? I'd love to read it.
  - sureglymop 3mo ago
    No, but I can give a small introduction. I installed krun from the arch package repositories: https://man.archlinux.org/man/extra/krun/krun.1.en https://man.archlinux.org/man/extra/krun/krun.1.en Then one can just pass `--runtime krun` to most podman subcommands. Alternatively, set the runtime key in the config file to make it the default. Podman itself has "hardening" techniques, e.g. turning off the network or volumes that can be combined with this.
- rvz 3mo ago
  libkrun is not production ready compared to Firecracker which the latter is used in 99.9% of many companies.
  - sureglymop 3mo ago
    For what exact reason is it not production ready? Or is that the stance of its maintainers?
PeterStuer 3mo ago
Setting up your own is not that hard and if you bought some compute before the Altman squeeze, very cheap.
- dbmikus 3mo ago
  Def! My personal belief is that the future of an "app" is a combo: 1. micro VM 2. agent on the VM 3. software bundled into the VM So, it should be stupid simple to run these local sandboxed apps/agents. Right now, not too hard for technical users (esp. with things like https://smolmachines.com/ https://smolmachines.com/ and https://microsandbox.dev/ https://microsandbox.dev/), but not as easy as clicking an app icon or typing `/path/to/binary` in the CLI
  - spockz 3mo ago
    Microsandbox claims to start faster than docker, and it is isolated from the host, and to work with OCI. Why would I still want to use docker? The only reason I can imagine is that I actually want to be able to dynamically share resources between containers instead of dividing up VMs a priori. Ah, the significant compute overhead: https://josecastillolema.github.io/podman-wasm-libkrun/ https://josecastillolema.github.io/podman-wasm-libkrun/. Much more cpu and ram usage at worse performance.
    - NamlchakKhandro 3mo ago
      Starting faster than a container will need evidence since starting a container on Linux is basically instant.
      - spockz 3mo ago
        It is instant for me when using podman but by no means instant when using docker. Docker on Linux native is stay way faster than on macOS and Windows. But so far running with podman has the lowest overhead I have seen.
    - mikeocool 3mo ago
reinitctxoffset 3mo ago
What people aren't getting with `firecracker` is utilization. Don't get me wrong, `firecracker` is great software and it's what I'm using for lightweight virtualization, but workloads are really bursty over really short periods of time now, even with the snapshot and restore that you can get if you're willing to hack on `firecracker` substantially, you hit walls where it's like, this is too much against the grain, this thing wasn't designed to bounce from 1 core to 32 to 8 to 16 to 4 to 32 to 1 seamlessly, and that's what it takes to get extreme utilization even with extremely good ML on the prediction. I am quite sure I'm not the only person working on post-firecracker KVM.
- binsquare 3mo ago
  I designed my take to basically eliminate the concept of vm being a rigid box of cpu/memory with CPU oversubscription and virtio-ballooning on memory + sparse ext4. That way it can be elastic in CPU, memory and somewhat disk. How far are you on your take?
znpy 3mo ago
We’re adopting agent-sandbox (https://agent-sandbox.sigs.k8s.io/ https://agent-sandbox.sigs.k8s.io/) as we already run most if our stuff in kubernetes and it’s been working very well, the only downside being it’s a moving target as it’s still essentially in development.

by dbmikus 3mo ago

stubbi 3mo ago

Thanks for sharing these!

sureglymop 3mo ago

Why isn't libkrun good enough for hosted stuff? I use it as a podman backend in a microservice architecture.

dbmikus 3mo ago
Firecracker has more tooling for the orchestration layer that manages many sandboxes at once. Stuff like K8S integration, an external REST API control plane, more first-class support for snapshotting, etc. You'd have to build more of that with libkrun The core tech of both are great though.
- kodama-lens 3mo ago
  Firecracker has more tooling, but setting ist up and managing it is also more complicated, at least for k8s workloads. Libkrun is so easy for k8s! Compile crun with Libkrun support, crate a symlink of crun with the name krun, done. Works like any normal pod. Firecracker with kata-containers is a lot more brittle and complicated. I've invested quite some time getting this running for a talk I'm working on
  - dbmikus 3mo ago
    Is the talk going to be shared online anywhere? Would be interested in checking it out later!
veverkap 3mo ago
That's super interesting - have you written up anything on this? I'd love to read it.
- sureglymop 3mo ago
  No, but I can give a small introduction. I installed krun from the arch package repositories: https://man.archlinux.org/man/extra/krun/krun.1.en https://man.archlinux.org/man/extra/krun/krun.1.en Then one can just pass `--runtime krun` to most podman subcommands. Alternatively, set the runtime key in the config file to make it the default. Podman itself has "hardening" techniques, e.g. turning off the network or volumes that can be combined with this.
rvz 3mo ago
libkrun is not production ready compared to Firecracker which the latter is used in 99.9% of many companies.
- sureglymop 3mo ago
  For what exact reason is it not production ready? Or is that the stance of its maintainers?

PeterStuer 3mo ago

Setting up your own is not that hard and if you bought some compute before the Altman squeeze, very cheap.

dbmikus 3mo ago
Def! My personal belief is that the future of an "app" is a combo: 1. micro VM 2. agent on the VM 3. software bundled into the VM So, it should be stupid simple to run these local sandboxed apps/agents. Right now, not too hard for technical users (esp. with things like https://smolmachines.com/ https://smolmachines.com/ and https://microsandbox.dev/ https://microsandbox.dev/), but not as easy as clicking an app icon or typing `/path/to/binary` in the CLI
- spockz 3mo ago
  Microsandbox claims to start faster than docker, and it is isolated from the host, and to work with OCI. Why would I still want to use docker? The only reason I can imagine is that I actually want to be able to dynamically share resources between containers instead of dividing up VMs a priori. Ah, the significant compute overhead: https://josecastillolema.github.io/podman-wasm-libkrun/ https://josecastillolema.github.io/podman-wasm-libkrun/. Much more cpu and ram usage at worse performance.
  - NamlchakKhandro 3mo ago
    Starting faster than a container will need evidence since starting a container on Linux is basically instant.
    - spockz 3mo ago
      It is instant for me when using podman but by no means instant when using docker. Docker on Linux native is stay way faster than on macOS and Windows. But so far running with podman has the lowest overhead I have seen.
  - mikeocool 3mo ago

reinitctxoffset 3mo ago

What people aren't getting with `firecracker` is utilization. Don't get me wrong, `firecracker` is great software and it's what I'm using for lightweight virtualization, but workloads are really bursty over really short periods of time now, even with the snapshot and restore that you can get if you're willing to hack on `firecracker` substantially, you hit walls where it's like, this is too much against the grain, this thing wasn't designed to bounce from 1 core to 32 to 8 to 16 to 4 to 32 to 1 seamlessly, and that's what it takes to get extreme utilization even with extremely good ML on the prediction. I am quite sure I'm not the only person working on post-firecracker KVM.

binsquare 3mo ago
I designed my take to basically eliminate the concept of vm being a rigid box of cpu/memory with CPU oversubscription and virtio-ballooning on memory + sparse ext4. That way it can be elastic in CPU, memory and somewhat disk. How far are you on your take?

znpy 3mo ago

We’re adopting agent-sandbox (https://agent-sandbox.sigs.k8s.io/ https://agent-sandbox.sigs.k8s.io/) as we already run most if our stuff in kubernetes and it’s been working very well, the only downside being it’s a moving target as it’s still essentially in development.