6 ms·

At CodeSandbox we use Firecracker for hosting development environments, and I agree with the points. Though I don't think that means you should not use Firecrac

At CodeSandbox we use Firecracker for hosting development environments, and I agree with the points. Though I don't think that means you should not use Firecracker for running long-lived workloads. We reclaim memory with a memory balloon device, for the disk trimming we discard (& compress) the disk, and for i/o speed we use io_uring (which we only use for scratch disks, the project disks are network disks). It's a tradeoff. It's more work and does require custom implementations. For us that made sense, because in return we get a lightweight VMM that we can more easily extend with functionality like memory snapshotting and live VM cloning [1][2]. [1]: https://codesandbox.io/blog/how-we-clone-a-running-vm-in-2-seconds https://codesandbox.io/blog/how-we-clone-a-running-vm-in-2-s... [2]: https://codesandbox.io/blog/cloning-microvms-using-userfaultfd https://codesandbox.io/blog/cloning-microvms-using-userfault...

alexeldeib 3y ago
These blogs are wonderful. I'd read them before figuring out firecracker snapshot/restore, but wanted to say it here. > i/o speed we use io_uring custom io_uring based driver for the VM block devices? or what do you mean here?
- CompuIves 3y ago
  Thank you! > custom io_uring based driver for the VM block devices? or what do you mean here? We're using the async io backend that's shipped with Firecracker for our scratch disks.
VWWHFSfQ 3y ago
I don't know if this is relevant, but I've been intrigued by DragonflyBSD's "vkernel" [0] feature which (supposedly) allows for cloning the entire runtime state of the machine (established TCP connections, etc.) into a completely new userland memory space. I think they use it mostly for kernel debugging right now, but it's interesting to think about the possibilities of being able to just clone an entire running operating system to a new computer without interrupting even a single instruction. [0] https://www.dragonflybsd.org/docs/handbook/vkernel/ https://www.dragonflybsd.org/docs/handbook/vkernel/

alexeldeib 3y ago

These blogs are wonderful. I'd read them before figuring out firecracker snapshot/restore, but wanted to say it here. > i/o speed we use io_uring custom io_uring based driver for the VM block devices? or what do you mean here?

CompuIves 3y ago
Thank you! > custom io_uring based driver for the VM block devices? or what do you mean here? We're using the async io backend that's shipped with Firecracker for our scratch disks.

VWWHFSfQ 3y ago

I don't know if this is relevant, but I've been intrigued by DragonflyBSD's "vkernel" [0] feature which (supposedly) allows for cloning the entire runtime state of the machine (established TCP connections, etc.) into a completely new userland memory space. I think they use it mostly for kernel debugging right now, but it's interesting to think about the possibilities of being able to just clone an entire running operating system to a new computer without interrupting even a single instruction. [0] https://www.dragonflybsd.org/docs/handbook/vkernel/ https://www.dragonflybsd.org/docs/handbook/vkernel/