6 ms·

citation needed

by osipov 6y ago

citation needed

aidanhs 6y ago
I can't comment on performance, but on a different note it seems to be generally accepted that 'real' container boot time is slow - fly.io mention this too [0]. To add some numbers: - 40ms for nsjail run an isolated command and exit [1] - 150[2]-250ms to boot a firecracker microvm - ~450ms for docker startup [3] There are probably very good reasons for the difference (e.g. docker has layered filesystems to set up), but the default experience makes a difference. [0] https://news.ycombinator.com/item?id=24853660 https://news.ycombinator.com/item?id=24853660 [1] `nsjail --user 9999 --group 9999 -macvlan_iface wlp2s0 --chroot / -Mo --macvlan_vs_ip 192.168.0.44 --macvlan_vs_nm 255.255.255.0 --macvlan_vs_gw 192.168.0.1 -- /bin/true` [2] https://blog.acolyer.org/2020/03/02/firecracker/ https://blog.acolyer.org/2020/03/02/firecracker/ [3] `docker run -d ubuntu:18.04 true`
- tptacek 6y ago
  We've related the timing we've seen on our platform, and the fact that Firecracker works surprisingly well with the performance envelope we need, but I wouldn't go so far as to say Fly has benchmarked Docker vs. Firecracker. We're just relating our experiences. Security isolation is dispositive for us; we'd be using Firecracker even if it slowed us down.
yjftsjthsd-h 6y ago
Citation very needed, given that Firecracker is a VM and Docker runs binaries on bare metal. Maybe faster startup, but I'd want to see testing methods and data before I believe even that.
eyberg 6y ago
Generally speaking containers replicate networking && storage layers so are definitely much slower than a vm on public cloud, not to mention they generally live on vms. However, in this context (being on a rpi) it probably wouldn't be the case. Now, as for the firecracker discussion: Firecracker trades a faster boot time for a slower runtime as evidenced by this issue here: https://github.com/nanovms/nanos/issues/483#issuecomment-650352427 https://github.com/nanovms/nanos/issues/483#issuecomment-650... There are plans to fix this but afaict this is the case today.

citation needed

by osipov 6y ago

citation needed

I can't comment on performance, but on a different note it seems to be generally accepted that 'real' container boot time is slow - fly.io mention this too [0]. To add some numbers: - 40ms for nsjail run an isolated command and exit [1] - 150[2]-250ms to boot a firecracker microvm - ~450ms for docker startup [3] There are probably very good reasons for the difference (e.g. docker has layered filesystems to set up), but the default experience makes a difference. [0] https://news.ycombinator.com/item?id=24853660 https://news.ycombinator.com/item?id=24853660 [1] `nsjail --user 9999 --group 9999 -macvlan_iface wlp2s0 --chroot / -Mo --macvlan_vs_ip 192.168.0.44 --macvlan_vs_nm 255.255.255.0 --macvlan_vs_gw 192.168.0.1 -- /bin/true` [2] https://blog.acolyer.org/2020/03/02/firecracker/ https://blog.acolyer.org/2020/03/02/firecracker/ [3] `docker run -d ubuntu:18.04 true`

tptacek 6y ago
We've related the timing we've seen on our platform, and the fact that Firecracker works surprisingly well with the performance envelope we need, but I wouldn't go so far as to say Fly has benchmarked Docker vs. Firecracker. We're just relating our experiences. Security isolation is dispositive for us; we'd be using Firecracker even if it slowed us down.

yjftsjthsd-h 6y ago

Citation very needed, given that Firecracker is a VM and Docker runs binaries on bare metal. Maybe faster startup, but I'd want to see testing methods and data before I believe even that.

eyberg 6y ago

Generally speaking containers replicate networking && storage layers so are definitely much slower than a vm on public cloud, not to mention they generally live on vms. However, in this context (being on a rpi) it probably wouldn't be the case. Now, as for the firecracker discussion: Firecracker trades a faster boot time for a slower runtime as evidenced by this issue here: https://github.com/nanovms/nanos/issues/483#issuecomment-650352427 https://github.com/nanovms/nanos/issues/483#issuecomment-650... There are plans to fix this but afaict this is the case today.