5 ms·

I guess, if you thought Docker/etc. was a security boundary

markasoftware 2mo ago
Runpod, digital ocean's gpu cloud, and at least a few others use Linux containers for isolation between tenants (look at Wiz's blog post about the nvidia container toolkit bug; digitalocean just puts everyone in a massive k8s cluster)
- stingraycharles 2mo ago
  Why aren’t they using a fast VM like Firecracker?
  - himata4113 2mo ago
    To squeeze out 5% more profit.
circularfoyers 2mo ago
I know there's a lot you can do in k8s to mitigate it, but I didn't think that prevented it outright.
insanitybit 2mo ago
They are a security boundary. The fact that you need a vulnerability to escape them is proof of that. They just don't have a particularly high cost of escape because reachable kernel vulnerabilities are so common.
- worthless-trash 2mo ago
  Some people clearly do use containers as deployment mechanism, with security not in mind.
  - zbentley 2mo ago
    s/some/most/ That's not meant to be snide, just true, I think.
    - CodesInChaos 2mo ago
      I never understood why kubernetes doesn't use a VM-per-pod model by default.
- dijit 2mo ago
  Escape from docker containers is trivially easy, if you are able to run as the root user in the container itself. Many (maybe most) containers actually default to running programs as root. Kernel exploit not required.
  - maple3142 2mo ago
    If you are given a shell with `docker run -it --rm alpine:3 sh`, can you read the /etc/shadow on the host without kernel exploit? Assuming the docker and kernel are sufficiently update-to-date (e.g. latest Docker on Debian Stable).
    - chlorion 2mo ago
      No. The "root" you get in docker is not actually root outside of the namespace the container in running in. Assuming no bugs in the kernel, it should not be able to do anything more than the UID that it's mapped from.
      - ptx 2mo ago
        Does Docker use user namespaces by default? Otherwise root in the container is actually root on the host, from what I read. Correct me if I'm wrong. (Privileges are still limited by seccomp filters blocking some syscalls, and there's SELinux to block some other stuff, but it's still the actual root user without user namespaces, I think?)
        insanitybit 2mo ago
        That's right. Docker still runs without user namespaces by default, which means that root is the same user inside and outside of the container. This does open up attack surface and configuration footguns. Confinement still leverages dropping some root caps, seccomp, various other namespaces, etc.
        inigyou 2mo ago
- password4321 2mo ago
  > They are a security boundary My mistake, leaving out some adjective one could interpret as a misunderstanding of containers as an effective (etc.) security boundary. Fool me 100+ times and all that. There must be at least a triple-digit number of CVEs by now demonstratimg that in practice containers are a thinner layer of security (perhaps not quite as thin as the classic recommendation of running SSH on a nonstandard port, but that might be leaning toward the safer side of analogies vs. malicious code!) rather than a boundary like virtualization (not perfect but a best practice for isolation).
XorNot 2mo ago
There was a virtual machine KVM escape found like 2 weeks ago. Nothing is a security boundary anymore.

Runpod, digital ocean's gpu cloud, and at least a few others use Linux containers for isolation between tenants (look at Wiz's blog post about the nvidia container toolkit bug; digitalocean just puts everyone in a massive k8s cluster)

stingraycharles 2mo ago
Why aren’t they using a fast VM like Firecracker?
- himata4113 2mo ago
  To squeeze out 5% more profit.

circularfoyers 2mo ago

I know there's a lot you can do in k8s to mitigate it, but I didn't think that prevented it outright.

insanitybit 2mo ago

They are a security boundary. The fact that you need a vulnerability to escape them is proof of that. They just don't have a particularly high cost of escape because reachable kernel vulnerabilities are so common.

worthless-trash 2mo ago
Some people clearly do use containers as deployment mechanism, with security not in mind.
- zbentley 2mo ago
  s/some/most/ That's not meant to be snide, just true, I think.
  - CodesInChaos 2mo ago
    I never understood why kubernetes doesn't use a VM-per-pod model by default.
dijit 2mo ago
Escape from docker containers is trivially easy, if you are able to run as the root user in the container itself. Many (maybe most) containers actually default to running programs as root. Kernel exploit not required.
- maple3142 2mo ago
  If you are given a shell with `docker run -it --rm alpine:3 sh`, can you read the /etc/shadow on the host without kernel exploit? Assuming the docker and kernel are sufficiently update-to-date (e.g. latest Docker on Debian Stable).
  - chlorion 2mo ago
    No. The "root" you get in docker is not actually root outside of the namespace the container in running in. Assuming no bugs in the kernel, it should not be able to do anything more than the UID that it's mapped from.
    - ptx 2mo ago
      Does Docker use user namespaces by default? Otherwise root in the container is actually root on the host, from what I read. Correct me if I'm wrong. (Privileges are still limited by seccomp filters blocking some syscalls, and there's SELinux to block some other stuff, but it's still the actual root user without user namespaces, I think?)
      - insanitybit 2mo ago
        That's right. Docker still runs without user namespaces by default, which means that root is the same user inside and outside of the container. This does open up attack surface and configuration footguns. Confinement still leverages dropping some root caps, seccomp, various other namespaces, etc.
        inigyou 2mo ago
password4321 2mo ago
> They are a security boundary My mistake, leaving out some adjective one could interpret as a misunderstanding of containers as an effective (etc.) security boundary. Fool me 100+ times and all that. There must be at least a triple-digit number of CVEs by now demonstratimg that in practice containers are a thinner layer of security (perhaps not quite as thin as the classic recommendation of running SSH on a nonstandard port, but that might be leaning toward the safer side of analogies vs. malicious code!) rather than a boundary like virtualization (not perfect but a best practice for isolation).

XorNot 2mo ago

There was a virtual machine KVM escape found like 2 weeks ago. Nothing is a security boundary anymore.