6 ms·

The spectrum of isolation: From bare metal to WebAssembly

97 points by ThierryBuilds 8mo ago

ThierryBuilds 8mo ago
I wrote this because I kept seeing developers (myself included) confuse language-level isolation like Python venv with OS-level isolation like Docker. I wanted to trace the actual technical boundaries between them. The article maps out the differences between common execution environments—from physical bare metal and VMs to containers, process sandboxes, and virtual environments—to create a mental model of where the "isolation boundary" actually sits for each tool.
- ianand 8mo ago
  Since you mention serverless it might be worth mentioning firecracker and v8 isolates.
  - ThierryBuilds 8mo ago
    Thank you for the feedback. I will definitely add them as example solutions for serverless.
  - pjmlp 8mo ago
    Or CGIs running on httpd inside HP-UX Vaults, that is how old the idea happens to be.
    - ignoramous 8mo ago
      > how old the idea happens to be TFA is missing a host of many a popular isolation techniques like Isolates, Code Interp / Binary Translators [0], Enclaves, Exclaves, Domains/Worlds, (RISC V) SEEs, TEEs, SEs, HSMs, pKVMs ... [0] https://news.ycombinator.com/item?id=38950949 https://news.ycombinator.com/item?id=38950949
- lateral_cloud 8mo ago
  Did you really write it though? Within the first paragraph it's fairly obvious this is heavily LLM-generated.
  - aragilar 8mo ago
    It also has weird definitions. Is nix a virtual environment? Is homebrew a virtual environment? Why is a sandbox different to a container? Type-1 vs Type-2 hypervisors are quite different, and there's no discussion about processes vs threads.
  - tadfisher 8mo ago
    I don't know what it is about LLM-generated text, but when I read it I cannot understand the meaning it is trying to convey. The words are all there, but it is fatiguing to repeatedly parse phrasing like "it's not X but Y" and "you aren't just X, you are Y". The entire article is organized as a sequence of these statements, and this is not hyperbole.
    - zipy124 8mo ago
      Because it is statistical. It has no understanding of the purpose of writing which is to convey information. It can only show you the statistically most likely text, although very good sometimes, it also has its limitations.
- fuzzfactor 8mo ago
  >1. Physical Machine (Bare Metal) This is the foundation. Nobody should ever forget this. But I would say this next part is about the opposite for bare metal though: >Use Case: High-performance computing (HPC), large databases, or legacy systems that require direct hardware access. To get the utmost reliability out of adequate hardware then bare metal is more suitable for almost everything except for special situations. Unless something is really wrong with the software or the overall hardware/software approach.
  - ThierryBuilds 8mo ago
    Thanks for the feedback. These are typical use cases where the convenience of higher level abstractions may be less important than the benefits of direct access to the hardware.
    - deleted 8mo ago
      [deleted]
shevy-java 8mo ago
WebAssembly somehow does not seem to be able to break-through, unlike HTML, CSS, JavaScript did.
- mickael-kerjean 8mo ago
  Or the people who write wasm don't talk too much about it. My OSS work (https://github.com/mickael-kerjean/filestash https://github.com/mickael-kerjean/filestash) has tons of it: 1. to create web versions of applications that are traditionally desktop only to render things like Parquet, PSD, TIFF, SQLite, EPS, ZIP, TGZ, and many more, where C libraries are often the reference implementations. There are almost a hundred supported file formats, most of which are supported through WASM 2. to create plugins that extend the backend and add your own endpoint or middleware as a way to enforce the code run in a constrained environment without the ability to send people's file out 3. in the workflow engine to enable people to run their own sandboxed scripts without giving those a blank check to go crazy
- thecupisblue 8mo ago
  It is more of a silent thing. Running in the background, internal libs, deployment tools, plugin tools. But also - it's lacking things like a unified positioning + required knowledge to understand it is quite large compared to average dev + most people have no real use for it. It's mostly too "abstract high level" and "low level" for most devs.
bflesch 8mo ago
> This website collects anonymous usage analytics data via GoatCounter and Umami. My uBlock origin shows that googlefonts.com and fonts.googleapis.com are being blocked. It irks me a bit that your message explicitly mentions two trackers but it fails to mention the Google tracking. Google is also not mentioned in your privacy policy. Is there a reason for this?
- nake89 8mo ago
  Your message sent me down a weird rabbit hole of trying to find privacy friendly alternative to google fonts. I found this: https://github.com/coollabsio/fonts https://github.com/coollabsio/fonts They claim to be a privacy friendly drop-in replacement. Their main website: https://fonts.coollabs.io/ https://fonts.coollabs.io/
  - cardanome 8mo ago
    The easiest solution is to simply self host your fonts.
    - mrob 8mo ago
      The easiest solution is to use the default font. This has the additional benefit of being the most legible font for every reader, because it's the one they have the most experience reading.
    - Piraty 8mo ago
      remember the times when common sense was to not accept the remote site's fonts, and thus web devs should not use them
      - bflesch 8mo ago
        Yes. I recommend everybody to do a deep dive into font file formats and you'll see a lot of monsters hiding.
    - yencabulator 8mo ago
bmitch3020 8mo ago
The spectrum comes with multiple tradeoffs, and isn't a simple "bare metal is more secure" narrative. Because as you move into VMs, containers, and code sandboxes, you lose isolation which increases risks, but you also gain capabilities to limit the application which decreases risk. So I believe the most secure approach is layered with much multiple types of isolation working together. For example, you may isolate a specific customer to bare metal so an escape doesn't compromise other customers. But within that bare metal, you may run containers because they make it easier to work with a read only root filesystem that's also trivial to upgrade. You can also add on user namespaces and seccomp in the container to minimize the risk of a container escape. And then the application may have its own sandbox that limits individual capabilities and which API calls it can run. Every use case is different, and some layers may not be available depending on that use case. But rather than picking one point on the spectrum, one should pick a list of technologies that best solve each use case.
- ThierryBuilds 8mo ago
  Agreed. Beyond security, we must consider other critical factors such as cost, ease of maintenance, and operational overhead.
- Loic 8mo ago
  This is the engineering approach in a factory. You always have multiple layers of security systems. The analogy is that each layer is a slice of Emmental cheese. You end up with a bad event, only if all the holes in the slices align.
creata 8mo ago
venv and sandboxes are such categorically different things that painting it as a spectrum the way this article does is more misleading than helpful. I also think the article shouldn't mention chroot. From the man page: > In particular, it is not intended to be used for any kind of security purpose, I guess it could be part of a sandbox, but there are better tools for that purpose. (I'm not sure what point there is in giving feedback on an article that's almost entirely LLM-generated, though.)
- ThierryBuilds 8mo ago
  The article is not only about security. Thanks
FjordWarden 8mo ago
Ah, I think I found the reason as to why WebAssembly (in a browser or some other sandboxed environment) is not a suitable substrate for near native performance. It is a very ironic reason: you can't implement a JIT compiler that targets WebAssembly in a sandbox running in WebAssembly. Sounds like an incredibly contrived thing to do but once speed is the goal then a copy-and-patch compiler is a valid strategy for implementing a interpreter or a modern graphics pipeline.
- syrusakbary 8mo ago
  > you can't implement a JIT compiler that targets WebAssembly in a sandbox running in WebAssembly That's not completely true. With dynamic linking (now supported in WASIX), you can generate and link Wasm modules at runtime easily.
- fwsgonzo 8mo ago
  This is true. A multi-tier JIT-compiler requires writable execute memory and the ability to flush icache. Loading segments dynamically is nice and covers a lot of the ground, but it won't be a magic solution to dynamic languages like JavaScript. Modern WASM emulators already implement a full compiler, linker and JIT-compiler in one, almost starting to look like v8. I'm not sure if adding in-guest JIT support is going in the right direction.
nickdothutton 8mo ago
No LPARs (IBM) or LDoms (Oracle), although I appreciate someone might never have to encounter those things these days. They sit above bare metal and below hypervisor VMs.

97 points by ThierryBuilds 8mo ago

I wrote this because I kept seeing developers (myself included) confuse language-level isolation like Python venv with OS-level isolation like Docker. I wanted to trace the actual technical boundaries between them. The article maps out the differences between common execution environments—from physical bare metal and VMs to containers, process sandboxes, and virtual environments—to create a mental model of where the "isolation boundary" actually sits for each tool.

ianand 8mo ago
Since you mention serverless it might be worth mentioning firecracker and v8 isolates.
- ThierryBuilds 8mo ago
  Thank you for the feedback. I will definitely add them as example solutions for serverless.
- pjmlp 8mo ago
  Or CGIs running on httpd inside HP-UX Vaults, that is how old the idea happens to be.
  - ignoramous 8mo ago
    > how old the idea happens to be TFA is missing a host of many a popular isolation techniques like Isolates, Code Interp / Binary Translators [0], Enclaves, Exclaves, Domains/Worlds, (RISC V) SEEs, TEEs, SEs, HSMs, pKVMs ... [0] https://news.ycombinator.com/item?id=38950949 https://news.ycombinator.com/item?id=38950949
lateral_cloud 8mo ago
Did you really write it though? Within the first paragraph it's fairly obvious this is heavily LLM-generated.
- aragilar 8mo ago
  It also has weird definitions. Is nix a virtual environment? Is homebrew a virtual environment? Why is a sandbox different to a container? Type-1 vs Type-2 hypervisors are quite different, and there's no discussion about processes vs threads.
- tadfisher 8mo ago
  I don't know what it is about LLM-generated text, but when I read it I cannot understand the meaning it is trying to convey. The words are all there, but it is fatiguing to repeatedly parse phrasing like "it's not X but Y" and "you aren't just X, you are Y". The entire article is organized as a sequence of these statements, and this is not hyperbole.
  - zipy124 8mo ago
    Because it is statistical. It has no understanding of the purpose of writing which is to convey information. It can only show you the statistically most likely text, although very good sometimes, it also has its limitations.
fuzzfactor 8mo ago
>1. Physical Machine (Bare Metal) This is the foundation. Nobody should ever forget this. But I would say this next part is about the opposite for bare metal though: >Use Case: High-performance computing (HPC), large databases, or legacy systems that require direct hardware access. To get the utmost reliability out of adequate hardware then bare metal is more suitable for almost everything except for special situations. Unless something is really wrong with the software or the overall hardware/software approach.
- ThierryBuilds 8mo ago
  Thanks for the feedback. These are typical use cases where the convenience of higher level abstractions may be less important than the benefits of direct access to the hardware.
  - deleted 8mo ago
    [deleted]

shevy-java 8mo ago

WebAssembly somehow does not seem to be able to break-through, unlike HTML, CSS, JavaScript did.

mickael-kerjean 8mo ago
Or the people who write wasm don't talk too much about it. My OSS work (https://github.com/mickael-kerjean/filestash https://github.com/mickael-kerjean/filestash) has tons of it: 1. to create web versions of applications that are traditionally desktop only to render things like Parquet, PSD, TIFF, SQLite, EPS, ZIP, TGZ, and many more, where C libraries are often the reference implementations. There are almost a hundred supported file formats, most of which are supported through WASM 2. to create plugins that extend the backend and add your own endpoint or middleware as a way to enforce the code run in a constrained environment without the ability to send people's file out 3. in the workflow engine to enable people to run their own sandboxed scripts without giving those a blank check to go crazy
thecupisblue 8mo ago
It is more of a silent thing. Running in the background, internal libs, deployment tools, plugin tools. But also - it's lacking things like a unified positioning + required knowledge to understand it is quite large compared to average dev + most people have no real use for it. It's mostly too "abstract high level" and "low level" for most devs.

bflesch 8mo ago

> This website collects anonymous usage analytics data via GoatCounter and Umami. My uBlock origin shows that googlefonts.com and fonts.googleapis.com are being blocked. It irks me a bit that your message explicitly mentions two trackers but it fails to mention the Google tracking. Google is also not mentioned in your privacy policy. Is there a reason for this?

nake89 8mo ago
Your message sent me down a weird rabbit hole of trying to find privacy friendly alternative to google fonts. I found this: https://github.com/coollabsio/fonts https://github.com/coollabsio/fonts They claim to be a privacy friendly drop-in replacement. Their main website: https://fonts.coollabs.io/ https://fonts.coollabs.io/
- cardanome 8mo ago
  The easiest solution is to simply self host your fonts.
  - mrob 8mo ago
    The easiest solution is to use the default font. This has the additional benefit of being the most legible font for every reader, because it's the one they have the most experience reading.
  - Piraty 8mo ago
    remember the times when common sense was to not accept the remote site's fonts, and thus web devs should not use them
    - bflesch 8mo ago
      Yes. I recommend everybody to do a deep dive into font file formats and you'll see a lot of monsters hiding.
  - yencabulator 8mo ago

bmitch3020 8mo ago

The spectrum comes with multiple tradeoffs, and isn't a simple "bare metal is more secure" narrative. Because as you move into VMs, containers, and code sandboxes, you lose isolation which increases risks, but you also gain capabilities to limit the application which decreases risk. So I believe the most secure approach is layered with much multiple types of isolation working together. For example, you may isolate a specific customer to bare metal so an escape doesn't compromise other customers. But within that bare metal, you may run containers because they make it easier to work with a read only root filesystem that's also trivial to upgrade. You can also add on user namespaces and seccomp in the container to minimize the risk of a container escape. And then the application may have its own sandbox that limits individual capabilities and which API calls it can run. Every use case is different, and some layers may not be available depending on that use case. But rather than picking one point on the spectrum, one should pick a list of technologies that best solve each use case.

ThierryBuilds 8mo ago
Agreed. Beyond security, we must consider other critical factors such as cost, ease of maintenance, and operational overhead.
Loic 8mo ago
This is the engineering approach in a factory. You always have multiple layers of security systems. The analogy is that each layer is a slice of Emmental cheese. You end up with a bad event, only if all the holes in the slices align.

creata 8mo ago

venv and sandboxes are such categorically different things that painting it as a spectrum the way this article does is more misleading than helpful. I also think the article shouldn't mention chroot. From the man page: > In particular, it is not intended to be used for any kind of security purpose, I guess it could be part of a sandbox, but there are better tools for that purpose. (I'm not sure what point there is in giving feedback on an article that's almost entirely LLM-generated, though.)

ThierryBuilds 8mo ago
The article is not only about security. Thanks

FjordWarden 8mo ago

Ah, I think I found the reason as to why WebAssembly (in a browser or some other sandboxed environment) is not a suitable substrate for near native performance. It is a very ironic reason: you can't implement a JIT compiler that targets WebAssembly in a sandbox running in WebAssembly. Sounds like an incredibly contrived thing to do but once speed is the goal then a copy-and-patch compiler is a valid strategy for implementing a interpreter or a modern graphics pipeline.

syrusakbary 8mo ago
> you can't implement a JIT compiler that targets WebAssembly in a sandbox running in WebAssembly That's not completely true. With dynamic linking (now supported in WASIX), you can generate and link Wasm modules at runtime easily.
fwsgonzo 8mo ago
This is true. A multi-tier JIT-compiler requires writable execute memory and the ability to flush icache. Loading segments dynamically is nice and covers a lot of the ground, but it won't be a magic solution to dynamic languages like JavaScript. Modern WASM emulators already implement a full compiler, linker and JIT-compiler in one, almost starting to look like v8. I'm not sure if adding in-guest JIT support is going in the right direction.

nickdothutton 8mo ago

No LPARs (IBM) or LDoms (Oracle), although I appreciate someone might never have to encounter those things these days. They sit above bare metal and below hypervisor VMs.