6 ms·

This is really interesting. I built a similar product (not released yet), but it uses Kubernetes as the infrastructure layer instead of Cloudflare OS. I guess a

by tinco 1mo ago

This is really interesting. I built a similar product (not released yet), but it uses Kubernetes as the infrastructure layer instead of Cloudflare OS. I guess all these years later I am left with the same existential question that plagued Sandstorm. What is its relevance in the context of Linux containers? This is not a rhetorical question, is there a real benefit for a Sandstorm grain over a docker style Linux container? Does a containerized process not have all the same benefits of a Sandstorm grain, with the added benefit of not requiring any modifications of the containerized software? One argument I could bring is that despite all its claims, OpenAI still had to switch from containers to MicroVMs because its agents under test still managed to break out from their containers. Is the security model of a Sandstorm grain so much better that agents wouldn't figure out how to break out of it?

kentonv 1mo ago
Sandstorm's use of containers was just a means to an end. The real innovation was the fine-grained instances -- each document in its own container. No other container platform did that. But honestly, it didn't work well, because of cold start times and memory usage. It's bad enough when a server takes seconds to start, but if every document you open has a long startup time and uses hundreds of MB of RAM, it's really painful. Cloudflare OS doesn't use containers. It uses Dynamic Workers, which are 100x more efficient: https://blog.cloudflare.com/dynamic-workers/ https://blog.cloudflare.com/dynamic-workers/ So the stuff I have been building in Workers for the past 9 years turns out to be the thing that Sandstorm needed all along. What a coincidence. :)
- echelon 1mo ago
  This is far too Cloudflare flavored to be interesting to me. It's using Workers (capital W) and the core Cloudflare primitives. I don't feel safe building on this or touching this. I'd be happier if a startup or neutral party built this in a more agnostic way.
  - kentonv 1mo ago
    I'm sorry but Workers is the technology that makes this all possible. I don't think I could have built this without Workers as a foundation. (I tried once, with Sandstorm, and like I said, it didn't work well.) Sometimes you just can't advance the state of the art while also maintaining broad compatibility. It's all open source, though. You can run the whole stack on your own machine.
    - ocdtrekkie 1mo ago
      For what it's worth, we also ran workerd inside a Sandstorm grain and not only did it work, it performed well. In the same way some people run Docker to run a single thing inside a VM solely for that thing... you can probably use workerd just to run a single Workers-based thing inside something else... and it'll probably work just swell.
    - echelon 1mo ago
      Workers is a product and you're paid to develop and sell it as an employee. I 100% understand the hustle. I do the same thing. I just don't trust a giant like Cloudflare that has done several things to weaken the open web and establish a position that is all too powerful. I don't want to give them an iota of support. Even using open source buys into their mindshare, distribution, ecosystem, and eventual supremacy and lock-in. I'd buy this from a smaller company for sure. Just not from Google, AWS, Cloudflare. If this was a YC startup I'd have given you my credit card info already. I don't know what your stock in Cloudflare is like, but your upside would be way bigger building this externally. You'd probably raise an extremely large seed round. Consider launching your own startup. Don't give Cloudflare all the upside. Edit: I didn't realize you'd already responded. Was clarifying my position and wishes for this to not be something made by Cloudflare.
      - deleted 1mo ago
        [deleted]
      - kentonv 1mo ago
        I tried it as a startup once, so I know what that's like. It's a lot of time spent running around begging for money (from investors and from customers) rather than building technology. Workers is my startup-within-Cloudflare. It won't make me a billionaire, but it has still made me more money than I actually know what to do with, while being able to delegate all the stuff I don't like doing to other parts of the company that already do it quite well. I have a lot of influence here. The CEO and CTO listen to me. E.g. I made my argument this all needed to be open source and self hostable, and they agreed enthusiastically. I don't think I could build this better as a separate company.
- valorzard 1mo ago
  dumb question: is this javascript/typescript only? can you use webassembly based languages instead?
  - kentonv 1mo ago
    In theory Wasm could be supported, but generally JS/TS is much more efficient when running inside isolates (our lightweight sandboxes) since you don't have to bundle a language runtime into the app. CFOS at present only exposes JS (with TS coming soon).
    - spankalee 1mo ago
      Please add Wasm support :) I'm making a whole new language to get around the problem you're talking about: it brings no runtime at all. One of my targets is a Sandstorm like system I've been slowly working on, but I'd love for it to be a fit for the actual Sandstorm successor.
    - nullpoint420 1mo ago
      Mmm so application-level sandboxing. Lovely. And I’m assuming your worker runtime is a process in a container on a shared node? What happens if the agent exploits your runtime? Does it get access to the whole container? VM? Node? Why should I ever choose this over MicroVMs? I have to design my architecture around your JS runtime. This isn’t an OS.
      - teacpde 1mo ago
        Workers are in the same process but have separate heap, global vars, gc etc, the isolation is managed and achieved by V8. It is much lightweight compared to micro VM or containers. But the downside is you have to use JS because of V8.
        nullpoint420 1mo ago
        I get that but it’s much less secure. ie. If an agent finds a bug in V8 it’s over
oooyay 1mo ago
Linux namespaces, and containers, are not security features in themselves. They end up having to be combined with SECCOMP and some sort of application kernel or SELinux in order to have an effective security apparatus. This is before you give it application aware security controls like policy.
abdullahkhalids 1mo ago
> is there a real benefit for a Sandstorm grain over a docker style Linux container? Linux containers are meant to be used by those with significant software engineering skills. Sandstorm was designed to be used, once installed by someone else, by grandma.
everforward 1mo ago
I don’t think containers, as normally used, provide that much security because so much of what you want to secure is at the boundaries between containers (the network) rather than inside the container itself. There’s not much point in preventing hyper visor escape style exploits if the agent can just SSH or psql log in to an adjacent container. Likewise there are near infinite ways to do weird things with the network to make stuff happen in another less controlled environment (eg sign up for AWS free account, make VM, do everything on that VM where controls don’t apply). There are tools to do this, but I’ve never seen anyone package them up in a way that was pleasant to interact with. It’s got the SELinux problem of being useful, but annoying enough that everyone just disables it the second it gets in the way. It’s surprisingly hard to manage what should be able to connect to what as you start scaling out the number of things on the network. Whoops, you forgot that random software embeds its own DNS over HTTPS resolver so the normal DNS profile won’t work and everything crashed kind of issues.
nijave 1mo ago
It has real business value. Letting non technical users run wild without the onerous layers of controls in traditional enterprise IT. We're already contending with users wanting to hook up every SaaS MCP to every other SaaS platform and then slap AI on top. Having a controlled sandbox for that would hugely simplify things.

by tinco 1mo ago

kentonv 1mo ago

Sandstorm's use of containers was just a means to an end. The real innovation was the fine-grained instances -- each document in its own container. No other container platform did that. But honestly, it didn't work well, because of cold start times and memory usage. It's bad enough when a server takes seconds to start, but if every document you open has a long startup time and uses hundreds of MB of RAM, it's really painful. Cloudflare OS doesn't use containers. It uses Dynamic Workers, which are 100x more efficient: https://blog.cloudflare.com/dynamic-workers/ https://blog.cloudflare.com/dynamic-workers/ So the stuff I have been building in Workers for the past 9 years turns out to be the thing that Sandstorm needed all along. What a coincidence. :)

echelon 1mo ago
This is far too Cloudflare flavored to be interesting to me. It's using Workers (capital W) and the core Cloudflare primitives. I don't feel safe building on this or touching this. I'd be happier if a startup or neutral party built this in a more agnostic way.
- kentonv 1mo ago
  I'm sorry but Workers is the technology that makes this all possible. I don't think I could have built this without Workers as a foundation. (I tried once, with Sandstorm, and like I said, it didn't work well.) Sometimes you just can't advance the state of the art while also maintaining broad compatibility. It's all open source, though. You can run the whole stack on your own machine.
  - ocdtrekkie 1mo ago
    For what it's worth, we also ran workerd inside a Sandstorm grain and not only did it work, it performed well. In the same way some people run Docker to run a single thing inside a VM solely for that thing... you can probably use workerd just to run a single Workers-based thing inside something else... and it'll probably work just swell.
  - echelon 1mo ago
    Workers is a product and you're paid to develop and sell it as an employee. I 100% understand the hustle. I do the same thing. I just don't trust a giant like Cloudflare that has done several things to weaken the open web and establish a position that is all too powerful. I don't want to give them an iota of support. Even using open source buys into their mindshare, distribution, ecosystem, and eventual supremacy and lock-in. I'd buy this from a smaller company for sure. Just not from Google, AWS, Cloudflare. If this was a YC startup I'd have given you my credit card info already. I don't know what your stock in Cloudflare is like, but your upside would be way bigger building this externally. You'd probably raise an extremely large seed round. Consider launching your own startup. Don't give Cloudflare all the upside. Edit: I didn't realize you'd already responded. Was clarifying my position and wishes for this to not be something made by Cloudflare.
    - deleted 1mo ago
      [deleted]
    - kentonv 1mo ago
      I tried it as a startup once, so I know what that's like. It's a lot of time spent running around begging for money (from investors and from customers) rather than building technology. Workers is my startup-within-Cloudflare. It won't make me a billionaire, but it has still made me more money than I actually know what to do with, while being able to delegate all the stuff I don't like doing to other parts of the company that already do it quite well. I have a lot of influence here. The CEO and CTO listen to me. E.g. I made my argument this all needed to be open source and self hostable, and they agreed enthusiastically. I don't think I could build this better as a separate company.
valorzard 1mo ago
dumb question: is this javascript/typescript only? can you use webassembly based languages instead?
- kentonv 1mo ago
  In theory Wasm could be supported, but generally JS/TS is much more efficient when running inside isolates (our lightweight sandboxes) since you don't have to bundle a language runtime into the app. CFOS at present only exposes JS (with TS coming soon).
  - spankalee 1mo ago
    Please add Wasm support :) I'm making a whole new language to get around the problem you're talking about: it brings no runtime at all. One of my targets is a Sandstorm like system I've been slowly working on, but I'd love for it to be a fit for the actual Sandstorm successor.
  - nullpoint420 1mo ago
    Mmm so application-level sandboxing. Lovely. And I’m assuming your worker runtime is a process in a container on a shared node? What happens if the agent exploits your runtime? Does it get access to the whole container? VM? Node? Why should I ever choose this over MicroVMs? I have to design my architecture around your JS runtime. This isn’t an OS.
    - teacpde 1mo ago
      Workers are in the same process but have separate heap, global vars, gc etc, the isolation is managed and achieved by V8. It is much lightweight compared to micro VM or containers. But the downside is you have to use JS because of V8.
      - nullpoint420 1mo ago
        I get that but it’s much less secure. ie. If an agent finds a bug in V8 it’s over

oooyay 1mo ago

Linux namespaces, and containers, are not security features in themselves. They end up having to be combined with SECCOMP and some sort of application kernel or SELinux in order to have an effective security apparatus. This is before you give it application aware security controls like policy.

abdullahkhalids 1mo ago

> is there a real benefit for a Sandstorm grain over a docker style Linux container? Linux containers are meant to be used by those with significant software engineering skills. Sandstorm was designed to be used, once installed by someone else, by grandma.

everforward 1mo ago

I don’t think containers, as normally used, provide that much security because so much of what you want to secure is at the boundaries between containers (the network) rather than inside the container itself. There’s not much point in preventing hyper visor escape style exploits if the agent can just SSH or psql log in to an adjacent container. Likewise there are near infinite ways to do weird things with the network to make stuff happen in another less controlled environment (eg sign up for AWS free account, make VM, do everything on that VM where controls don’t apply). There are tools to do this, but I’ve never seen anyone package them up in a way that was pleasant to interact with. It’s got the SELinux problem of being useful, but annoying enough that everyone just disables it the second it gets in the way. It’s surprisingly hard to manage what should be able to connect to what as you start scaling out the number of things on the network. Whoops, you forgot that random software embeds its own DNS over HTTPS resolver so the normal DNS profile won’t work and everything crashed kind of issues.

nijave 1mo ago

It has real business value. Letting non technical users run wild without the onerous layers of controls in traditional enterprise IT. We're already contending with users wanting to hook up every SaaS MCP to every other SaaS platform and then slap AI on top. Having a controlled sandbox for that would hugely simplify things.