9 ms·

> Wasm sandboxing is even safe to run in the same process as other code (at least modulo Spectre-type vulnerabilities)... If you want strong security with this

> Wasm sandboxing is even safe to run in the same process as other code (at least modulo Spectre-type vulnerabilities)... If you want strong security with this big a modulus you may have been looking for RSA. (ugh, sorry) Spectre V1 (speculative bounds check bypass / type confusion) is basically game over for intra-process memory isolation without introducing expensive or complicated mitigations: speculation barriers at every branch, or a more optimized pass like Speculative Load Hardening (https://llvm.org/docs/SpeculativeLoadHardening.html https://llvm.org/docs/SpeculativeLoadHardening.html) that can bring the overhead _down_ to 20-50%. From what I've seen, the best understanding right now is that the process is the smallest defensible unit of isolation. That's not to say that using WASM as an intermediate compilation target won't eliminate some threats! The runtime can definitely help to improve memory safety, for example. But if you care about confidentiality of the data in your process, any code that is untrusted enough to require sandboxing should also be run out-of-process, e.g. across an IPC boundary.

monadic2 6y ago
I’m open to a slow but secure processor. Certainly for my personal computing.
- pjmlp 6y ago
  I feel the same about bounds checking in programming languages, apparently not everyone does.
  - KMag 6y ago
    I vaguely remember a story about a hardware manufacturer (Burroughs? Symbolics?) that had a processor with instructions that could bounds-check array accesses with zero latency overhead. It was a great feature for Algol/Lisp, but some customers asked for an option in their Fortran compiler to disable the bounds check. The sales engineer replied that the bounds checks were zero-cost, but the customers replied that the bounds checks broke their progrems... their programs had silent (or at least unnoticed) array bounds bugs, and the customers preferred to keep those bugs, thank you very much! I wish I had kept the link. I've tired a couple times to find the story. Does this ring any bells for anyone?
    - pjmlp 6y ago
      Burroughs definitely had bounds checking. https://en.wikipedia.org/wiki/Burroughs_large_systems https://en.wikipedia.org/wiki/Burroughs_large_systems Its system programming language (initially ESPOL then NEWP), also has support for explicit unsafe code blocks, and there is zero Assembly support. All CPU low level operations are exposed via intrisics. All of this in 1961, almost 10 years before C was invented. Still being sold nowadays, and naturally Unisys uses security as one of the selling features. https://www.unisys.com/offerings/clearpath-forward/clearpath-forward-products/clearpath-mcp-software https://www.unisys.com/offerings/clearpath-forward/clearpath... Regarding bounds checking, what I keep around is Hoare's Turing award speech. "Many years later we asked our customers whether they wished us to provide an option to switch off these checks in the interests of efficiency on production runs. Unanimously, they urged us not to--they already knew how frequently subscript errors occur on production runs where failure to detect them could be disastrous. I note with fear and horror that even in 1980, language designers and users have not learned this lesson. In any respectable branch of engineering, failure to observe such elementary precautions would have long been against the law."
      - monadic2 6y ago
        I used to agree heavily with this, but nowadays we have effective techniques of memory safety on the software level and I find myself ambivalent on which level of abstraction this occurs. I do think it means C is probably eventually doomed as a system language, though. I’m curious if we’ll ever see e.g. rust in the *bsd codebases.
        pjmlp 6y ago
        Except we don't, that is why ARM, Apple, Microsoft, Google, Oracle are all pursuing variants of hardware memory tagging for taming C, as those software solutions have proven not to work.
        monadic2 6y ago
        Sure, but hardware memory tagging is also not proven to work. Anyway, it's unclear with which criteria you're judging "proven not to work" with as we're both typing via the software right now, unlike a hardware solution.
- mehrdadn 6y ago
  Try underclocking your CPU to like 30% of its max speed and see if your programs still run comfortably. This would've made sense in a world where programs didn't become more bloated and slower over time, but it would have visible repercussions given our software today.
  - pjmlp 6y ago
    That would be positive though, maybe then not so many people would be running Python sites on Django or Electron apps.
    - andybak 6y ago
      It sounds like you want to actively discourage the use of Python and Django irrespective of performance concerns? Care to elaborate or was it just flippant snide?
      - pjmlp 6y ago
        More like discourage the use of pure scripting without any regard for JIT/AOT toolchains or coding without performance considerations, in opposition to what we used to care about. The example with Python and Django was what came quickest to mind, but I can gladly expand it to include Ruby and Rails, or any other stack that falls under the same Web sites/desktop applications with scripting languages umbrella.
        andybak 6y ago
        The usual retort is "the scripting language is rarely the bottleneck" as well as "developer time is more expensive than hardware". Between these two get-out clauses, aren't you talking about a very small minority of use-cases? Those where the scripting language is the bottleneck and the problem can't simply be solved by spending a few $ more on your VPS?
        pjmlp 6y ago
zozbot234 6y ago
Spectre vulnerabilities only matter wrt. information disclosure. Really we need OS's that are explicitly aware of information domains, and only flush address space mappings when switching from a more privileged to a less privileged domain.
- mdriley 6y ago
  so, some things: 1. Information disclosure is pretty important, especially if your process has AWS credentials in the environment block or it's doing mTLS with a long-lived key. 2. Those operating systems already exist, see e.g. https://twitter.com/aionescu/status/948818841747955713 https://twitter.com/aionescu/status/948818841747955713 3. Spectre V1 is within the same process, so this isn't a question of address mappings across differently-privileged domains. It's the same domain (i.e. address space). 4. Flushing address spaces across privilege domains isn't a concern on modern processors thanks to tagged TLBs and process-context or address space identifiers (PCID, ASID)
syrusakbary 6y ago
There are some interesting approaches for running Wasm code confidentially. I'd recommend to take a look into Enarx, as they are pioneering the space with SEV/SGX integration into Wasm workloads. https://opensource.com/article/19/5/enarx-security https://opensource.com/article/19/5/enarx-security
- mdriley 6y ago
  SGX is a tool in the toolbox, but it solves a different problem: isolating a small section of especially privileged code from the rest of a larger, less-trusted application. The sandbox described in the article is trying to do roughly the opposite: protect the main application from an isolated section of untrusted code. Also, SGX requires extreme care in deployment due to side-channel attacks, see e.g. https://software.intel.com/security-software-guidance/insights/deep-dive-load-value-injection https://software.intel.com/security-software-guidance/insigh... SEV is also interesting, but requires code to run in a separate VM -- which satisfies my requirement above that it at least be in a different process.
reitzensteinm 6y ago
If you had control over the ISA, and were jitting code as you would with WASM, couldn't you defeat Spectre 1 with an instruction that essentially introduces a masked region that the processor will not read or speculate outside of until the mode is disabled? If the attacker controls the instruction stream it's game over (since they can disable the mask), but if they're just writing wasm that generates malicious speculative in process loads, it would prevent that. Then an attack would have to trick library calls in to doing the same job, which is certainly possible, but much harder. Of course this would negate many of the benefits of staying in the same process, so I'm not necessarily saying it's a good idea.
- mdriley 6y ago
  When you say "control over the ISA", I'll assume you mean "precise control over the emitted instructions". In which case: yes! That's Speculative Load Hardening (https://llvm.org/docs/SpeculativeLoadHardening.html https://llvm.org/docs/SpeculativeLoadHardening.html). SLH tries to squash side-channels by preventing any speculatively-loaded data from being forwarded to dependent instructions until proving that branch prediction followed the right path. But this undoes a lot of the performance that microarchitectures have added through branch prediction, since dependent memory loads (think linked list entries, or C++ vtables) are stalled behind full resolution of the branch condition. If you're doing nontrivial compute, you can end up ahead performance-wise by splitting the computation into a separate process and invoking it via IPC. Now you don't need SLH because the untrusted process doesn't have long-lived secrets in its address space.
  - reitzensteinm 6y ago
    No, I mean specifically if you're building your own CPUs and can add instructions. You add the ability to set a hardware mask that all values are passed through before they're used as addresses for loads and stores including speculation. Loads and stores that fall outside the masked region will simply wrap around. In your JIT, you enter and leave this mode before and after running user code to ensure it can't escape its region. This would be a lot of work to pull off and would require custom hardware and software, but (at least as far as I can tell) it should work.
KMag 6y ago
Spectre makes me wish we'd give something like IBM's project DAISY / Transmeta's Crusoe another chance. If we have one mode with a simple and high-density instruction set similar to Hitachi's SH4 / ARM's Thumb2, along with a VLIW mode (similar to switching between Thumb2 and regular ARM), and some hardware support for tracing and dynamic recompilation (including reservoir sampling of instructions causing pipeline stalls), we might get decent performance and code density, while being able to move all of the speculation into the dynamic recompilation layer. Having the processor natively support the dense non-VLIW instruction set means you don't pay much in the way of startup latency, and once you're warmed up, your hot spots have all of their cross-DLL calls inlined / no longer indirect, and your virtual function calls devirtualized / speculative inlined. Spots where the dynamic recompiler was wrong about pipeline stalls eventually use dynamic information to recompile and shuffle instructions around to avoid pipeline stalls. You don't pay the transistor and power budget for out-of-order execution or speculative execution hardware. Hardware speculative execution gets replaced with predicated instructions and shadow register save/restores. I think IBM's DAISY was much closer to being on the right track vs. Intel's EPIC / Itanium. If your system is built for dynamic recompilation and re-optimization of native code on the fly, your compiler doesn't have to be as good at statically predicting execution paths and pipeline stalls. Transmeta's main problem was that they were emulating all x86 instructions before they warmed up, and after warm-up they were still emulating all x86 instructions outside their hot code path. With a simpler instruction set like SH4 / Thumb2, they could hopefully have hardware support for the code outside of the hot spots / pre-warmup. Given the amount of time spent running JavaScript, hopefully one would also look at the bytecodes for V8, JavaScriptCore, and SpiderMonkey for inspiration as far as making the non-VLIW instruction set an efficient and compact JavaScript JIT target.
- titzer 6y ago
  > If we have one mode with a simple and high-density instruction set similar to Hitachi's SH4 / ARM's Thumb2, along with a VLIW mode (similar to switching between Thumb2 and regular ARM), and some hardware support for tracing and dynamic recompilation (including reservoir sampling of instructions causing pipeline stalls), we might get decent performance and code density, while being able to move all of the speculation into the dynamic recompilation layer. Trading hardware complexity for software complexity is not the right strategy to combat sidechannels. They just reappear up the stack. Any speculative optimization that depends on program values can potentially leak information through timing.
rubber_duck 6y ago
>From what I've seen, the best understanding right now is that the process is the smallest defensible unit of isolation. In what scenario would you be more comfortable running a potentially malicious process versus a potentially malicious WASM code behind a API sandbox where you control the exposed APIs. Sure there are process level sandboxes but that seems like a much larger bug surface area with expertise outside of the domain of people building the apps, compared to a WASM sandbox with a host app defined bridge API.
- the8472 6y ago
  > Sure there are process level sandboxes but that seems like a much larger bug surface area A process sandbox allows you to pass through syscalls you deem harmless without reimplementing them and only building the bridge API for things that require complex policies. With wasm you need to implement wrappers for everything. Let's say your wasm module is pure compute and does IPC via shared memory locks. You now at least have to implement the locking API. In the process sandbox case you only whitelist the futex syscall and that's it. And in practice you'll have to use the process sandbox anyway due to CPU vulnerabilities or bugs in the bridge API.
  - rubber_duck 6y ago
    >A process sandbox allows you to pass through syscalls you deem harmless without reimplementing them and only building the bridge API for things that require complex policies. With wasm you need to implement wrappers for everything. Yeah except you have to know what OS your application is going to run on, kernel version, set up the sandbox correctly and trust that the kernel doesn't have any privilege escalation bugs - likely none of which is your area of expertise as an application developer. A sandbox runtime with exposed APIs is much closer to your domain and you can model it to be much more domain specific so the surface you need to wrap should be smaller.
    - the8472 6y ago
      I think the surface mostly depends on how much you want to give the sandboxed application, not on the sandboxing technology you choose. If want to build something emulating the capabilities of a desktop operating system including gfx acceleration (e.g. webgpu) then it will be much larger than some isolated compute function in the fashion of network edge workers. I am not aware of a single CVE against openbsd's pledge. On the other hand there are plenty of javascript engine escapes.
voidmain 6y ago
> Spectre V1 (speculative bounds check bypass / type confusion) is basically game over for intra-process memory isolation without introducing expensive or complicated mitigations Code which is executed deterministically cannot receive timing channels (or, indeed, learn anything about its environment) and hence cannot exploit Spectre. This seems potentially practical for this sort of library sandboxing problem.
- azakai 6y ago
  Exactly, by default code sandboxed by wasm is fully deterministic and can't do any timing measurements, not unless you explicitly give it access to an import that does such a measurement. On the web, a website might run arbitrary wasm + JS which means it might let wasm time things. But if you use wasm to sandbox a specific library then the situation is different and you control the wasm imports.
  - saagarjha 6y ago
    > by default code sandboxed by wasm is fully deterministic Sorry? I'm not sure what you mean by "fully deterministic" here, because as far as I was aware you can do basically anything including choosing to not terminate inside of WASM.
    - azakai 6y ago
      You can have an infinite loop, sure, but aside from that wasm semantics are precisely defined in a deterministic way (well, except for minor issues with float NaN bits). That is, if a computation terminates, it will always terminate and with the same results. That's the case because wasm itself has no way to tell the time, generate a random number, etc., and each operation's semantics are well-defined. (If infinite loops are a concern, you can do what wasm VMs do on the web which is to show a "stop script?" dialog after too much time passes.)
      - monocasa 6y ago
        WASM is not deterministic from a timing perspective.
unoti 6y ago
This is interesting. If data confidentiality requires process isolation, then how can CloudFlare workers be safe?
- kentonv 6y ago
  There are really two branches of Spectre defense research, and each side sneers at the other and declares their approach doesn't work. One side says "we can block speculation at the trust boundary, especially the process boundary by having the kernel flush all caches, etc." The argument against this is that every new attack has to be explicitly mitigated. New attacks are coming at a rather fast rate and it's almost certain that some bad guys are aware of attacks that the good guys haven't uncovered yet. (It's also ridiculously expensive to use these defenses at a fine-grained level.) The other side says "we can make it basically infeasible to extract side channels by limiting the non-determinism -- such as timers -- that allow malicious programs to observe microarchitectural side effects." The first side says this is wrong, you can use repeated attempts and statistics to get over any amount of noise. It just takes longer. Maybe you have to run the attack for weeks to leak anything of value but it's still possible. The reality is that no one has actually solved Spectre. However, both sides have done things that raise the barrier to attack. The best anyone can do right now is try to raise that barrier as high as they can. It seems to be working -- we don't really see Spectre attacks in the wild. Workers uses a mix of ideas in a pragmatic defense. I'll have an extended post about it on the Cloudflare blog tomorrow.
  - mdriley 6y ago
    (disclosure: I work at Google on side-channel stuff and https://github.com/google/safeside https://github.com/google/safeside) I'm really excited to read that post! I agree that the best we can claim right now is that we've made Spectre and other speculative attacks "expensive enough" that they're unlikely to be the most profitable area for attack. That said, I'd be a bit worried about the assertion we haven't seen Spectre attacks "in the wild". It is incredibly difficult to put together a set of metrics that would convincingly detect attempts at even a straightforward speculative information disclosure. (haha, two branches, I get it)

monadic2 6y ago

I’m open to a slow but secure processor. Certainly for my personal computing.

pjmlp 6y ago
I feel the same about bounds checking in programming languages, apparently not everyone does.
- KMag 6y ago
  I vaguely remember a story about a hardware manufacturer (Burroughs? Symbolics?) that had a processor with instructions that could bounds-check array accesses with zero latency overhead. It was a great feature for Algol/Lisp, but some customers asked for an option in their Fortran compiler to disable the bounds check. The sales engineer replied that the bounds checks were zero-cost, but the customers replied that the bounds checks broke their progrems... their programs had silent (or at least unnoticed) array bounds bugs, and the customers preferred to keep those bugs, thank you very much! I wish I had kept the link. I've tired a couple times to find the story. Does this ring any bells for anyone?
  - pjmlp 6y ago
    Burroughs definitely had bounds checking. https://en.wikipedia.org/wiki/Burroughs_large_systems https://en.wikipedia.org/wiki/Burroughs_large_systems Its system programming language (initially ESPOL then NEWP), also has support for explicit unsafe code blocks, and there is zero Assembly support. All CPU low level operations are exposed via intrisics. All of this in 1961, almost 10 years before C was invented. Still being sold nowadays, and naturally Unisys uses security as one of the selling features. https://www.unisys.com/offerings/clearpath-forward/clearpath-forward-products/clearpath-mcp-software https://www.unisys.com/offerings/clearpath-forward/clearpath... Regarding bounds checking, what I keep around is Hoare's Turing award speech. "Many years later we asked our customers whether they wished us to provide an option to switch off these checks in the interests of efficiency on production runs. Unanimously, they urged us not to--they already knew how frequently subscript errors occur on production runs where failure to detect them could be disastrous. I note with fear and horror that even in 1980, language designers and users have not learned this lesson. In any respectable branch of engineering, failure to observe such elementary precautions would have long been against the law."
    - monadic2 6y ago
      I used to agree heavily with this, but nowadays we have effective techniques of memory safety on the software level and I find myself ambivalent on which level of abstraction this occurs. I do think it means C is probably eventually doomed as a system language, though. I’m curious if we’ll ever see e.g. rust in the *bsd codebases.
      - pjmlp 6y ago
        Except we don't, that is why ARM, Apple, Microsoft, Google, Oracle are all pursuing variants of hardware memory tagging for taming C, as those software solutions have proven not to work.
        monadic2 6y ago
        Sure, but hardware memory tagging is also not proven to work. Anyway, it's unclear with which criteria you're judging "proven not to work" with as we're both typing via the software right now, unlike a hardware solution.
mehrdadn 6y ago
Try underclocking your CPU to like 30% of its max speed and see if your programs still run comfortably. This would've made sense in a world where programs didn't become more bloated and slower over time, but it would have visible repercussions given our software today.
- pjmlp 6y ago
  That would be positive though, maybe then not so many people would be running Python sites on Django or Electron apps.
  - andybak 6y ago
    It sounds like you want to actively discourage the use of Python and Django irrespective of performance concerns? Care to elaborate or was it just flippant snide?
    - pjmlp 6y ago
      More like discourage the use of pure scripting without any regard for JIT/AOT toolchains or coding without performance considerations, in opposition to what we used to care about. The example with Python and Django was what came quickest to mind, but I can gladly expand it to include Ruby and Rails, or any other stack that falls under the same Web sites/desktop applications with scripting languages umbrella.
      - andybak 6y ago
        The usual retort is "the scripting language is rarely the bottleneck" as well as "developer time is more expensive than hardware". Between these two get-out clauses, aren't you talking about a very small minority of use-cases? Those where the scripting language is the bottleneck and the problem can't simply be solved by spending a few $ more on your VPS?
        pjmlp 6y ago

zozbot234 6y ago

Spectre vulnerabilities only matter wrt. information disclosure. Really we need OS's that are explicitly aware of information domains, and only flush address space mappings when switching from a more privileged to a less privileged domain.

mdriley 6y ago
so, some things: 1. Information disclosure is pretty important, especially if your process has AWS credentials in the environment block or it's doing mTLS with a long-lived key. 2. Those operating systems already exist, see e.g. https://twitter.com/aionescu/status/948818841747955713 https://twitter.com/aionescu/status/948818841747955713 3. Spectre V1 is within the same process, so this isn't a question of address mappings across differently-privileged domains. It's the same domain (i.e. address space). 4. Flushing address spaces across privilege domains isn't a concern on modern processors thanks to tagged TLBs and process-context or address space identifiers (PCID, ASID)

syrusakbary 6y ago

There are some interesting approaches for running Wasm code confidentially. I'd recommend to take a look into Enarx, as they are pioneering the space with SEV/SGX integration into Wasm workloads. https://opensource.com/article/19/5/enarx-security https://opensource.com/article/19/5/enarx-security

mdriley 6y ago
SGX is a tool in the toolbox, but it solves a different problem: isolating a small section of especially privileged code from the rest of a larger, less-trusted application. The sandbox described in the article is trying to do roughly the opposite: protect the main application from an isolated section of untrusted code. Also, SGX requires extreme care in deployment due to side-channel attacks, see e.g. https://software.intel.com/security-software-guidance/insights/deep-dive-load-value-injection https://software.intel.com/security-software-guidance/insigh... SEV is also interesting, but requires code to run in a separate VM -- which satisfies my requirement above that it at least be in a different process.

reitzensteinm 6y ago

If you had control over the ISA, and were jitting code as you would with WASM, couldn't you defeat Spectre 1 with an instruction that essentially introduces a masked region that the processor will not read or speculate outside of until the mode is disabled? If the attacker controls the instruction stream it's game over (since they can disable the mask), but if they're just writing wasm that generates malicious speculative in process loads, it would prevent that. Then an attack would have to trick library calls in to doing the same job, which is certainly possible, but much harder. Of course this would negate many of the benefits of staying in the same process, so I'm not necessarily saying it's a good idea.

mdriley 6y ago
When you say "control over the ISA", I'll assume you mean "precise control over the emitted instructions". In which case: yes! That's Speculative Load Hardening (https://llvm.org/docs/SpeculativeLoadHardening.html https://llvm.org/docs/SpeculativeLoadHardening.html). SLH tries to squash side-channels by preventing any speculatively-loaded data from being forwarded to dependent instructions until proving that branch prediction followed the right path. But this undoes a lot of the performance that microarchitectures have added through branch prediction, since dependent memory loads (think linked list entries, or C++ vtables) are stalled behind full resolution of the branch condition. If you're doing nontrivial compute, you can end up ahead performance-wise by splitting the computation into a separate process and invoking it via IPC. Now you don't need SLH because the untrusted process doesn't have long-lived secrets in its address space.
- reitzensteinm 6y ago
  No, I mean specifically if you're building your own CPUs and can add instructions. You add the ability to set a hardware mask that all values are passed through before they're used as addresses for loads and stores including speculation. Loads and stores that fall outside the masked region will simply wrap around. In your JIT, you enter and leave this mode before and after running user code to ensure it can't escape its region. This would be a lot of work to pull off and would require custom hardware and software, but (at least as far as I can tell) it should work.

KMag 6y ago

Spectre makes me wish we'd give something like IBM's project DAISY / Transmeta's Crusoe another chance. If we have one mode with a simple and high-density instruction set similar to Hitachi's SH4 / ARM's Thumb2, along with a VLIW mode (similar to switching between Thumb2 and regular ARM), and some hardware support for tracing and dynamic recompilation (including reservoir sampling of instructions causing pipeline stalls), we might get decent performance and code density, while being able to move all of the speculation into the dynamic recompilation layer. Having the processor natively support the dense non-VLIW instruction set means you don't pay much in the way of startup latency, and once you're warmed up, your hot spots have all of their cross-DLL calls inlined / no longer indirect, and your virtual function calls devirtualized / speculative inlined. Spots where the dynamic recompiler was wrong about pipeline stalls eventually use dynamic information to recompile and shuffle instructions around to avoid pipeline stalls. You don't pay the transistor and power budget for out-of-order execution or speculative execution hardware. Hardware speculative execution gets replaced with predicated instructions and shadow register save/restores. I think IBM's DAISY was much closer to being on the right track vs. Intel's EPIC / Itanium. If your system is built for dynamic recompilation and re-optimization of native code on the fly, your compiler doesn't have to be as good at statically predicting execution paths and pipeline stalls. Transmeta's main problem was that they were emulating all x86 instructions before they warmed up, and after warm-up they were still emulating all x86 instructions outside their hot code path. With a simpler instruction set like SH4 / Thumb2, they could hopefully have hardware support for the code outside of the hot spots / pre-warmup. Given the amount of time spent running JavaScript, hopefully one would also look at the bytecodes for V8, JavaScriptCore, and SpiderMonkey for inspiration as far as making the non-VLIW instruction set an efficient and compact JavaScript JIT target.

titzer 6y ago
> If we have one mode with a simple and high-density instruction set similar to Hitachi's SH4 / ARM's Thumb2, along with a VLIW mode (similar to switching between Thumb2 and regular ARM), and some hardware support for tracing and dynamic recompilation (including reservoir sampling of instructions causing pipeline stalls), we might get decent performance and code density, while being able to move all of the speculation into the dynamic recompilation layer. Trading hardware complexity for software complexity is not the right strategy to combat sidechannels. They just reappear up the stack. Any speculative optimization that depends on program values can potentially leak information through timing.

rubber_duck 6y ago

>From what I've seen, the best understanding right now is that the process is the smallest defensible unit of isolation. In what scenario would you be more comfortable running a potentially malicious process versus a potentially malicious WASM code behind a API sandbox where you control the exposed APIs. Sure there are process level sandboxes but that seems like a much larger bug surface area with expertise outside of the domain of people building the apps, compared to a WASM sandbox with a host app defined bridge API.

the8472 6y ago
> Sure there are process level sandboxes but that seems like a much larger bug surface area A process sandbox allows you to pass through syscalls you deem harmless without reimplementing them and only building the bridge API for things that require complex policies. With wasm you need to implement wrappers for everything. Let's say your wasm module is pure compute and does IPC via shared memory locks. You now at least have to implement the locking API. In the process sandbox case you only whitelist the futex syscall and that's it. And in practice you'll have to use the process sandbox anyway due to CPU vulnerabilities or bugs in the bridge API.
- rubber_duck 6y ago
  >A process sandbox allows you to pass through syscalls you deem harmless without reimplementing them and only building the bridge API for things that require complex policies. With wasm you need to implement wrappers for everything. Yeah except you have to know what OS your application is going to run on, kernel version, set up the sandbox correctly and trust that the kernel doesn't have any privilege escalation bugs - likely none of which is your area of expertise as an application developer. A sandbox runtime with exposed APIs is much closer to your domain and you can model it to be much more domain specific so the surface you need to wrap should be smaller.
  - the8472 6y ago
    I think the surface mostly depends on how much you want to give the sandboxed application, not on the sandboxing technology you choose. If want to build something emulating the capabilities of a desktop operating system including gfx acceleration (e.g. webgpu) then it will be much larger than some isolated compute function in the fashion of network edge workers. I am not aware of a single CVE against openbsd's pledge. On the other hand there are plenty of javascript engine escapes.

voidmain 6y ago

> Spectre V1 (speculative bounds check bypass / type confusion) is basically game over for intra-process memory isolation without introducing expensive or complicated mitigations Code which is executed deterministically cannot receive timing channels (or, indeed, learn anything about its environment) and hence cannot exploit Spectre. This seems potentially practical for this sort of library sandboxing problem.

azakai 6y ago
Exactly, by default code sandboxed by wasm is fully deterministic and can't do any timing measurements, not unless you explicitly give it access to an import that does such a measurement. On the web, a website might run arbitrary wasm + JS which means it might let wasm time things. But if you use wasm to sandbox a specific library then the situation is different and you control the wasm imports.
- saagarjha 6y ago
  > by default code sandboxed by wasm is fully deterministic Sorry? I'm not sure what you mean by "fully deterministic" here, because as far as I was aware you can do basically anything including choosing to not terminate inside of WASM.
  - azakai 6y ago
    You can have an infinite loop, sure, but aside from that wasm semantics are precisely defined in a deterministic way (well, except for minor issues with float NaN bits). That is, if a computation terminates, it will always terminate and with the same results. That's the case because wasm itself has no way to tell the time, generate a random number, etc., and each operation's semantics are well-defined. (If infinite loops are a concern, you can do what wasm VMs do on the web which is to show a "stop script?" dialog after too much time passes.)
    - monocasa 6y ago
      WASM is not deterministic from a timing perspective.

unoti 6y ago

This is interesting. If data confidentiality requires process isolation, then how can CloudFlare workers be safe?

kentonv 6y ago
There are really two branches of Spectre defense research, and each side sneers at the other and declares their approach doesn't work. One side says "we can block speculation at the trust boundary, especially the process boundary by having the kernel flush all caches, etc." The argument against this is that every new attack has to be explicitly mitigated. New attacks are coming at a rather fast rate and it's almost certain that some bad guys are aware of attacks that the good guys haven't uncovered yet. (It's also ridiculously expensive to use these defenses at a fine-grained level.) The other side says "we can make it basically infeasible to extract side channels by limiting the non-determinism -- such as timers -- that allow malicious programs to observe microarchitectural side effects." The first side says this is wrong, you can use repeated attempts and statistics to get over any amount of noise. It just takes longer. Maybe you have to run the attack for weeks to leak anything of value but it's still possible. The reality is that no one has actually solved Spectre. However, both sides have done things that raise the barrier to attack. The best anyone can do right now is try to raise that barrier as high as they can. It seems to be working -- we don't really see Spectre attacks in the wild. Workers uses a mix of ideas in a pragmatic defense. I'll have an extended post about it on the Cloudflare blog tomorrow.
- mdriley 6y ago
  (disclosure: I work at Google on side-channel stuff and https://github.com/google/safeside https://github.com/google/safeside) I'm really excited to read that post! I agree that the best we can claim right now is that we've made Spectre and other speculative attacks "expensive enough" that they're unlikely to be the most profitable area for attack. That said, I'd be a bit worried about the assertion we haven't seen Spectre attacks "in the wild". It is incredibly difficult to put together a set of metrics that would convincingly detect attempts at even a straightforward speculative information disclosure. (haha, two branches, I get it)