9 ms·

Reality Check for Cloudflare Wasm Workers and Rust

156 points by comagoosie 5y ago

brundolf 5y ago
> I guess I’ll stick with my error prone Javascript Workers or, more likely, spend an afternoon migrating to a minimal Typescript setup. If the OP wants a zero-config typescript experience (assuming Deno isn't available on Cloudflare workers), I can't recommend esbuild enough
- wtetzner 5y ago
  There's also js_of_ocaml if a good type system is desired.
  - brundolf 5y ago
    I think that's out of scope for the OP's needs/wants. They like TypeScript for catching basic API mistakes, the only thing they don't want is configuration headaches. I didn't get the sense they would be interested in learning a new language for this use-case, especially since I'm going to guess Cloudflare doesn't publish OCaml types for their JavaScript API.
    - wtetzner 5y ago
      That's fair. Guess I just got excited about an opportunity to plug js_of_ocaml ;)
  - Yoric 5y ago
    When we wrote opalang, the size of generated JS was problematic, despite serious efforts at minimizing it. Does js_of_ocaml do better?
    - wtetzner 5y ago
      I'm not familiar with opalang, but I haven't noticed size issues with js_of_ocaml (though I've only used it for personal projects). It works by compiling OCaml bytecode to JS. I think the main thing you'd need to be careful about in terms of code size is which libraries you end up relying on. EDIT: There's also bucklescript, which goes from OCaml to JS (skips the bytecode), but I think it is more relaxed in terms of preserving OCaml behavior. i.e., it relies on how JS behaves for certain operations, instead of trying to preserve how natively compiled OCaml would behave.
- comagoosie 5y ago
  I think this is an excellent suggestion (I'm OP / author), and one one can add just a dash to this for typechecking. Minimal setups are appreciated, especially when one has many small projects.
  - brundolf 5y ago
    Glad I could help! Esbuild won't do the actual type-checking for you, but your editor will (hopefully also without configuration) I'll put it this way: I've spent enough time with Webpack and Babel and TSC at this point that I can troubleshoot most issues without too much difficulty. But despite that I reach for esbuild every time I possibly can, because I just don't want to mess with all that stuff if I don't have to.
dafelst 5y ago
Great overview OP, and it's nice to see a kind of "in-between" scenario tested, i.e. not a super fast web request or transformation, rather something more akin to a lightweight batch job. It may not be quite a "recommended" use case but it is always interesting (for me at least) to see how these sorts of services' capabilitied can be pushed and or (gently) abused. The memory and code size limitations do seem very restrictive right now, which is a shame though. Seeing WASM evolve as the new sandboxed runtime target dejour is a super interesting and I love that it is bringing more variety of very powerful but traditionally backend or systems languages to the web.
dgreensp 5y ago
I think the one-sentence version of this is that Workers are meant for small, undemanding tasks (for example, they have tight memory limits and don’t have great performance), so using them to do “serious number crunching” at the edge, which is the advertised use case, seems questionable. I think the blurb about the downsides of Wasm is just too generic, it’s a sort of “why Wasm isn’t preferable to JS in all cases” for the uninitiated. It may not be meant to imply that number crunching is the use case.
- brundolf 5y ago
  I for one hadn't thought about the cost of shipping your own standard library with every bundle, so it was informative for me
  - wibagusto 5y ago
    WASM tasks shouldn’t need a full standard library. If you statically compile against any library it should only keep the pieces used.
    - brundolf 5y ago
      That's fair, but still, you'll be pulling in a lot of really fundamental stuff that JavaScript gets for free. String manipulation, fundamental data structures, iterators, HTTP, JSON (de)serialization, etc.
      - mst 5y ago
        One of the examples he mentioned was 200Kb. That's still a lot more than "a single file of javascript", certainly, but it's not that bad. Trade-offs all the way down, as ever.
    - merb 5y ago
      I'm not sure but if you need any string manipulation at all it will be really hard to not use the rust std?
up6w6 5y ago
iirc the compilation time of wasm in Cloudflare Workers is very problematic[1] and right now it contradicts their idea of running low latency fast scripts, does anyone know if anything has changed ? https://community.cloudflare.com/t/fixed-cloudflare-workers-slow-with-moderate-sized-webassembly-bindings/184668/10 https://community.cloudflare.com/t/fixed-cloudflare-workers-...
- kentonv 5y ago
  Yes, it has changed, which is why the thread you linked has "[FIXED]" in the title. Details can be found later on in the thread.
  - nostrebored 5y ago
    Your replies throughout the thread have been pretty hostile. You're much closer to this than anyone here. Empathy to that goes a long way.
    - eloff 5y ago
      I think you're assuming tone that isn't there. Look at it again, it's a factual statement. Nothing angry or accusational. As you say, a little empathy goes a long way. Try to interpret tone on the internet in a charitable manner, because you're just guessing at it most of the time.
      - nostrebored 5y ago
        I work in customer facing roles. Responses like that are considered hostile. If I responded like that in a professional capacity -- like the parent comment -- I'd have a lot of questions to answer. And from the reply, the response wasn't even correct. Bad form.
        kentonv 5y ago
        You're right, I should have been more careful when writing that comment. I was trying to express that the original issue was fixed (and the "new issue" isn't actually an issue as far as we can tell), I didn't mean for it to come out hostile but I definitely see how it looks that way. I could have done better.
wibagusto 5y ago
Correct me if I’m wrong but the memory copying issue is not an issue if you pass an array buffer into WASM from JavaScript. In that scenario there’s no data copying. E.g. similar to how you’d pass the canvas data to WASM for direct manipulation.
- azakai 5y ago
  No, this is an issue currently, both for network data and canvas data. All wasm instructions can do is read and write from the wasm Memory that the wasm is initialized with. They can't even refer to separate things like a new ArrayBuffer from JS. So you do need to copy. Newer wasm additions like reference types allow an ArrayBuffer to be referred to inside wasm, but only as an opaque reference to the entire thing (an externref). There is still no ability to actually read and write from it inside wasm. The solution to this is BYOB ("bring your own buffer") APIs, which JS is adding. They are experimental atm though. Here is the relevant one here: https://developer.mozilla.org/en-US/docs/Web/API/ReadableStreamBYOBReader/read https://developer.mozilla.org/en-US/docs/Web/API/ReadableStr... Note how you pass in a view to the JS API. That can be a view into the wasm memory, letting the browser directly write data into there, and then wasm can operate on it immediately.
  - wibagusto 5y ago
    So the workaround is to initialize WASM with an array buffer which allows direct manipulation of the data in WASM it seems. The canvas bytes can be used in WASM so long as the bytes are passed in at initialization. In that case, JavaScripts only duty is to pack input data into the byte array and pass control to WASM which writes to an output buffer (also shared byte array) and passes control back to JavaScript to process result avoiding an unnecessary data copy. You’d have to have some types in JS which can process the data using bit shifts and whatnot though in a lower level way.
- Matthias247 5y ago
  So how do you fill that arraybuffer from Javascript datatypes (e.g. strings which describe the request)? The answer is "by copying the relevant data" - which is exactly what the bridging code between JS and WASM does. You can only avoid that if the source data is already plain byte arrays and not javascript objects.
jfrunyon 5y ago
I believe a zip file could be streamed - most of the file metadata is duplicated between both the 'central directory record' trailer and a header in front of each file. In other words, the first thing in the zip file is a header that you can use to extract the first file, followed by that file, followed by the next file's header...
- stavros 5y ago
  You can, yes. You can even download the header, open the zip file, choose which files to extract, and only download those. This is possible with HTTP today, and has been for decades.
  - jfrunyon 5y ago
    > You can even download the header, open the zip file, choose which files to extract, and only download those. That bit isn't true, unfortunately; the file list is stored at the end of the file. You either have to (sequentially, file-by-file) scan the zip to find the file you want, or look at the end. Even more unfortunately, there are several variable-length fields between the interesting stuff like the file list and the end of the archive, and the length of those fields is stored before the field itself, so you can't simply use a Range request to retrieve the last <x> bytes from the end, either. (And even more more unfortunately, the very last thing in the file is a "comment" field, which could conceivably contain the magic number that you have to look for in order to read the trailer/footer record.) The Zip format is truly awful. https://en.wikipedia.org/wiki/ZIP_(file_format)#Central_directory_file_header https://en.wikipedia.org/wiki/ZIP_(file_format)#Central_dire... https://github.com/python/cpython/blob/7e465a6b8273dc0b6cb484c65664f618afa22484/Lib/zipfile.py#L256 https://github.com/python/cpython/blob/7e465a6b8273dc0b6cb48...
    - pugz 5y ago
      Something I’ve wondered for a while: is there a good modern alternative to zip?

156 points by comagoosie 5y ago

> I guess I’ll stick with my error prone Javascript Workers or, more likely, spend an afternoon migrating to a minimal Typescript setup. If the OP wants a zero-config typescript experience (assuming Deno isn't available on Cloudflare workers), I can't recommend esbuild enough

wtetzner 5y ago
There's also js_of_ocaml if a good type system is desired.
- brundolf 5y ago
  I think that's out of scope for the OP's needs/wants. They like TypeScript for catching basic API mistakes, the only thing they don't want is configuration headaches. I didn't get the sense they would be interested in learning a new language for this use-case, especially since I'm going to guess Cloudflare doesn't publish OCaml types for their JavaScript API.
  - wtetzner 5y ago
    That's fair. Guess I just got excited about an opportunity to plug js_of_ocaml ;)
- Yoric 5y ago
  When we wrote opalang, the size of generated JS was problematic, despite serious efforts at minimizing it. Does js_of_ocaml do better?
  - wtetzner 5y ago
    I'm not familiar with opalang, but I haven't noticed size issues with js_of_ocaml (though I've only used it for personal projects). It works by compiling OCaml bytecode to JS. I think the main thing you'd need to be careful about in terms of code size is which libraries you end up relying on. EDIT: There's also bucklescript, which goes from OCaml to JS (skips the bytecode), but I think it is more relaxed in terms of preserving OCaml behavior. i.e., it relies on how JS behaves for certain operations, instead of trying to preserve how natively compiled OCaml would behave.
comagoosie 5y ago
I think this is an excellent suggestion (I'm OP / author), and one one can add just a dash to this for typechecking. Minimal setups are appreciated, especially when one has many small projects.
- brundolf 5y ago
  Glad I could help! Esbuild won't do the actual type-checking for you, but your editor will (hopefully also without configuration) I'll put it this way: I've spent enough time with Webpack and Babel and TSC at this point that I can troubleshoot most issues without too much difficulty. But despite that I reach for esbuild every time I possibly can, because I just don't want to mess with all that stuff if I don't have to.

dafelst 5y ago

Great overview OP, and it's nice to see a kind of "in-between" scenario tested, i.e. not a super fast web request or transformation, rather something more akin to a lightweight batch job. It may not be quite a "recommended" use case but it is always interesting (for me at least) to see how these sorts of services' capabilitied can be pushed and or (gently) abused. The memory and code size limitations do seem very restrictive right now, which is a shame though. Seeing WASM evolve as the new sandboxed runtime target dejour is a super interesting and I love that it is bringing more variety of very powerful but traditionally backend or systems languages to the web.

dgreensp 5y ago

I think the one-sentence version of this is that Workers are meant for small, undemanding tasks (for example, they have tight memory limits and don’t have great performance), so using them to do “serious number crunching” at the edge, which is the advertised use case, seems questionable. I think the blurb about the downsides of Wasm is just too generic, it’s a sort of “why Wasm isn’t preferable to JS in all cases” for the uninitiated. It may not be meant to imply that number crunching is the use case.

brundolf 5y ago
I for one hadn't thought about the cost of shipping your own standard library with every bundle, so it was informative for me
- wibagusto 5y ago
  WASM tasks shouldn’t need a full standard library. If you statically compile against any library it should only keep the pieces used.
  - brundolf 5y ago
    That's fair, but still, you'll be pulling in a lot of really fundamental stuff that JavaScript gets for free. String manipulation, fundamental data structures, iterators, HTTP, JSON (de)serialization, etc.
    - mst 5y ago
      One of the examples he mentioned was 200Kb. That's still a lot more than "a single file of javascript", certainly, but it's not that bad. Trade-offs all the way down, as ever.
  - merb 5y ago
    I'm not sure but if you need any string manipulation at all it will be really hard to not use the rust std?

up6w6 5y ago

iirc the compilation time of wasm in Cloudflare Workers is very problematic[1] and right now it contradicts their idea of running low latency fast scripts, does anyone know if anything has changed ? https://community.cloudflare.com/t/fixed-cloudflare-workers-slow-with-moderate-sized-webassembly-bindings/184668/10 https://community.cloudflare.com/t/fixed-cloudflare-workers-...

kentonv 5y ago
Yes, it has changed, which is why the thread you linked has "[FIXED]" in the title. Details can be found later on in the thread.
- nostrebored 5y ago
  Your replies throughout the thread have been pretty hostile. You're much closer to this than anyone here. Empathy to that goes a long way.
  - eloff 5y ago
    I think you're assuming tone that isn't there. Look at it again, it's a factual statement. Nothing angry or accusational. As you say, a little empathy goes a long way. Try to interpret tone on the internet in a charitable manner, because you're just guessing at it most of the time.
    - nostrebored 5y ago
      I work in customer facing roles. Responses like that are considered hostile. If I responded like that in a professional capacity -- like the parent comment -- I'd have a lot of questions to answer. And from the reply, the response wasn't even correct. Bad form.
      - kentonv 5y ago
        You're right, I should have been more careful when writing that comment. I was trying to express that the original issue was fixed (and the "new issue" isn't actually an issue as far as we can tell), I didn't mean for it to come out hostile but I definitely see how it looks that way. I could have done better.

wibagusto 5y ago

Correct me if I’m wrong but the memory copying issue is not an issue if you pass an array buffer into WASM from JavaScript. In that scenario there’s no data copying. E.g. similar to how you’d pass the canvas data to WASM for direct manipulation.

azakai 5y ago
No, this is an issue currently, both for network data and canvas data. All wasm instructions can do is read and write from the wasm Memory that the wasm is initialized with. They can't even refer to separate things like a new ArrayBuffer from JS. So you do need to copy. Newer wasm additions like reference types allow an ArrayBuffer to be referred to inside wasm, but only as an opaque reference to the entire thing (an externref). There is still no ability to actually read and write from it inside wasm. The solution to this is BYOB ("bring your own buffer") APIs, which JS is adding. They are experimental atm though. Here is the relevant one here: https://developer.mozilla.org/en-US/docs/Web/API/ReadableStreamBYOBReader/read https://developer.mozilla.org/en-US/docs/Web/API/ReadableStr... Note how you pass in a view to the JS API. That can be a view into the wasm memory, letting the browser directly write data into there, and then wasm can operate on it immediately.
- wibagusto 5y ago
  So the workaround is to initialize WASM with an array buffer which allows direct manipulation of the data in WASM it seems. The canvas bytes can be used in WASM so long as the bytes are passed in at initialization. In that case, JavaScripts only duty is to pack input data into the byte array and pass control to WASM which writes to an output buffer (also shared byte array) and passes control back to JavaScript to process result avoiding an unnecessary data copy. You’d have to have some types in JS which can process the data using bit shifts and whatnot though in a lower level way.
Matthias247 5y ago
So how do you fill that arraybuffer from Javascript datatypes (e.g. strings which describe the request)? The answer is "by copying the relevant data" - which is exactly what the bridging code between JS and WASM does. You can only avoid that if the source data is already plain byte arrays and not javascript objects.

jfrunyon 5y ago

I believe a zip file could be streamed - most of the file metadata is duplicated between both the 'central directory record' trailer and a header in front of each file. In other words, the first thing in the zip file is a header that you can use to extract the first file, followed by that file, followed by the next file's header...

stavros 5y ago
You can, yes. You can even download the header, open the zip file, choose which files to extract, and only download those. This is possible with HTTP today, and has been for decades.
- jfrunyon 5y ago
  > You can even download the header, open the zip file, choose which files to extract, and only download those. That bit isn't true, unfortunately; the file list is stored at the end of the file. You either have to (sequentially, file-by-file) scan the zip to find the file you want, or look at the end. Even more unfortunately, there are several variable-length fields between the interesting stuff like the file list and the end of the archive, and the length of those fields is stored before the field itself, so you can't simply use a Range request to retrieve the last <x> bytes from the end, either. (And even more more unfortunately, the very last thing in the file is a "comment" field, which could conceivably contain the magic number that you have to look for in order to read the trailer/footer record.) The Zip format is truly awful. https://en.wikipedia.org/wiki/ZIP_(file_format)#Central_directory_file_header https://en.wikipedia.org/wiki/ZIP_(file_format)#Central_dire... https://github.com/python/cpython/blob/7e465a6b8273dc0b6cb484c65664f618afa22484/Lib/zipfile.py#L256 https://github.com/python/cpython/blob/7e465a6b8273dc0b6cb48...
  - pugz 5y ago
    Something I’ve wondered for a while: is there a good modern alternative to zip?