11 ms·
Stable Diffusion WebGPU demo
- ccooffee 3y ago> You need latest Chrome with "Experimental WebAssembly" and "Experimental WebAssembly JavaScript Promise Integration (JSPI)" flags enabled! Darn, guess I'll have to wait for stuff to land in Firefox.
- flangola7 3y agoWhat does the second flag do
- bhaney 3y agoTurns on an implementation of this proposal https://github.com/WebAssembly/js-promise-integration/blob/main/proposals/js-promise-integration/Overview.md https://github.com/WebAssembly/js-promise-integration/blob/m...
- insanitybit 3y agoWhy is it that implementing something in wasm is stalled for so long but doing it as a js feature is so fast? Anyone have insights? As an outsider it feels like wasm is being developed in an impossibly slow way.
- chrisweekly 3y agoI don't know for sure, but gut says the primary factor must be the ratio of devs working on JS runtimes vs WASM (over 10,000:1?)
- kevingadd 3y agoImplementing something new in JS can be done relatively easily using a slow path, where you just write some privileged JS or C++ and then wrap it, without doing any optimizations. Then if it gets popular the vendors can optimize it at their own pace. Implementing a new feature in WebAssembly is a bit more complex due to its execution model and security constraints. I expect it's also just the case that a lot of these new WASM features are very complex - promise integration is super nontrivial to get right, so are WebAssembly GC and SIMD.
- insanitybit 3y agoOK, I'd believe that I guess.
- rictic 3y agoJS Promises in something like their modern form were first played around with in ~2010, and it was ~2016 before browsers were shipping them natively. Good standards can take a while!
- pjmlp 3y agoBecause it basically covers what PNaCL, Java plugin, Flash plugin, Silverlight asm.js were doing. Anything beyond those use cases it is really meh, specially given how clunky compiling and debugin WASM code tends to be. Then we have all those startups trying to reivent bytecode executable formats in the server, as if it wasn't something that has been done every couple of years since late 1950's.
- insanitybit 3y ago> Because it basically covers what PNaCL, Java plugin, Flash plugin, Silverlight asm.js were doing. Right but it doesn't right now? Like you can't just write arbitrary code as you would with a Java plugin, or a PNaCL C++ plugin. Wasm is extremely difficult to use for those use cases. > Then we have all those startups trying to reivent bytecode executable formats in the server, as if it wasn't something that has been done every couple of years since late 1950's. Yes, because people really want this and the solutions have all been fraught with security issues historically.
- pjmlp 3y agoWhat makes you think WASM isn't without flaws, just because their advocates say so? "Everything Old is New Again: Binary Security of WebAssembly" https://www.usenix.org/conference/usenixsecurity20/presentation/lehmann https://www.usenix.org/conference/usenixsecurity20/presentat... "Swivel: Hardening WebAssembly against Spectre" https://www.usenix.org/conference/usenixsecurity21/presentation/narayan https://www.usenix.org/conference/usenixsecurity21/presentat...
- insanitybit 3y agoI didn't say WASM is without flaws, I said the predecessors had flaws but that the premise is valuable, which is why we keep trying it over and over again. Notably, the first paper is about exploitation of webassembly processes. That's valuable but the flaws of previous systems wasn't that the programs in those systems were exploitable but that the virtual machines were. Some of this was due to the fact that the underlying virtual machines, like the JVM, were de-facto unconstrained and the web use case attempted to add constraitns on after the fact; obviously webassembly has been designed differently. I hope wasm sees more mitigations, but I also expect that wasm is going to be a target primarily for memory safe languages where these problems are already significantly less significant. And to reiterate, the issue was not the exploitation of programs but exploitation of the virtual machines isolation mechanisms.
- bsimpson 3y agoI tried turning them on in Canary, but I still couldn't get it to work.
- veb 3y agoI tried on latest (normal) Chrome, beta and canary/nightly and enabled both options one by one and let it relaunch but still wouldn't work at all. ¯\_(ツ)_/¯ How are others getting it working?! From someone else's comment, this one works fine: https://websd.mlc.ai/#text-to-image-generation-demo https://websd.mlc.ai/#text-to-image-generation-demo
- aislamov 3y agoWhat error message do you get?
- veb 3y agoHold on, to run your demo does one have to click the "Load Model" button before doing anything? 'cos what I see is a form that is greyed out with the error message still at the top: > You need latest Chrome with "Experimental WebAssembly" and "Experimental WebAssembly JavaScript Promise Integration (JSPI)" flags enabled! Now I'm wondering whether the top message goes away once the flags are enabled?
- aislamov 3y ago> Hold on, to run your demo does one have to click the "Load Model" button before doing anything? Yes. I thought it won't be good if it would download 3.5gb once you open the page. >Now I'm wondering whether the top message goes away once the flags are enabled? No, I haven't added any checks for that (and I'm not sure how the first one can be properly checked), so it's just an info bar. Which is, eventually, misleading.
- marcod 3y agoIt's working on Edge / win 11 ;) If you can't run it, here's how the output looks with default settings https://i.imgur.com/WCQc8hO.png https://i.imgur.com/WCQc8hO.png
- cortesoft 3y agoYou refuse to even try something on chrome?
- ccooffee 3y agoNah, I don't use Chrome so I don't have it installed. I'm not a web developer, so testing across different platforms isn't useful to me. I've used StableDiffusion before, so hacking around to make this demo work in my browser isn't particularly interesting either.
- ChuckMcM 3y agoYes. You don't?
- INGSOCIALITE 3y agoI agree with the poster 100%. Im convinced any Google applications immediately suck every iota of data they possibly can at install time / first launch. It’s not worth it to me either.
- tpowell 3y agoI enabled the requested chrome:flags in Brave, but it still doesn't work. I haven't downloaded Chrome on any of my M1 Macs, and don't plan to start now.
- radiKal07 3y agosame here, I use Chromium but it's the latest version and I enabled those flags... website still does not work
- moffkalast 3y agoAt least Firefox now allows for CSS scrollbar styling, so you have that going for you which is nice.
- brucethemoose2 3y agoAlso already done here: https://websd.mlc.ai/#text-to-image-generation-demo https://websd.mlc.ai/#text-to-image-generation-demo MLC uses Apache TVM to generate and autotune the webGPU code, and its respectably performant.
- simonw 3y agoThe MLC team got that working back in March: https://github.com/mlc-ai/web-stable-diffusion https://github.com/mlc-ai/web-stable-diffusion Even more impressively, they followed up with support for several Large Language Models: https://webllm.mlc.ai/ https://webllm.mlc.ai/
- aislamov 3y agoThat's really impressive and much more performant. I was following a different approach: to run any ONNX model without prior modifications.
- FL33TW00D 3y agoONNX is bloated! I got some LLMs working on my own Rust + WebGPU framework a few months ago: https://summize.fleetwood.dev/ https://summize.fleetwood.dev/ I've since moved away from ONNX and to a more GGML style.
- taminka 3y agowhat's the difference between onnx and ggml style?
- FL33TW00D 3y agoONNX consumes a .onnx file, which is a definition of the network and weights. GGML instead just consumes the weights, and defines the network in code. Being bound to ONNX means moving at a slower velocity - the field moves so fast that you need complete control.
- michaelmior 3y agoI haven't used ONNX or GGML, but presumably using GGML means you need to reimplement the network architecture?
- 3y ago
- bb88 3y agoAnyone have metrics on this on performance in webgpu vs running this natively?
- aislamov 3y agoUNET takes about a 1:10 on WebGPU and around a minute on CPU in one thread. VAE is 2 minutes on CPU and about 10 seconds on GPU. It should be because most GPU ops for VAE are already implemented but for UNET are not. So in the latter case browser is just tossing data from GPU to CPU and back on each step
- seeknotfind 3y agoIf this is fast enough, then you could use it to render images locally for personal use. Websites could deliver prompts only, perhaps rendering different images for different users. At that point, what does it mean for copyrights? Is the model itself copyrighted or does the system break down?
- schoen 3y ago> If this is fast enough, then you could use it to render images locally for personal use. Websites could deliver prompts only, perhaps rendering different images for different users. That's a fascinating possibility, but we're very far from that world right now: elsewhere in the thread it's mentioned that this actively uses 8 GB of RAM. And I doubt many web designers would accept the risk that a model misinterprets a prompt, produces distorted output (like the wrong number of fingers on someone's hands), or accidentally produces sexual or violent content in a context where it's not intended. For many generative image models today, people often pick the best of a dozen or more images, and the others that they throw away may actually be quite bad. The quality and predictability of the models would need to be significantly higher than it is now in order to routinely dynamically illustrate web sites. But I don't want to say that we'll never get there. All of the recent models are doing things now that would have been considered inconceivable just a few years ago. (Compare https://xkcd.com/1425/ https://xkcd.com/1425/ where it may even be a challenge to explain the issue behind the joke to some younger readers!)
- yboris 3y agoLink to repo from the FAQ at the bottom of the page: https://github.com/dakenf/stable-diffusion-webgpu-minimal https://github.com/dakenf/stable-diffusion-webgpu-minimal
- DustinBrett 3y agoI integrated MLC-AI's Stable Diffusion into my website as a custom background generator. It grabs the model data from Hugging Face. https://dustinbrett.com/ https://dustinbrett.com/
- taf2 3y agoThat is really impressive loaded on my iPhone 12 mini and my phone was not over heating
- mNovak 3y agoHow smooth and fast this is just makes me sad that every app on my phone requires 200MB to show a couple views and send API calls.
- stavros 3y agoExactly. We have supercomputers in our pockets, yet showing a simple todo list takes seconds. Where does all the power go?!
- archerx 3y agoOver engineered frameworks that try to solve everyone’s problems at the same time in exchange for terrible performance.
- INGSOCIALITE 3y agoPeople don’t code for performance. Every application is built on top of “framework” bloat. Remember when people would boast about how little lines of code their program had or how little memory their programs used?
- franzb 3y agoImpressive website, everything just works super nicely and smoothly! Ended up playing Doom more than I should have :)
- aussieguy1234 3y agoNext up....WebGPU chatbot? ChatGPT style chat with your own GPU in the browser?
- brucethemoose2 3y agoAlso already done... With some limitations. LLMs are especially tricky for WebGPU because the good models are so RAM/VRAM heavy.
- moffkalast 3y agohttps://webllm.mlc.ai/ https://webllm.mlc.ai/ Already done.
- baddash 3y agoOut of curiosity, what are use cases/applications of this? So what I know is that this generates images via browser rather than server. The only thing I can think of is not having to refresh the page in order to change an image or generate a new image. Which... hmm, well, that could mean websites whose visual design changes in real-time? And maybe changes in a way that would be functionally relevant/useful? That does seem pretty cool, although I'm not sure how useful Stable Diffusion is for generating UI components/visual aspects of a site.
- 8n4vidtmkvmk 3y agoIt's useful because it's expensive to generate these images en masse. It moves the compute cost to the client.
- ShamelessC 3y agoWhat? The use case is to easily run it on your own computer. Without needing to be a programmer or install an application.
- asynchronous 3y agoJust like what most web apps allow- quick applications in a sandboxed environment on almost every type of hardware
- lionkor 3y agoAny hardware! As long as that hardware is overpowered for the job, so that the browser overhead is acceptable. Oh and it needs internet. Oh and it needs a reasonably large screen because padding and margins. Oh and it needs quite a bit of RAM to start. Maybe not any hardware.
- deleted 3y ago[deleted]
- Eduard 3y agoPrivacy is a big plus for (purely) locally running models. Sensitive prompts will not leak to some remote party.
- 3cats-in-a-coat 3y ago"This will load 3.5GB and use 8GB of your RAM". Interesting what browsers have become. The web ate the operating systems.
- pjmlp 3y agoThat is why there is a company selling laptops where the OS is the browser, and another one doing the same for smart TVs.
- solardev 3y agoWish Windows would take a cue from that other operating system and stop shipping with so much cruft. Can't remember the last time I used Windows for anything more than launching Chrome and Steam...
- pjmlp 3y agoActually Windows was one of the first to follow upon this idea with Active Desktop and packaged Web applications. I guess you would be better off with SteamOS then.
- solardev 3y agoWe still have that in the form of PWAs. I don't mean the web "runtime"/webview, but all the cruft that Windows ships with... the endless ads, multiple UI and config layers, Office trial, the stupid games, OneDrive, Skype/Teams, the endless notifications, Bing everywhere... it's an over-monetized in your face nightmare on every fresh boot. If not for DirectX and Windows-only games, I'd totally ditch it. Maybe when Proton gets there.
- scioto 3y agoI only use Windows for games.
- 3cats-in-a-coat 3y agoThere's however the important detail that this company has been doggedly working for achieving that end by first co-opting the browser of a competitor (Safari WebKit), then forking it, then taking over the web standards process, and putting in every API possible on the web, including access to USB devices and so on, so they can make an OS around it. Because if it's the web, Google sees it. And if everything is the web, then Google sees everything.
- Kiro 3y ago> Having DevTools open will slow everything down to about 2x. What causes this?
- capableweb 3y agoBrowser dev tools (and most of them really) hook into the application flow to do additional work compared to running it without any developer tools. Depending on what the application does, it can mean that it has to do a lot of extra work and need a lot of extra memory, just to be able to process and store all the extra information that it needs. I don't know the specifics of why the slowdown is so extreme in this case, usually it has a negligible impact. But I'm guessing it's related to what I wrote above.
- chris37879 3y agoAnother one that slows a lot of things down is if the application uses the console. Before you open the inspector, those methods are no-ops and just get skipped, basically, but once open, all of those strings have to get copied and it can slow things down quite a bit. This isn't unique to the web, either, adding the verbose flag to most linux file utilities and then operating on a large set of files will be slower than without the verbose flag, too, just because printing to stdout takes time.
- deleted 3y ago[deleted]
- kurishutofu 3y agoConsider using a service worker for the CPU bound process so the main thread doesn't hang.
- jb1991 3y agoSo I assume this is using WebGPU "compute" kernels, is that right?
- amelius 3y agoIt might as well mine some bitcoins while the GPU is not being used. Perhaps we could have an ad-free internet after all.
- k__ 3y agoAds create more overhead to pay all the executives. A miner would probably just generate enough work to pay the engineers and infrastructure.
- amelius 3y agoWhen will WebGPU come to Firefox? (Not going to install Chrome for a random shiny thing)