6 ms·
Yes. MLC's inference engine runs on WebGPU/WASM. https://github.com/mlc-ai/web-llm-chat https://github.com/mlc-ai/web-llm-chat https://github.com/mlc-ai/mlc-l
by SparkyMcUnicorn 1y ago
Yes. MLC's inference engine runs on WebGPU/WASM.
https://github.com/mlc-ai/web-llm-chat https://github.com/mlc-ai/web-llm-chat
https://github.com/mlc-ai/mlc-llm https://github.com/mlc-ai/mlc-llm
https://github.com/mlc-ai/web-llm https://github.com/mlc-ai/web-llm
- mg 1y agoYeah, something like that, but without the WebGPU requirement. Neither FireFox nor Chromium support WebGPU on Linux. Maybe behind flags. But before using a technology, I would wait until it is available in the default config. Lets see when browsers will bring WebGPU to Linux.
- SparkyMcUnicorn 1y agoThis should be what you're looking for. It doesn't utilize the GPU, but WebGL support is in the TODOs. https://github.com/ngxson/wllama https://github.com/ngxson/wllama https://huggingface.co/spaces/ngxson/wllama https://huggingface.co/spaces/ngxson/wllama
- simonw 1y agoFirefox Nightly on macOS now supports WebGPU, and the documentation says the Linux build supports it too.