6 ms·
That's really impressive and much more performant. I was following a different approach: to run any ONNX model without prior modifications.
by aislamov 3y ago
That's really impressive and much more performant. I was following a different approach: to run any ONNX model without prior modifications.
- FL33TW00D 3y agoONNX is bloated! I got some LLMs working on my own Rust + WebGPU framework a few months ago: https://summize.fleetwood.dev/ https://summize.fleetwood.dev/ I've since moved away from ONNX and to a more GGML style.
- taminka 3y agowhat's the difference between onnx and ggml style?
- FL33TW00D 3y agoONNX consumes a .onnx file, which is a definition of the network and weights. GGML instead just consumes the weights, and defines the network in code. Being bound to ONNX means moving at a slower velocity - the field moves so fast that you need complete control.
- michaelmior 3y agoI haven't used ONNX or GGML, but presumably using GGML means you need to reimplement the network architecture?
- FL33TW00D 3y agoYou do! But it offers quite a fluid API making it pretty simple. You can see my attempt at a torchesque API here: https://twitter.com/fleetwood___/status/1679889450623459328 https://twitter.com/fleetwood___/status/1679889450623459328
- naillo 3y agoDo you have any good resources or links on using ggml with wasm?
- bkitano19 3y agoHey! This is what I've been working on, would love to chat, feel free to email
- FL33TW00D 3y agoSure! My email is in my profile.
- FL33TW00D 3y agoI think the Whisper example is your best bet! https://github.com/ggerganov/whisper.cpp/tree/master/examples/talk.wasm https://github.com/ggerganov/whisper.cpp/tree/master/example...