5 ms·
Yeah, the same ideas should work for qwen. You can try porting this engine to use Owen. Owen 3.6-35b-a3b was my initial idea, but I switched to Gemma because o
by gitpusher42 2mo ago
Yeah, the same ideas should work for qwen. You can try porting this engine to use Owen.
Owen 3.6-35b-a3b was my initial idea, but I switched to Gemma because of its simpler architecture and kernels
- febed 2mo agoCurious if the same idea could work with gpt-oss-120b? So one could run at least slowly on a Mac
- gitpusher42 2mo agoYeah, gpt-oss-120b is also MoE, so the same ssd-streaming and caching ideas should work. Feel free to fork and try implementing it!