6 ms·
I can see this pattern of many people using Qwen 3.8 27B for local inference both on Apple Silicon and x86. This implies the model must be very good, given al
by larodi 5d ago
I can see this pattern of many people using Qwen 3.8 27B for local inference both on Apple Silicon and x86. This implies the model must be very good, given all these peoples' opinion converges on it.
- Scarjit 5d agoFor it's size it's pretty good. I run it and the older Qwen 3.6 35B A3B, which is significantly faster, but also a bit dumber, on both my PC and my MacBook. Wouldn't trust it for long form coding, but for shorter stuff it's really good.
- snek_case 5d agoHow far can you get with Qwen 3.6 35B A3B? What scope of task is it able to do?
- noir_lord 5d agoI've played with a fair few local models, Qwen 3.8 27B is the only one I've currently kept around (Muse Glimmer came the closest but will vacillate wildly if pushed, Qwen doesn't seem to do that (as much), I don't use it to generate code, it's essentially a glorified linter, it's pretty good at spotting dumb stuff you miss when you proofcheck your own code, especially things like comment drift where I alter a function and it makes the comment slightly (or sometimes wildly) out of sync with behaviour. It's fine for that (and I happen to have a 24GB VRAM GPU anyway since I game on the same PC). It's neat but for me not world changing. It's also just fun to be able to poke stuff and see what it can and can't do (but I could see how it could also become a time trap in cases where it gets kinda close and you want to fix that).
- suprjami 5d agoQwen 3.6 27B and 3.8 27B are the darlings of local inference at the moment. The only other thing anyone is using is Qwen 3.8 Flash Next, only by memory-rich people. Depending on which benchmarks you believe, these models (and the Ornith 1.5 finetune of Qwen 35B-A3B) are competitive at about Opus 4.5 to 4.7 level. That matches my experience in real tasks over the last few months. Not bad for something you can run at home for a couple of thousand dollars.
- nasutton12 5d agothe early 1.x versions of chad were tied to ornith. i still miss the speed of that moe. https://huggingface.co/nathansutton/Ornith-1.0-35B-UD-Q2_K_XL-MLX https://huggingface.co/nathansutton/Ornith-1.0-35B-UD-Q2_K_X...
- suprjami 5d agoI skimmed chad the other day. There's very little to it (by design). afaics you should be able to replicate what chad does by copying the chad system prompt into a SYSTEM.md for Pi. Then you could use it with whatever model/provider you want.
- nasutton12 5d agopi is a fantastic harness! they are a good default in the same way llama.cpp is. it works with everything and that is the point. i was steering chad in the opposite direction. one model & one set of silicon -> taken to the max. swap out your CHAD_MODEL and it still runs, you just leave the drafter and the kernels behind.
- Terretta 3d agoLove the idea… Except the "one model" is too small for a 64GB Mac much less 128GB, sad since the Q3 is proven less competent. Offering a Q boost (with no leave behinds) on first run would be a bump worth some vibe coding while.