7 ms·
Hoping this is real. It's too bad to see the signals from Qwen that they will not be releasing a 35B-A3B for the 3.8 lineup. The MoE architecture makes a huge
by montroser 28d ago
Hoping this is real. It's too bad to see the signals from Qwen that they will not be releasing a 35B-A3B for the 3.8 lineup. The MoE architecture makes a huge difference for being able to run these local models on reasonable consumer hardware.
- parsimo2010 28d agoHonest question/suggestion for the HN audience- Since Qwen released the weights for Qwen3.8 2.4T-A95B and we already have the staring point of Qwen3.6 35B-A3B, couldn't someone distill the bigger model and make a "pseudo" Qwen3.8 35B-A3B? Sure, it wouldn't be an official Qwen release but couldn't someone improve on Qwen 3.6 and get the thing everyone is asking for? I am calling this a suggestion for the audience because I don't have the will/resources to do this.
- WASDx 28d ago"Qwen3.8 35B-A3B" and 4B/9B variants are already on huggingface distilled by hobbyists.
- karlkloss 27d ago>I can't find any. Do you have a link?
- KronisLV 27d agohttps://huggingface.co/Lord-H4D3ZS/Qwen3.8-Distill-35B-A3B-Coder-Abliterated https://huggingface.co/Lord-H4D3ZS/Qwen3.8-Distill-35B-A3B-C... > Base / architecture: Qwen/Qwen3.6-35B-A3B (Qwen3_5MoeForCausalLM, 256 experts, ~3B active). The "3.8" in the name refers to the teacher, not the base. Not endorsement, haven't run it myself, just found the link.
- parsimo2010 26d ago[dead]
- halJordan 28d agoYes, of course. But no one really wants to be the guy actually renting an entire B300.
- mirekrusin 28d agoPersonally I find speculative decoding much better strategy than MoE – performance wise it's there at 90-100 t/s on 2x4090, great intelligence – really great fit.
- d4rkp4ttern 28d agoA lot of people, including me, don’t want to bother with GPUs, they’d rather run it on their M1-M5 MacBook. For example the 35B-A3B is very usable even on a M1 64GB MacBook.
- mirekrusin 28d agoSpeculative decoding also works on Mac, 64G is more than what I have, m5 max should handle up to ~40 t/s with optimized setup (and with a lot of vram you can get great wins on concurrency – that harness can take advantage of for single user task as well), but agree memory bandwidth in mac or spark is still too slow, next gen for both will be great hardware to have for sure.
- smcleod 28d agoI get around 70tk/s on the m5 max, with 5bit AWQ / oQ5 slowing only to around 40tk/s at higher context.
- c0m47053 28d agoMoE is great on systems that lack the VRAM to host the full model. On my 16GB VRAM system, I can get 100 tok/s with Q4 Qwen 3.6 35b a3b, and 15 tok/s with 27b. MTP is a trade-off, as it pushes some more of the model off the GPU. I have managed to get usable quants of Laguna S2 and even DeepSeek V4 flash on this setup. There is clearly some intelligence loss compared to similar sized dense models, but I feel like it stomps on the 9-12b models I could run fully on GPU
- dofm 28d agoQuick tests suggest it’s pretty good at reasoning and tool use (keen to search to check its thinking) and it seems to waste much less time thinking, too. So it feels very fast. But it does not seem to be better than Qwen 3.6 35B at coding. A bit worse, I think, though I will test it more. If you have a machine that can fit a 35B model in VRAM, I would suggest testing Muse Glimmer with (from memory) Reasoning strength: low in the system prompt. Despite being a dense model, this is actually capable of solving code problems faster than the Qwen MoE, despite having only one fifth of the raw token performance.
- verdverm 28d agoI'm running qwen3.8 27B dense on reasonable hardware (oem spark) tbh, I have stopped using MoE in the name of speed, the dense (with more active parameters) makes a real difference in output quality
- smcleod 28d agoThe smarter 27B is so fast with MTP I've found I really don't need the 35B-A3B. You get around 70tk/s on a M5 Max lowering to around 40tk/s at higher context sizes.
- seanmcdirmid 28d agoI've benched 3.8 27B being significantly slower and less quality than 3.6 35B-A4B (both 4-bit quant, MTP, both using turboquant 4-bit served by oMLX), to the point that I'm not even using it right now (on an M3 Max). What's your use case and what did you observe? I might be missing something.
- smcleod 28d agoI believe you mean 35B-A3B, there was no such thing as A4B. I use 27B and other models for software development, and quite a few research or similar agents. I cannot imagine a world where the old 35B-A3B model is smarter / more capable than 3.8 27B - the difference is night and day for coding at least. Where 35B-A3B was fast and felt like a Haiku model, 27B feels like a strong Sonnet when given the right tools. I don't use turbo quant so can't comment on that, but with the A3B model you're using you probably won't get much from using MTP with small MoE models like that.
- xscott 28d agoPeople over-quantize things, muck with the temperature and other settings based on superstitions or results from models they think are similar. There's lots of ways to make 3.8 27B dumber.
- razster 28d agoI've had great success with allowing the model to review optimal settings and my system specs. It comes up with the right configuration. Running Pi harness. I just had Ornith 1.5 take a moment to configure itself, now it's reviewing a large project I'm working on, so far its really impressive for my needs. Qwen3.8 27b Unsloth(Dynamic 3.0) is also perfect. These two are working together, and I'm in a sweet-spot, I now have all I need.
- hgoel 28d agoThe way it was said made it sound like they had something better than a 35B-A3B coming.
- huseyinkeles 28d agoI've been experimenting with it on a M4 Pro 24G for the last few hours and it's been very promising using 32k context. getting around 30-40 tps With Qwen3.8 27B I could not get anywhere near 32k context window, that made it very unusable for agentic coding, although it was very smart.
- vkaku 28d agoThis is all real. More real things coming soon.
- Sha1rholder 28d ago> The MoE architecture makes a huge difference for being able to run these local models on reasonable consumer hardware That's not true. For computers without unified memory architecture (which is the vast majority) VRAM capacity is the bottleneck for local models. In that case a dense model can deliver significantly more intelligence than an MoE model of the same size. And for a typical consumer/gamer Nvidia GPU, dense models are fast enough.
- slim 28d agollama.cpp can run MoE with some layers in vram and some layers in ram
- dannyw 28d agoExpert offloading significantly helps with the VRAM capacity. Most MoE architectures have a few experts that are always running; this, the router, KV, and whatever else you have space for can stay in fast VRAM; and the remaining experts can be offloaded.
- regexorcist 28d agoWhat signals? Looking more like a tiered release.