6 ms·
I am so excited for Qwen 3.8 27B. It’s a shame how slow prefill (~3-400) is on a strix halo but it’s such a good model for agentic tasks.
by syntaxing 1mo ago
I am so excited for Qwen 3.8 27B. It’s a shame how slow prefill (~3-400) is on a strix halo but it’s such a good model for agentic tasks.
- tarr11 1mo agoWhat type of agentic tasks are you using it for (eg how complex)?
- syntaxing 1mo agoFor personal stuff, I use it with AnythingLLM. It replaced any Google search for me. For coding, I run opencode though I have been debating switching to Pi. I would argue it’s at Sonnet 3 level.
- CamperBob2 1mo agoHow are you running it on a Strix Halo? The weights aren't out yet, are they?
- 13rac1 1mo agoI interpret @syntaxing as meaning they are looking forward to running Qwen3.8-27B, but are frustrated by prefill times with other models, such as Qwen3.6-27B.
- syntaxing 1mo agoI meant Qwen3.6. Unsloth supposedly has early preview of the model and the VRAM requirement is the same so most people expect similar model size and type.
- LoganDark 1mo agoI find that 35B-A3B is much easier to run on my M4 Max (both prefill and generation)
- markasoftware 1mo agoIt's well known 35b is much faster (on any hardware) and quite a bit dumber
- dofm 1mo agoThis really very much depends on how you are using it, I think. If you intend to leave it to solve long context problems and write whole prototypes, the 27B is going to be much better. But if you are sort of pair-programming with the model, the speed obviously matters and I think then the 35B is acceptably smart, and when it's wrong it'll be wrong much more quickly. It seems very good on SQL and PHP, and I assume on typical JS and Python. I would rather work that way, so I hope they do produce a small MoE model.
- LoganDark 1mo agoI had no idea. Where can we learn stuff like this?
- markasoftware 1mo agoLearn about MoE models and also just look at the benchmarks of the two models. For example https://artificialanalysis.ai/models/comparisons/qwen3-6-27b-vs-qwen3-6-35b-a3b https://artificialanalysis.ai/models/comparisons/qwen3-6-27b... clearly shows both the intelligence and speed differences. It's a bit degenerate but you can get some useful info from e.g. the /r/localllama subreddit
- colingauvin 1mo agoPrefill is survivable if you cache well. But what kills me is the context. Qwen 27 needs a ton of room for KV Cache. I guess not an issue on a 128 GB Halo or Spark, but if you are running of consumer/prosumer GPUs it's miserable to be compacting every 120k tokens.