6 ms·
I'm hoping too that they'll put out some MoE variants. Qwen3.5:122b:a10b can run about twice as fast as this 27b dense model. Edit: Like its predecessors, 3.8
by Casteil 1mo ago
I'm hoping too that they'll put out some MoE variants.
Qwen3.5:122b:a10b can run about twice as fast as this 27b dense model.
Edit: Like its predecessors, 3.8 seems really inclined to overthinking, and on a 27b dense model that's kind of painful.
I think I'm going to stick with gemma4:26b-a3b as my go-to because it runs about 4x as fast and tends to only need a fraction of the tokens in its 'thinking' stage to get the same or similar answer.
- Phemist 1mo agoDid you try the claude reasoning traces finetune for qwen3.6? I find that it works muuch better. I assume the same 3.8 finetune will be released at some pointas well. Edit: link - https://huggingface.co/rico03/Qwen3.6-27B-Claude-Opus-Reasoning-Distilled https://huggingface.co/rico03/Qwen3.6-27B-Claude-Opus-Reason...
- satvikpendem 1mo agoReduce or turn down thinking: https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates
- Casteil 1mo agoYeah, that's probably the answer given that it apparently defaults to 'xhigh'.
- dannyw 1mo agoProbably helps it score a little bit better in benchmarks :) `medium` seems like a nice balance so far; along with some light steering to vary think effort as needed for task and being pragmatic.
- wgd 1mo agoInterestingly 'medium' is the closest thing the _model itself_ has to a default thinking level. The chat template injects directions [1] at the very start of the system message when the reasoning effort is 'xhigh' or 'low' but 'medium' implicitly just means no added reasoning-level instructions. [1] The specific directions are "Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer." and "Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration."
- mirekrusin 1mo ago[flagged]
- isomorphic 1mo ago"inclined to overthinking," holy cats you're not kidding! On a Mac mini M4 Pro 64GB I prompted it with "svg owl" and it thought for 17m12s, outputting 36.3KiB of thinking chatter. It did end up producing a 20.2KiB HTML+JS+SVG file with a very nice owl, including cursor-tracking animation, but it ran for more than a half hour! The MoE models are stupendously faster.
- kzrdude 1mo agoI think that's called test-time scaling i.e using more tokens at infer time to squeeze out higher model performance. That's must be part of the explanation for good benchmark results.
- Casteil 1mo agoYep.. it's pretty obnoxious for real-world use with the default 'xhigh' thinking. Ridiculous amount of "Wait, actually.." which might help for complex coding tasks but makes it unbearable for general purpose use.
- try-working 1mo agothat seems to be how most Chinese models achieve increased benchmark scores. GLM and Kimi models are "thinkslop" models that reason over their own thinking, which increases cost and decreases speed significantly. That's why GPT is in a different tier altogether - faster, smarter, and sometimes cheaper.
- nubg 1mo agowhat harness did you use?
- isomorphic 1mo agoNo agentic harness; I just prompted literally "svg owl" into LM Studio 0.4.21+2. macOS Tahoe 26.6.1, Mac mini M4 Pro (14 core, 20 graphics cores) 64GB RAM. (I put the machine config so the 17m12s of thinking has some context.) It got about 12.75 tok/s and used 21,769 tokens. For comparison's sake, Qwen 3.6 35B A3B MLX, same prompt, same machine, thought for 1.59 seconds, used 2,398 tokens, and hit 80.83 tok/s. It actually output just a straight SVG file of an owl (not a bad owl either!), whereas Qwen 3.8 27B spent a zillion times longer gold-plating everything and output HTML+JS+SVG with interactive animation, sound, etc., etc.