5 ms·
I've found it tends toward long thinking loops even for simple tasks (and any quantization seems to increase their length), but those do exit eventually, unlike
by Balinares 21d ago
I've found it tends toward long thinking loops even for simple tasks (and any quantization seems to increase their length), but those do exit eventually, unlike with Qwen 3.6.
I use the Unsloth UD_Q2_K_XL GGUF with default parameters, along with that custom template linked elsewhere in the thread, and no K/V cache quantization.
- alfiedotwtf 20d agoFor smaller models, you'll probably find anything below Q4 will need handholding. Check Unsloth's graphs at the different quantisations VS error rates and you'll see why.
- Balinares 14d agoThank you, I know about those. And I'll stick with this quant. Normally I'd be with yout there, but Qwen 3.8 is turning out to be good at self-correcting, and the free memory I can use for extra context is worthwhile.