5 ms·

Thanks!! I had disabled that previously while debugging, I can confirm this is helping accuracy from what I can tell so far. (And speed since the cache is prese

by rayboy1995 4mo ago

Thanks!! I had disabled that previously while debugging, I can confirm this is helping accuracy from what I can tell so far. (And speed since the cache is preserved more often!)

satvikpendem 4mo ago
Use the MTP models which 2x token generation speed, for example: https://unsloth.ai/docs/models/qwen3.6#mtp-guide https://unsloth.ai/docs/models/qwen3.6#mtp-guide
- rayboy1995 4mo ago
  Very interesting I'll have to check this out thank you. This is why I love HN.