5 ms·
I’m not an expert, but my understanding is that MTPs are smaller LLMs fine-tuned to "mimic" / predict a specific model’s response. It’s possible that the MTP yo
by ch_sm 15d ago
I’m not an expert, but my understanding is that MTPs are smaller LLMs fine-tuned to "mimic" / predict a specific model’s response. It’s possible that the MTP you’re using isn’t trained well enough on Qwen 3.8. What accept rate are you getting?
- nixon_why69 15d agoFor qwen, it's an additional transformer layer at the very back, it ships as part of the model.
- jwr 11d agoAcceptance rate is good, but MTP doesn't help in my case because of my machine's memory bandwidth constraints (M4 Max). Turn out it's better to turn MTP off.