6 ms·
What do you mean by draft model? And how would one disable it? Cheers
by marcalc 1y ago
What do you mean by draft model? And how would one disable it? Cheers
- _neil 1y agoA draft model is something that you would explicitly enable. It uses a smaller model to speculatively generate next tokens, in theory speeding up generation. Here’s the LM Studio docs on it: https://lmstudio.ai/docs/app/advanced/speculative-decoding https://lmstudio.ai/docs/app/advanced/speculative-decoding