7 ms·
Unfortunately there's no gguf quants of the assistant model yet: https://huggingface.co/models?other=base_model:quantized:google/gemma-4-12B-it-assistant https:
by thomasjb 4mo ago
Unfortunately there's no gguf quants of the assistant model yet:
https://huggingface.co/models?other=base_model:quantized:google/gemma-4-12B-it-assistant https://huggingface.co/models?other=base_model:quantized:goo...
- kristjansson 4mo agoI think MTP Gemma4 support is still WIP https://github.com/ggml-org/llama.cpp/pull/23398 https://github.com/ggml-org/llama.cpp/pull/23398 ?
- thot_experiment 4mo ago[dead]
- dofm 4mo agoThis has been my impression. The underlying LiteRT-LM framework used in the edge gallery does support the MTP drafters for the smaller models, but according to: https://developers.google.com/edge/litert-lm/models/gemma-4 https://developers.google.com/edge/litert-lm/models/gemma-4 > Note: LiteRT-LM supports E2B and E4B models today, with support for larger models coming soon. So even Google aren't shipping MTP support for the 26B and 31B models yet.
- kristjansson 3mo agoUpdate: not any more!