5 ms·
> gemma (unsloth/gemma-4-26B-A4B-it-GGUF) models Since you're running quantized (at UD-Q4_K_XL) , check out the "qat" models (unsloth/gemma-4-26B-A4B-it-qat-G
by kpw94 3mo ago
> gemma (unsloth/gemma-4-26B-A4B-it-GGUF) models
Since you're running quantized (at UD-Q4_K_XL) , check out the "qat" models (unsloth/gemma-4-26B-A4B-it-qat-GGUF) !
- https://huggingface.co/unsloth/gemma-4-26B-A4B-it-qat-GGUF https://huggingface.co/unsloth/gemma-4-26B-A4B-it-qat-GGUF
(With "Jun 9 Update: Added MTP support.")
- https://blog.google/innovation-and-ai/technology/developers-tools/quantization-aware-training-gemma-4/ https://blog.google/innovation-and-ai/technology/developers-...
- me_bx 3mo agoTIL: > Quantization-Aware Training (QAT) [...] allows preserving similar quality to bfloat16 while dramatically reducing the memory requirements to load the model
- SubiculumCode 3mo agoHow is the the QAT models at coding? I looked for opinions since the release and haven't found much.