24 ms·
I have a few private “vibe check” questions and the 4 bit QAT 27B model got them all correctly. I’m kind of shocked at the information density locked in just 13
by Samin100 1y ago
I have a few private “vibe check” questions and the 4 bit QAT 27B model got them all correctly. I’m kind of shocked at the information density locked in just 13 GB of weights. If anyone at Deepmind is reading this — Gemma 3 27B is the single most impressive open source model I have ever used. Well done!
- itake 1y agoI tried to use the -it models for translation, but it completely failed at translating adult content. I think this means I either have to train the -pt model with my own instruction tuning or use another provider :(
- andhuman 1y agoHave you tried Mistral Small 24b?
- itake 1y agoMy current architecture is an on-device model for fast translation and then replace that with a slow translation (via an API call) when its ready. 24b would be too small to run on device and I'm trying to keep my cloud costs low (meaning I can't afford to host a small 24b 24/7).
- jychang 1y agoTry mradermacher/amoral-gemma3-27B-v2-qat-GGUF
- itake 1y agoMy current architecture is an on-device model for fast translation and then replace that with a slow translation (via an API call) when its ready. 24b would be too small to run on device and I'm trying to keep my cloud costs low (meaning I can't afford to host a small 27b 24/7).