6 ms·
Great release Daniel. Applaud the consistency you have shown. Can you release slightly bigger quant versions? Would enjoy something that runs well on 8x32 v10
by bradfox2 2y ago
Great release Daniel. Applaud the consistency you have shown.
Can you release slightly bigger quant versions? Would enjoy something that runs well on 8x32 v100 and 8x80 A100.
- danielhanchen 2y agoThanks! Oh I did release 4bit quants, 5bit, 6bit etc all at https://huggingface.co/unsloth/DeepSeek-R1-GGUF https://huggingface.co/unsloth/DeepSeek-R1-GGUF if that helps - they're not dynamic though but it should function fine :)