5 ms·
This account has prepackaged and quantized models for everything, really awesome. https://huggingface.co/TheBloke https://huggingface.co/TheBloke
by hfjjbf 3y ago
This account has prepackaged and quantized models for everything, really awesome.
https://huggingface.co/TheBloke https://huggingface.co/TheBloke
- brucethemoose2 3y agoIndeed. But TheBloke doesn't (yet) do exl2 quantization. That one is almost better done yourself, as the bpw target is arbitrary, and requires profiling text that should probably fit your intended use (though many just use wikitext as a good default).