5 ms·
Towards Optimal LLM Quantization
- aa6864aa 2y agoHow does it compare with AWQ, SqueezeLLM, or newer quantization methods?
- aviel 2y agoAny benchmarks with Falcon 2?
- bejager 2y agowe don't support Falcon 2 yet but new models are always on our radar to be added to the platform.
- deleted 2y ago[deleted]
- eonlav 2y agoDecent platform support - any plans for a Rust SDK?
- bejager 2y agoWe continuously work on expanding SDK support, Rust is also on the list.
- dynamix 2y agoIs there a way for me to compress a custom fine-tuned model of my own?
- bejager 2y agonot yet but it's something we have in mind as a future feature.
- abcd98 2y agoHow do you integrate with vLLM?