6 ms·
I love the idea of openrouter. I hadn't realized until recently though that you don't necessarily know what quantization a certain provider is running. And of c
by pickettd 2y ago
I love the idea of openrouter. I hadn't realized until recently though that you don't necessarily know what quantization a certain provider is running. And of course context size can vary widely from provider to provider for the same model. This blog post had great food for thought https://aider.chat/2024/11/21/quantization.html https://aider.chat/2024/11/21/quantization.html
- avereveard 2y agoTo expand a little, some providers may apply more aggressive optimization in periods of high load.