5 ms·
Plus, the speed at which it replies is amazing too. Claude/Chatgpt now seem like inefficient inference engines compared to it.
by attentionmech 2y ago
Plus, the speed at which it replies is amazing too. Claude/Chatgpt now seem like inefficient inference engines compared to it.
- IanCal 2y agoI've been trying through openrouter today and it seems quite slow, but this may just be a scaling thing. I tried the chat when it first came out and that was extremely fast.
- attentionmech 2y agoYea, they might be scaling is harder or may be more tricks up their sleeves when it comes to serving the model.