4 ms·
The numbers are pretty incredible. Will the competition be able to match them?
by smsx 2y ago
The numbers are pretty incredible. Will the competition be able to match them?
- shwaj 2y agoGroq is claiming 284 tokens/second on Llama 3.1 70b, so they’re in the same ballpark. https://groq.com/12-hours-later-groq-is-running-llama-3-instruct-8-70b-by-meta-ai-on-its-lpu-inference-enginge/ https://groq.com/12-hours-later-groq-is-running-llama-3-inst...
- wmf 2y agoIf Groq 2 is 2x faster it will match Cerebras WSE-3.