8 ms·
Maybe Google is adding compute infra. I noticed today that the API has 500 tok/s throughput. It's quite compelling now.
by low_tech_punk 1mo ago
Maybe Google is adding compute infra. I noticed today that the API has 500 tok/s throughput. It's quite compelling now.