5 ms·
I thought they were saying it was more efficient, as in tokens per watt. I didn’t see a direct comparison on that metric but maybe I didn’t look well enough.
by binary132 8mo ago
I thought they were saying it was more efficient, as in tokens per watt. I didn’t see a direct comparison on that metric but maybe I didn’t look well enough.
- nl 8mo agoProbably. Companies sell on efficiency when they know they lose on performance.
- tormeh 8mo agoIf you have an efficient chip you can just have more of them and come out ahead. This isn't a CPU where single core performance is all that important.
- avereveard 8mo agoEh if there's a human on the other side single stream performance is going to matter to them.
- fennecfoxy 8mo agoOnly if the price is right...
- binary132 8mo agoRight, but datacenters also very much operate on electrical cost so it’s not without merit.