6 ms·
Grok4 was trained on 100k or 200k GPUs (as far as I understand) Grok5 might need 1MM or 2MM. So the question is what about metas / zucks plans? How many GPUs
by MichaelRazum 1y ago
Grok4 was trained on 100k or 200k GPUs (as far as I understand)
Grok5 might need 1MM or 2MM.
So the question is what about metas / zucks plans? How many GPUs will Manhattan get? Looks like, that to get the next unlock you need crazy amounts of compute.
- jiggawatts 1y agoMeta had the equivalent of about 600K H100 cards a year ago, but they were geographically distributed and used mostly for inference. These giant data centres will allow these companies to put about a million in one location and possibly into a single giant training cluster.