5 ms·
Hopefully in this world, someone figures out how to deliver the performance equivalent of a B300 GPU for about 1/100th the power of a current B300 (which can be
by not2b 16d ago
Hopefully in this world, someone figures out how to deliver the performance equivalent of a B300 GPU for about 1/100th the power of a current B300 (which can be up to 1400 watts), or the world will bake.
- avaer 16d agoAnalog AI/approximate computing? There's research into it, worth looking at if you're interested. The idea is you tolerate some loss/degradation (which neural networks do) but gain orders of magnitude power efficiency.
- i5heu 16d ago8 biillion * 1400 watts = 11.2 terawatts current global average electricity production = approximately 3.6 terawatts But then again these systems do not use 1400 watts all the time. We would probably have much more then 11.2 terawatts demand if all humans turn on all their electrical consumers at the same time.
- not2b 16d agoSo demand quadruples if they are always on, and doubles if they are on 1/3 of the time. Either way, too much.
- sriniwasx 16d agoSpecialized chips like what cereberus is building combined with small models can give great result in the long run, but at the pace AI is growing it's not very practical yet.
- tossandthrow 16d agoCompared to 20-30kw spend on cruising the highway in a car (considerably higher for older ICEs) 1.4kw does not really seem to dent the energy consumption. Especially if cognitive technologies mean that we need to travel less (eg communiting to work, or ineffecient supply chains).
- eleventen 15d agoWhat percentage of your time is spent cruising at highway speed? Presumably that one GPu would be saturated all the time.
- paimapi 15d ago2-3 hours at 20kW daily (assuming commute, etc) vs 20ish hours always on tasks really is close to equivalent which is to say that the last thing our planet needs is another universalized technology that outputs as much total emissions as cars in an ideal world, we'd keep LLMs/CNNs/etc specialized and academic until we are hitting diminishing returns on optimizing fundamental microprocessor tech like GAA. but the pursuit of market dominance and mass adoption is our current operating philosophy, and so we have things like this top graph: https://hai.stanford.edu/news/inside-the-ai-index-12-takeaways-from-the-2026-report https://hai.stanford.edu/news/inside-the-ai-index-12-takeawa... >Grok 4's estimated training emissions reached 72,816 tons of CO2 equivalent, or roughly the same amount of greenhouse gas emissions created from driving 17,000 cars for one year
- tomaskafka 15d agoI think personalized suriveillance and manipulation (thought control, manipulating people through filtering of content they receive) on a level any dictator would drool over can be gotten for a fraction of 1 GPU per person.
- keysersoze33 15d agoMost of the energy consumption comes from moving data between memory and compute units rather than the math itself. While HBM stacked on a silicon interposer was a signification efficiency improvement over traditional DDR, there's still room for improvement. The high energy costs will drive this further, e.g. near-memory/in-memory compute (PIM) or tighter 3D packaging and possibly optical interconnects. (And of course training/model optimisations)
- threetonesun 15d agoRight, if everyone in the world has a GPU we would have solved so many problems with power generation and... I'm not going to do the math but I think we'd be mining asteroids too? I'd probably use it as a bookend at the point.