8 ms·
I work in the datacenter space. The power consumption of a data center is the "canonical" way to describe their size. Almost every component in a datacenter is
by elbasti 1y ago
I work in the datacenter space. The power consumption of a data center is the "canonical" way to describe their size.
Almost every component in a datacenter is upgradeable—in fact, the compute itself only has a lifespan of ~5 years—but the power requirements are basically locked-in. A 200MW data center will always be a 200MW data center, even though the flops it computes will increase.
The fact that we use this unit really nails the fact that AI is basically refining energy.
- pseudosavant 1y agoI love that term "refining energy". We need to plan for massive growth in electricity production to have the supply to refine.
- tmalsburg2 1y agoSounds smart but it’s abusing the semantics of “refine” and is therefore ultimately vacuous.
- pseudosavant 1y agoI think it is really just the difference between chemically refining something and electrically refining something. Raw AC comes in, then gets stepped down, filtered, converted into DC rails, gated, timed, and pulsed. That’s already an industrial refinement process. The "crude" incoming power is shaped into the precise, stable forms that CPUs, GPUs, RAM, storage, and networking can actually use. Then those stable voltages get flipped billions of times per second into ordered states, which become instructions, models, inferences, and other high-value "product." It sure seems like series of processes for refining something.
- reubenmorais 1y agoAll life is basically refining energy - standing up to entropy and temporarily winning the fight.
- HPsquared 1y agoIt's all about putting the entropy somewhere else and keeping your own little area organised.
- xnickb 1y agoPeople of the earth, remember: unnecessary arm and leg movements increase the entropy! Fear of the heat death of the universe! Lie down when possible!
- antihipocrat 1y agoYes, in a very local context it appears so, but net entropy across the system from life's activities is increased
- ithkuil 1y ago"the purpose of life is to hydrogenate carbon dioxide" -- Michael Russel
- aurareturn 1y agoA 200MW data center will always be a 200MW data center, even though the flops it computes will increase. This here underscores how important TSMC's upcoming N2 node is. It only increases chip density by ~1.15x (very small relative to previous nodes advancements) but it uses 36% less energy at the same speed as N3 or 18% faster than N3 at the same energy. It's coming at the right time for AI chips used by consumers and energy starved data centers. N2 is shaping up to be TSMC's most important node since N7.
- alberth 1y ago> N2 is shaping up to be TSMC's most important node since N7 Is it? N2, from an energy & perf improvement seems on par with any generation node update. N2:N3 N3:N5 N5:N7 Power ~30% ~30% ~30% Perf ~15% ~15% ~15% https://www.tomshardware.com/news/tsmc-reveals-2nm-fabrication-process https://www.tomshardware.com/news/tsmc-reveals-2nm-fabricati...
- aurareturn 1y agoYes. It has more tape outs at this stage of development than both N5 or N3. It’s wildly popular for chip designers it seems.
- alberth 1y agoI thought Apple gets exclusive access to the latest node for the first 1-2 years. Is that not the case?
- aurareturn 1y agoNo. That's not the case. Maybe for a few months only.
- alberth 1y agoCorrect me if I'm wrong but didn't TSMC launch N3 in 2022, and still only Apple uses this latest/smallest node. Both AMD and NVIDIA are using N4.
- kulahan 1y agoThat's pretty interesting. Is it just because the power channels are the most fundamental aspect of the building? I'm sorta surprised you can't rip out old cables and drop in new ones, or something to that effect, but I also know NOTHING about electricity.
- libraryofbabel 1y agoNot an expert, but it’s probably related to cooling. Every joule of that electricity that goes in must also leave the datacenter as heat. And the whole design of a datacenter is centered around cooling requirements.
- vrighter 1y agoExactly. To add to that, I'd like to point out that when this person says every joule, he is not exaggerating (only a teeny tiny bit). The actual computation itself barely uses any energy at all.
- jacquesm 1y agoIt is the opposite of refining energy. Electrical energy is steak, what leaves the datacenter is heat, the lowest form of energy that we might still have a use for in that concentration (but most likely we are just dumping it in the atmosphere). Refining is taking a lower quality energy source and turning it into a higher quality one. What you could argue is that it adds value to bits. But the bits themselves, their state is what matters, not the energy that transports them.
- elbasti 1y agoI think you're pushing the metaphor a bit far, but the parallel was to something like ore. A power plant "mines" electron, which the data center then refines into words. or whatever. The point is that energy is the raw material that flows into data centers.
- fuzzfactor 1y agoMaybe more like converting energy to data, as a more specific type of refinement.
- phkahler 1y agoUsing energy to decrease the entropy of data. Or to organize and structure data.
- fuzzfactor 1y agoI like that. Take random wild electrons and put them neatly into rows & columns where they can sit a spell.
- LaGrange 1y agoThis is OpenAI, they are not decreasing the entropy. This is refining coal into waste heat and CO2.
- protocolture 1y agoThis. A local to me ~40W datacenter used to be in really high demand, and despite having excess rack space, had no excess power. It was crazy.
- nixass 1y ago40W - is that ant datacenter? :)
- protocolture 1y agoYeah, it was the companies pilot site, and everything about it is tiny. But it very quickly became the best place in town for carrier interconnection. So every carrier wanted in. Even when bigger local DC's went in, a lot of what they were doing was just landing virtual cross connects to the tiny one, because thats where everyone was.
- inemesitaffia 1y agoYou lost a M or K next to your W. I still have an Edison bulb that consumes more power.
- protocolture 1y agoYep I see that haha.
- pabs3 1y ago> the power requirements are basically locked-in Why is that? To do with the incoming power feed or something else?
- djtriptych 1y agoguessing massive capital outlays and maybe irreversible site selection/preparation concerns.
- brendoelfrendo 1y agoBasically, yes. When you stand up something that big, you need to work with the local utilities to ensure they have the capacity for what you're doing. While you can ask for more power later on, if the utilities can't supply it or the grid can't transport it, you're SOL.
- pabs3 1y agoYou could in theory supplement it with rooftop solar and batteries, especially if you can get customers who can curtail their energy use easily. Datacentres have a lot of roof space, they could at least reduce their daytime energy costs a bit. I wonder why you don't see many doing solar, do the economics not work out yet?
- XorNot 1y agoCompared to their volume they absolutely do not: you get about ~1kW / m^2 of solar. Some quick googling suggests a typical DC workload would be about 50 kW / m^2, rising too 100 for AI workloads.
- brendoelfrendo 1y agoI'd have to do the math, but I doubt that makes sense given the amount of power these things are drawing. I've heard of DCs having on-site power generation, but it's usually in the form of diesel generators used for supplemental or emergency power. In one weird case, I heard about a DC that used on-site diesel as primary power and used the grid as backup.
- casey2 1y agoWhere do the cards go after 5 years? I don't see a large surplus of mid sized cloud providers coming to buy them (cause AI isn't profitable), Maybe other countries (possibly illegally)? Flood the consumer market with cards they can't use? TSMCs' more than doubled packaging and they are planning on doubling again
- pjc50 1y agoRefining it into what? Stock prices?