6 ms·
crypto gpus didnt run at 100% power draw so the strain wasn't that massive
by rxyz 2mo ago
crypto gpus didnt run at 100% power draw so the strain wasn't that massive
- azeemba 2mo agoWhy weren't crypto gpus running at 100%?
- dantillberg 2mo agoMost crypto mining on GPUs would use 100% of memory bandwidth, but only a fraction of the compute available. This is a consequence of ASIC resistance of their mining algorithms -- custom silicon can only offer a modest benefit over GPUs if the hard part is memory bandwidth.
- NuclearPM 2mo agoWhy is that true? Can’t you just make more stuff parallel and shrink the ASIC chips accordingly?
- jmalicki 2mo agoNo - inability to do so is part of the design of a good cryptographic hash, quite explicitly. https://en.wikipedia.org/wiki/Avalanche_effect https://en.wikipedia.org/wiki/Avalanche_effect
- noosphr 2mo agoSo does llm inference. You're lucky if you hit 40% of the advertised flops.
- fwipsy 2mo agoRight, but datacenter GPUs optimized for LLM training/inference would have a bandwidth:compute ratio scaled to that workload.
- noosphr 2mo agoNo they don't.
- eru 2mo agoWhy not? Seems like they would be poorly optimised?
- noosphr 2mo agoBecause llm inference is not the only workload a GPU can do and custom silicon cost $10b a chip.
- eru 2mo agoAI optimised cards is the biggest cashcow for nvidia. They are definitely doing custom silicon. And Google et al have cards that don't even pretend to be able to do graphics.
- noosphr 2mo agoThere is more than one workload in AI. Inference for llms is memory constrained on even a single card. Training for llms is memory constrained on the level of racks. In both cases you hardly ever see more than 40% of advertised flops used.
- eru 2mo agoSo the question is: why are they designing AI cards so unbalanced?
- dapperdrake 2mo agoThen where are the HDMI ports on Nvidia's current data center GPU product lines?
- fishgoesblub 2mo agoMost were powerlimited to a degree to get the most hash/watt out of them.
- 3eb7988a1663 2mo agoTo advertise the maximum possible performance numbers, CPUs and GPUs massively shoot up the power draw for that final few percentage points. If you are worried about actual calculations/$, cutting back on the power will save your energy bill for practically no loss. When I tested "ECO" mode on my AMD CPU, performance was ~97% of regular mode, and temperatures dropped 5-10C. I think ECO mode drops TDP from something like 100W->65W. Cheaper and cooler to run for basically no cost. If you are bitcoin mining or selling GPU capacity, you are incredibly conscious of your energy bill, so it only makes sense to optimize the power draw.
- skeptic_ai 2mo agoLast point should also be for llm. Not just crypto
- fc417fc802 2mo agoNot true. A B200 "only" draws 1 kW. At 16¢ USD / kWh that's $1400 / year. When the hardware costs multiple tens of thousands you might be looking at a yearly electricity bill of ~3% of that. And that's before accounting for the datacenter itself. The difference is that much of the hardware crypto was running on was dirt cheap in comparison. Add to that a lot of crypto was being run on discount gear in cobbled together setups. Some people were even stealing electricity which often involved installing large clusters of relatively cheap devices in places without proper cooling.
- jpc0 2mo agoCapex != Opex Your are measuring two very different things. And you are only considering electricity bills for the GPU. Higher TDP also means more heat which means more cooling. It's not really easy to make an off hand comment about costs. A 30-40% decreaae in electricity usage and heat output for a fleet of GPUs definitely makes a massive difference to your opex bottom line.
- dapperdrake 2mo ago
- metalliqaz 2mo agoI bought a RTX 3070 off a miner when Eth went to proof of stake. It was clean, cheap, and is still going strong for daily gaming. In its working life it was undervolted and probably cooled better than in my rig.
- BizarroLand 2mo agoI got an entire Prebuilt PC when POW ended from a miner. AMD 5950x, 64gb ram, 3090, 2tb SSD, all for $1700. This was 4 years ago when the 3090 was $1500 by itself, and with minor upgrades it's still going strong today. It's wild to think that the system now is worth at least as much as I paid for it then if not much more than that. I saw a similar one going for $2500.
- latchkey 2mo agowe over clocked and under volted... the goal was to lower power and run the clocks as fast as possible.