6 ms·

I have a question, is the short lifespan of GPUs because they get worn out and are destroyed, or because they get outdated by the ever expanding demands of the

by Drakim 4mo ago

I have a question, is the short lifespan of GPUs because they get worn out and are destroyed, or because they get outdated by the ever expanding demands of the AI bubble? Because if it's the later, I would assume that growth would not continue at the same rate after the bubble bursts?

throwawayffffas 4mo ago
They get worn out. Training workloads have high utilization high thermals and eventually things degrade and break.
- progval 4mo ago
  Are there estimates of their failure rate?
  - throwawayffffas 4mo ago
    From toms hardware, the figures look like 27% fail after 3 years. https://www.tomshardware.com/pc-components/gpus/datacenter-gpu-service-life-can-be-surprisingly-short-only-one-to-three-years-is-expected-according-to-unnamed-google-architect https://www.tomshardware.com/pc-components/gpus/datacenter-g...
gravypod 4mo ago
It's, from my understanding, a little bit of both. There's a failure rate of GPUs and fans. There's also changing in standards like PCIe and software stacks. LLM inference is mainly memory bandwidth constrained so I think it's highly likely that a company will create silicon with just an insane number of memory chips and less compute. These ASICs will probably do the same thing the crypto ASICs did. If we look back 1 decade, no one uses a GTX 950 for anything.
- joefourier 4mo ago
  You'd be surprised, people are somehow buying Tesla P40s and M40s on eBay for almost $300 and $180 respectively (M40 being the same gen as GTX 950). Google Colab still offers T4s and it's taken them years to add modern GPUs. Hope they're powering them with renewables at least. And people in general are holding on to their old machines for very long periods of time now, especially CPUs. I've had to support first gen Intel i7s at work! That's pre AVX.
  - gravypod 4mo ago
    Just a note, P40 came out at $5700 in 2016 dollars. In 2026 dollars that is $8000 (wow!). If you bought 100k today, assuming a 1% failure rate per year your $800M investment can be traded in for about $30M. I think it is reasonable to assume a similar depreciation in GPUs. Meaning you'd need to have made more than (800M - 30M) * (1 + income tax rate) + (power + maintenance). Some say the margines on inference are already there for new GPUs but they are right margines.
joefourier 4mo ago
Outside of training the biggest LLMs at big labs, GPU lifespan isn't as short as the OP made it out to sound. A100s are 6 years old and still a reliable work-horse, and the 80GB version hasn't depreciated that much on the used market. On the consumer side, 3090s are actually still selling for very close to 2020 MSRP. Even the ancient V100 (soon to be 10 years old!) had somewhat of resurgence on the second-hand market, with a healthy market for interconnects in China. If I had a datacenter and power consumption was not a concern, I'd be holding on to my A100s for years at least for inference.
- throwawayffffas 4mo ago
  Oh yeah, not meant to be all doom and gloom. Lighter workloads greatly increase hardware lifespan. And the GPUS are like at most 50% of the data-center cost I think. You get to keep the building, the cooling, the power interconnects, the networking and everything else. Additionally the demand drives new power infrastructure, and new fabs that will definitely outlive the bubble.
pjc50 4mo ago
As with compute hardware, someone will have a chart keeping track of "additional electricity cost per unit of compute versus state-of-the-art hardware", to determine when it's cheaper to just turn it off and replace with newer hardware.

by Drakim 4mo ago

throwawayffffas 4mo ago

They get worn out. Training workloads have high utilization high thermals and eventually things degrade and break.

progval 4mo ago
Are there estimates of their failure rate?
- throwawayffffas 4mo ago
  From toms hardware, the figures look like 27% fail after 3 years. https://www.tomshardware.com/pc-components/gpus/datacenter-gpu-service-life-can-be-surprisingly-short-only-one-to-three-years-is-expected-according-to-unnamed-google-architect https://www.tomshardware.com/pc-components/gpus/datacenter-g...

gravypod 4mo ago

It's, from my understanding, a little bit of both. There's a failure rate of GPUs and fans. There's also changing in standards like PCIe and software stacks. LLM inference is mainly memory bandwidth constrained so I think it's highly likely that a company will create silicon with just an insane number of memory chips and less compute. These ASICs will probably do the same thing the crypto ASICs did. If we look back 1 decade, no one uses a GTX 950 for anything.

joefourier 4mo ago
You'd be surprised, people are somehow buying Tesla P40s and M40s on eBay for almost $300 and $180 respectively (M40 being the same gen as GTX 950). Google Colab still offers T4s and it's taken them years to add modern GPUs. Hope they're powering them with renewables at least. And people in general are holding on to their old machines for very long periods of time now, especially CPUs. I've had to support first gen Intel i7s at work! That's pre AVX.
- gravypod 4mo ago
  Just a note, P40 came out at $5700 in 2016 dollars. In 2026 dollars that is $8000 (wow!). If you bought 100k today, assuming a 1% failure rate per year your $800M investment can be traded in for about $30M. I think it is reasonable to assume a similar depreciation in GPUs. Meaning you'd need to have made more than (800M - 30M) * (1 + income tax rate) + (power + maintenance). Some say the margines on inference are already there for new GPUs but they are right margines.

joefourier 4mo ago

Outside of training the biggest LLMs at big labs, GPU lifespan isn't as short as the OP made it out to sound. A100s are 6 years old and still a reliable work-horse, and the 80GB version hasn't depreciated that much on the used market. On the consumer side, 3090s are actually still selling for very close to 2020 MSRP. Even the ancient V100 (soon to be 10 years old!) had somewhat of resurgence on the second-hand market, with a healthy market for interconnects in China. If I had a datacenter and power consumption was not a concern, I'd be holding on to my A100s for years at least for inference.

throwawayffffas 4mo ago
Oh yeah, not meant to be all doom and gloom. Lighter workloads greatly increase hardware lifespan. And the GPUS are like at most 50% of the data-center cost I think. You get to keep the building, the cooling, the power interconnects, the networking and everything else. Additionally the demand drives new power infrastructure, and new fabs that will definitely outlive the bubble.

pjc50 4mo ago

As with compute hardware, someone will have a chart keeping track of "additional electricity cost per unit of compute versus state-of-the-art hardware", to determine when it's cheaper to just turn it off and replace with newer hardware.