7 ms·
TLDR: Don’t buy H100s. The market has flipped from shortage ($8/hr) to oversupplied ($2/hr), because of reserved compute resales, open model finetuning, and dec
by hislaziness 2y ago
TLDR: Don’t buy H100s. The market has flipped from shortage ($8/hr) to oversupplied ($2/hr), because of reserved compute resales, open model finetuning, and decline in new foundation model co’s. Rent instead.
Is the AI infra bubble already bursting?
- TechDebtDevin 2y agoNo, but the prices will likely converge with MSRP pricing. A lot of datacenter were filled with h100s that cost a premium to get ahold of.
- pico_creator 2y agoCovered in the article. They are below MSRP essentially
- hislaziness 2y agoIt is not just MSRP, management and operations cost too. The article goes into the details of this.
- pico_creator 2y agoQ_Q yes - ur right on that - and i wrote the article (about a month ago)
- pico_creator 2y agoI’m hopping more for an open weights AI boom With cheap compute for everyone to finetune :)
- justahuman74 2y agoYes, please only rent instead - sincerely, all of the cloud providers
- pico_creator 2y ago~Cough~ not all cloud provider (there are many still willing to charge you an arm and a leg) Only the ones who can give you below MSRP essentially
- Sohcahtoa82 2y agoAn H100 is what, $50,000 MSRP? At $2/hr, that's 2.8 years to RoI. And that's just for the GPU and not the other hardware you'll need to plug it into, and doesn't include the power, and also assumes you're using it 100% of the time. Really, you're probably looking at 3.5+ years to RoI. I'd rather rent than buy in that scenario.
- swyx 2y ago(editor here) we've been commenting on the Winds of AI Winter for a while now :) https://latent.space/p/mar-jun-2024 https://latent.space/p/mar-jun-2024