6 ms·

This is a great build, thanks for sharing your learnings. The best build I have seen so far had 6x4090's. Video: https://www.youtube.com/watch?v=C548PLVwjHA ht

by gzer0 2y ago

This is a great build, thanks for sharing your learnings. The best build I have seen so far had 6x4090's. Video: https://www.youtube.com/watch?v=C548PLVwjHA https://www.youtube.com/watch?v=C548PLVwjHA Specifications - GPU Accelerator - 6 x 24GB NVIDIA GeForce RTX 4090 - Processor - Intel Xeon W7-3465X, 28C/56T, 2.5GHz - 4.8GHz - Memory - 256GB (8x32GB) DDR5 ECC 4800MHz - System Drive - 2TB Samsung 980 PRO NVMe PCIe 4.0 M.2 SSD - Storage Drive - 4TB Samsung 870 EVO SSD - Operating System - Ubuntu 20.04 An interesting choice to go with 256GB of DDR5 ECC; if spending so much on the 6x4090's, might as well try to hit 1 TB of RAM as well. The cost of this... not even sure. Astronomical.

manmal 2y ago
On Reddit there's reports of 8x4090, or even 8xH100. I don't know where people get this kind of money for this, and why they don't rent infra instead.
- FridgeSeal 2y ago
  Probably because they are after a lot of fast, local storage, and _that_ is where rented ML infra providers will sting you. Edit: could also just be more-money-than-sense. Never discount stupidity.
  - dogma1138 2y ago
    Hardware can be resold, and bought 2nd hand also. The 4090 will likely maintain 50% of its current value due to its memory capacity over the next 12-18 months. CapEx vs OpEx is a thing even if you are not a business…
- KuriousCat 2y ago
  Why do you think RoI is better when infra is rented?
- taskforcegemini 2y ago
  I mean, at some point someone has to buy them to be able to offer services on them to others. Renting comes with certain limitations owners don't have. And some people have too much money to not invest in fun.
belter 2y ago
Don't forget to talk to your local power company one year in advance. They will need to upgrade your local substation transformer... :-)
- amluto 2y ago
  This build is 3kVA max. That’s about 1/3 of a current gen EV, only 15% of an original Tesla Model S with dual chargers, and about equal to a standard American oven. This is much more polite to the grid than, say, a couple of tea kettles or especially a reasonably sized electric tankless water heater.
  - dkkergoog 2y ago
    [dead]

by gzer0 2y ago

manmal 2y ago

On Reddit there's reports of 8x4090, or even 8xH100. I don't know where people get this kind of money for this, and why they don't rent infra instead.

FridgeSeal 2y ago
Probably because they are after a lot of fast, local storage, and _that_ is where rented ML infra providers will sting you. Edit: could also just be more-money-than-sense. Never discount stupidity.
- dogma1138 2y ago
  Hardware can be resold, and bought 2nd hand also. The 4090 will likely maintain 50% of its current value due to its memory capacity over the next 12-18 months. CapEx vs OpEx is a thing even if you are not a business…
KuriousCat 2y ago
Why do you think RoI is better when infra is rented?
taskforcegemini 2y ago
I mean, at some point someone has to buy them to be able to offer services on them to others. Renting comes with certain limitations owners don't have. And some people have too much money to not invest in fun.

belter 2y ago

Don't forget to talk to your local power company one year in advance. They will need to upgrade your local substation transformer... :-)

amluto 2y ago
This build is 3kVA max. That’s about 1/3 of a current gen EV, only 15% of an original Tesla Model S with dual chargers, and about equal to a standard American oven. This is much more polite to the grid than, say, a couple of tea kettles or especially a reasonably sized electric tankless water heater.
- dkkergoog 2y ago
  [dead]