6 ms·

At $7.2k + tax: * RAM - $1500 - Crucial Pro 128GB Kit (2x64GB) DDR5 RAM, 5600MHz CP2K64G56C46U5, up to 4 sticks for 128GB or 256GB, Amazon * GPU - $4700 - RTX

At $7.2k + tax: * RAM - $1500 - Crucial Pro 128GB Kit (2x64GB) DDR5 RAM, 5600MHz CP2K64G56C46U5, up to 4 sticks for 128GB or 256GB, Amazon * GPU - $4700 - RTX Pro 5000 48GB, Microcenter * CPU/Mobo bundle - $1100 - AMD Ryzen 7 9800X3D, MSI X870E-P Pro, ditch the 32GB RAM, Microcenter * Case - $220, Hyte Y70, Microcenter * Cooler - $155, Arctic Cooling Liquid Freezer III Pro, top-mount it, Microcenter * PSU - $180, RM1000x, Microcenter * SSD - $400 - Samsung 990 pRO 2TB gen 4 NVMe M.2 * Fans - $100 - 6x 120mm fans, 1x 140mm fan, of your choice Look into models like Qwen 3.5

cmxch 6mo ago
Surprised to see X3D given the reports of failures. I’ve opted for a regular 9900x and X670E-E just to have a bit more assurance.
aurareturn 6mo ago
$7.2k just to run at best Qwen3.5-35B-A3B doesn't seem worth it at all. This is certainly not the most effective use of $7k for running local LLMs. The answer is a 16" M5 Max 128GB for $5k. You can run much bigger models than your setup while being an awesome portable machine for everything else.
- emidoots 6mo ago
  Performance (tok/s and PP) or quality (model size)? Pick one. In terms of GPU memory bandwidth (models fitting in the ~48GB of RTX 5000 Pro card), the RTX card I described above has over 2x the bandwidth of an M5 Max. If leveraging system RAM (the 128GB-256GB outside the GPU) to run larger models, then the memory bandwidth is ~6x slower than M5 Max. For models fitting in the ~48GB RTX memory, like dense Qwen3.5 27B models, the RTX will be 2-4x faster than M5 Max. For models that don't fit in the 48GB RTX memory, the M5 Max will be 5-20x faster. Also worth considering future upgrades: Do you plan to throw away the machine in a few years, or pick up multiple used RTX 6000 Pro cards when people start ditching them?
ac29 6mo ago
> RAM - $1500 - Crucial Pro 128GB Kit (2x64GB) DDR5 RAM I knew prices went up, but that's wild. I bought 64GB (2x32) of RAM a year ago for $90.

cmxch 6mo ago

Surprised to see X3D given the reports of failures. I’ve opted for a regular 9900x and X670E-E just to have a bit more assurance.

aurareturn 6mo ago

$7.2k just to run at best Qwen3.5-35B-A3B doesn't seem worth it at all. This is certainly not the most effective use of $7k for running local LLMs. The answer is a 16" M5 Max 128GB for $5k. You can run much bigger models than your setup while being an awesome portable machine for everything else.

emidoots 6mo ago
Performance (tok/s and PP) or quality (model size)? Pick one. In terms of GPU memory bandwidth (models fitting in the ~48GB of RTX 5000 Pro card), the RTX card I described above has over 2x the bandwidth of an M5 Max. If leveraging system RAM (the 128GB-256GB outside the GPU) to run larger models, then the memory bandwidth is ~6x slower than M5 Max. For models fitting in the ~48GB RTX memory, like dense Qwen3.5 27B models, the RTX will be 2-4x faster than M5 Max. For models that don't fit in the 48GB RTX memory, the M5 Max will be 5-20x faster. Also worth considering future upgrades: Do you plan to throw away the machine in a few years, or pick up multiple used RTX 6000 Pro cards when people start ditching them?

ac29 6mo ago

> RAM - $1500 - Crucial Pro 128GB Kit (2x64GB) DDR5 RAM I knew prices went up, but that's wild. I bought 64GB (2x32) of RAM a year ago for $90.