5 ms·

Spark: OS: Windows/Ubuntu Mbw: 300GB/s Cuda cores: 6000 GPU accelerated containers: yes M5 max: OS: macOS Mbw: 600GB/s Cuda cores: 0 GPU acce

by moondev 4mo ago

Spark: OS: Windows/Ubuntu Mbw: 300GB/s Cuda cores: 6000 GPU accelerated containers: yes M5 max: OS: macOS Mbw: 600GB/s Cuda cores: 0 GPU accelerated containers: no

xyzzy123 4mo ago
I feel like the shape of the market right now for "home lab" inference is: The sparks are good if your ultimate plan is to spend even more on NVidia hardware in future to run your dev setups at usable speeds. Or, you're developing for a work cluster. If you mainly want to run local models at acceptable speeds portably, buy a mac with lots of RAM. If you’re happy with non-portable / racked, buy 3090s (dense) or mac studios (MoEs). Buy newer cards if you are restricted on power or slots. If you are rich, buy a6000 blackwells.