6 ms·
I deeply crave prosumer hardware that can sit on my shelf and handle massive models, like 200-400B at a reasonable quant. Something like Groq or Digits but at t
by IAmNotACellist 1y ago
I deeply crave prosumer hardware that can sit on my shelf and handle massive models, like 200-400B at a reasonable quant. Something like Groq or Digits but at the cost of a high-end gaming PC, like $3k. This has to be a massive market, considering that even ancient Pascal-series GPUs that were once $50 are going for $500.
- almostgotcaught 1y ago> This has to be a massive market It's not - it's absolutely a vanishingly small market.
- renewiltord 1y agoAt home people would rather use the cloud.
- deleted 1y ago[deleted]
- numa7numa7 1y agoNvidia's working on it. 200B at $3k https://www.nvidia.com/en-us/products/workstations/dgx-spark/ https://www.nvidia.com/en-us/products/workstations/dgx-spark...
- zozbot234 1y ago> I deeply crave prosumer hardware that can sit on my shelf and handle massive models, like 200-400B at a reasonable quant. So, an Apple Mac Studio?
- darksaints 1y agoI have that irresistible urge too, but I have to keep reminding myself that I could spend $2000 in credits over the course of a year, and get the performance and utility of a $40k server, with scalable capacity, and without any risk that that investment will be obsolete when Llama5 comes out.
- sofixa 1y agoThe Framework Desktop is one not absurdly expensive option. The memory speed isn't great (200 something GB/s), but any faster with those requirements at least doubles the price (e.g. a Mac Studio, only the highest tier M chips have faster memory).