6 ms·

I somehow find it better to give 2 frontier model companies 100-200/month than dropping 10 grand on a hardware that will get old in no time with bad TPS. I real

I somehow find it better to give 2 frontier model companies 100-200/month than dropping 10 grand on a hardware that will get old in no time with bad TPS. I really want to have a fully local model but seems like one more generation wait and we will be there?

tyleo 23d ago
Lots of people use the Mac Mini to run the frontier models over night or while traveling. I have a rack in my basement and have thought about throwing one in. You can get a cheaper machine but Apple feels a little more, “rack and forget,” if you have less price sensitivity. Mac Mini + MacBook Neo w/ ssh can be a better setup than MacBook Pro for many people.
- throwaway219450 22d ago
  If it’s purely for experimentation then why not the DGX Spark/GB10? It’s up about 10% from release RRP which is quite good (you might argue it was overpriced then, but prosumer and workstation GPU prices are up 100%). 4TB NVMe is not cheap these days - it’d cost at least $500 for a stick - and you get 128GB at a similar bandwidth to an M5 Pro. Nevermind that Apple still insists on providing base systems with only 512GB of non-upgradable SSD. The equivalent spec mini (4TB/64/10G) is almost $5k for half the VRAM. Not as good a CPU compared to the M5/6 but you also get 20 cores and full CUDA.
  - tyleo 22d ago
    That does not sound like rack and forget. Just looking at prices it’s also way more expensive.
henry2023 23d ago
You just described why the datacenter business is hard and as a corollary why space datacenters will not be economically viable.
jameshart 22d ago
You could spend 100 a month leasing one of these for three years and get the best of both worlds
- chis 22d ago
  You’re still stuck paying for a 3 year contract for deprecating hardware. And the 256gb model which is still not enough is $230 a month. And it runs models 8 months behind frontier

tyleo 23d ago

Lots of people use the Mac Mini to run the frontier models over night or while traveling. I have a rack in my basement and have thought about throwing one in. You can get a cheaper machine but Apple feels a little more, “rack and forget,” if you have less price sensitivity. Mac Mini + MacBook Neo w/ ssh can be a better setup than MacBook Pro for many people.

throwaway219450 22d ago
If it’s purely for experimentation then why not the DGX Spark/GB10? It’s up about 10% from release RRP which is quite good (you might argue it was overpriced then, but prosumer and workstation GPU prices are up 100%). 4TB NVMe is not cheap these days - it’d cost at least $500 for a stick - and you get 128GB at a similar bandwidth to an M5 Pro. Nevermind that Apple still insists on providing base systems with only 512GB of non-upgradable SSD. The equivalent spec mini (4TB/64/10G) is almost $5k for half the VRAM. Not as good a CPU compared to the M5/6 but you also get 20 cores and full CUDA.
- tyleo 22d ago
  That does not sound like rack and forget. Just looking at prices it’s also way more expensive.

henry2023 23d ago

You just described why the datacenter business is hard and as a corollary why space datacenters will not be economically viable.

jameshart 22d ago

You could spend 100 a month leasing one of these for three years and get the best of both worlds

chis 22d ago
You’re still stuck paying for a 3 year contract for deprecating hardware. And the 256gb model which is still not enough is $230 a month. And it runs models 8 months behind frontier