7 ms·
I'd consider a 1tb machine at 20k, but I'm not going to pick up a 256gb one at all. 1TB fits a frontier-ish model in memory without massive quantization, which
by jdcasale 23d ago
I'd consider a 1tb machine at 20k, but I'm not going to pick up a 256gb one at all. 1TB fits a frontier-ish model in memory without massive quantization, which is a very interesting capability for a non-rack piece of compute.
- epolanski 23d agoBut why...? At that point just rent proper GPUs in the cloud, you'd have way more power and pay only what you use for.
- albrewer 23d agoBecause I spent $15k in AI costs last month doing real work at my day job; I'd like to do the same thing for myself but I don't like lighting cash on fire.
- jdcasale 22d agoI think that within the next year or two we'll have the ability to put a frontier-class model on our desks, and I like the idea of playing around with that today to get ahead of it. I could be wrong, I often am. I'm not optimizing for $$$ spent today and agree that this isn't the cheapest solution to a specific capability problem, but I don't super mind paying for a cool piece of compute that I can do some fun (and educational) stuff with.