blints

7 ms·

1.
by blints 5d ago
"How dare people I don't like give away software"
2.
by blints 21d ago
One does not simply "buy a few b200". They come in 8 packs minimally, at >5x that budget.
3.
by blints 22d ago
I don't know that "consumer hardware" is a useful distinction anymore, it's just "what's your budget and what's your speed requirement".
4.
by blints 22d ago
It can be more than "slightly", particularly if the model you're interested in (or will be interested in in 6 months) doesn't fit on the mac studio. You also need to account for eg storing 10TB of random checkpoints, lo
5.
by blints 23d ago
The relevant comparison isn't one mac studio to one RTX 6000, it's a 24 channel DDR5 system, which also has ~1.2TB/s of memory bandwidth (or more when Xeon 6 compatible 8800mt/s memory becomes widely available), vastly h
6.
by blints 23d ago
It's unclear to me how bandwidth scales with multiple connections. Many-to-many does not seem ideal. Daisy chaining would be fine for straight pipeline work. There doesn't seem to be an equivalent of a ethernet switch for thund
7.
by blints 23d ago
It is. That's the mac mini. For local LLMs you would want the Mac Studio, which tops out at 1.2TB/s.
8.
by blints 23d ago
10 grand for 256GB memory. Likely double that for 512GB, but won't be available or finalized until October. Thunderbolt 5 is highest bandwidth external IO available at 120Gb/s. 1.2TB/s claimed max internal memory bandwidth
9.
by blints 2mo ago
Their claim is even stronger than that, they have complaints about their models being used as a validation step for other model output, which is standard practice in the industry.
10.
by blints 3mo ago
Most companies do not model themselves as "building on [AI model du jour]" yet. They model themselves as building products with those tools, which they consider as relatively substitutable.