5 ms·
There is no "future proof" for >1T param models, there is no present or future where you can run a model that size on consumer hardware.
by root_axis 23d ago
There is no "future proof" for >1T param models, there is no present or future where you can run a model that size on consumer hardware.
- bewareofscams 23d ago6 Mac Studios for 100k USD. Consider it a rule of thumb now - 100k to run 1T params, scales linearly.
- MPSimmons 23d agoOut of curiosity, what do you do to distribute the weights and inference across the multiple machines?
- fzzzy 23d agoThunderbolt 5, and software
- NamlchakKhandro 22d agoat a 100k$ you're not buying apple products to be limited by the 120gb/s thunderbolt port.
- root_axis 23d agoIf you have 100k you can just buy a few b200s.
- blints 22d agoOne does not simply "buy a few b200". They come in 8 packs minimally, at >5x that budget.
- blints 23d agoI don't know that "consumer hardware" is a useful distinction anymore, it's just "what's your budget and what's your speed requirement".
- kridsdale1 23d agoConsumer hardware means: - 120v input plug - not rack-mounted - has a video out port
- iAMkenough 23d agoAh, a MacBook Professional is Consumer /s
- kridsdale1 23d agoYes?
- iAMkenough 23d agoPro?
- oblio 23d agoWant to bet that consumers with enough money buy it?
- deleted 23d ago[deleted]
- kllrnohj 23d agoApple's use of "Professional" is just a fancy alias for high end or premium, it in no way indicates anything about being "for professionals" (most extreme example: "professional" iPhone models) Hence why they had to make up the "Studio" brand for the workstation market, because they'd already fully removed any meaning from "Professional"
- nl 23d agoIn computing "Pro" is not the opposite of "Consumer". Putting aside the fact it is a marketing label, "Pro" usually means "designed for work" while "Consumer" (in this context) means "doesn't need a special environment". In computing the distinction is primarily noise, power and cooling requirements. If a computer is designed to use home power and is quiet enough to use without annoying people and doesn't require specialist cooling then it is a consumer device, even if it is used for work.