5 ms·
These GPUs are extremely inflated in price because Nvidia effectively has a monopoly on hardware that is used to train models. Apple Silicon tends to have good
by teaearlgraycold 14d ago
These GPUs are extremely inflated in price because Nvidia effectively has a monopoly on hardware that is used to train models. Apple Silicon tends to have good inference software available but as soon as you want to train even a YOLO model bits and pieces fall back to software implementations. Try to train an LLM and it'll get even worse.
The M3 Ultra's GPU performance is around a 4070 Ti. The M5 Ultra more like a 5080. They're both amazing deals compared to Nvidia for local inference because of their massive pool of high bandwidth memory. But a single RTX PRO 6000 should be 2 or 3x the compute of an M5 Ultra.
- segmondy 14d agoI can't stand Nvidia, but these GPUs are not inflated because of monopoly. They are expensive because demand exceeds supply. That's it. The world wants compute and we don't have enough of it!
- teaearlgraycold 14d agoI said inflated. They would always be expensive. But they charge more than their competitors per flop and per byte because they’re the only manufacturer that can run CUDA. If they lost that moat their prices would go down a notch.