6 ms·
> The apple computer owner will probably be running local models that are better than today’s frontier on the same hardware. Hardware is not magically getting
by SXX 15d ago
> The apple computer owner will probably be running local models that are better than today’s frontier on the same hardware.
Hardware is not magically getting more memory or bandwidth.
Believing there will be some magical optimizations to compensate for it is just dellusion.
- chlorion 15d agoThen explain how equal parameter size models can grow in capability every few months or year?
- srcreigh 15d agoOpen weight models have been getting better/smaller every year. Also, from what I can tell, MLX inference is not as well optimized as CUDA, and the M5 Ultra has additional kinds of AI compute which is unavailable on other M models. With the massive 1.2 TB/s 512GB Mac studios coming out, I think MLX will get a lot more attention. In short: Todays models should run faster next year, and next year's models should also be more efficient.