6 ms·

A lot of folks are/were saying that Apple has missed the boat when it comes to AI, in some (important) aspects that is correct, but I think it's worth rememberi

A lot of folks are/were saying that Apple has missed the boat when it comes to AI, in some (important) aspects that is correct, but I think it's worth remembering that Apple added the Neural Engine to A-series chips in 2017, before the AI hoopla really kicked off: * https://en.wikipedia.org/wiki/Neural_Engine https://en.wikipedia.org/wiki/Neural_Engine * https://apple.fandom.com/wiki/Neural_Engine https://apple.fandom.com/wiki/Neural_Engine "AI" has grown much more since then, and there have been important developments that Apple has not deployed (well), but I think they were looking ahead a little more than most at the time (even if events 'got away' from them subsequently).

echelon 4d ago
It may not matter that they're "sleeping on it". AI is going to disrupt all tech companies. Clearly software and SaaS has no moat. I just saw someone vibe code a fully functional Photoshop in a week of Astra use. I'm thinking even big tech is not immune to this. Even the mighty infrastructure players. Vibe hardware is just getting started, and I'm hoping we see competitors to iPhone and Android and MacBook. It's less expensive to build and test things now, which might lead to a Cambrian explosion of new hardware startups. The world will greatly benefit from this. It's funny that American tech giants invented AI, because AI is going to unseat American tech giants.
- tensor 4d ago
  > It's funny that American tech giants invented AI, because AI is going to unseat American tech giants. Ah yes, let's ignore that 40 years of research done on neural networks and give all the credit instead to the tech giants that came in as soon as there was enough of a breakthrough to start making them useful in language tasks. Definitely the American tech giants and not the Canadian and international researchers who spent their lives building the foundations. /s
- tpmoney 4d ago
  SaaS' moat has always been someone else worrying about the hosting/upgrades/management part for you. Stores still sell sliced bread even though bread machines make bread for you and everyone already has a knife. People often buy convenience, even when they could do something themselves.
gleenn 4d ago
IMHO, I think Apple is poised extremely well. They didn't blow billions of dollars chasing models that are becoming commoditized. So many interesting models can now be run locally. All the big AI players have to pay even more to run those models when Apple will happily sell you the hardware, and you pay for the electricity. There will always be a place for some many-billion parameter model but as time progresses I think fast local models that keep data on site will always be a valuable, and Apple will happily sell you something to run them.
mathisfun123 4d ago
> but I think it's worth remembering that Apple added the Neural Engine to A-series chips in 2017 but missing the boat means you haven't succeeded not that you had some kind of initial foothold. haven't you seen the big short: "i may have been early but i'm not wrong" "it's the same thing" https://www.youtube.com/watch?v=pLLgNi5UmB0&t=116s https://www.youtube.com/watch?v=pLLgNi5UmB0&t=116s
bigyabai 4d ago
Apple's Neural Engine is equivalent to the "NPU" hardware you would find on many cheap ARM SBCs. It's a dedicated coprocessor with a very low performance target, not intended for giant transformers or LLM acceleration. These NPUs are not particularly hard to design, and their critical flaw is that they don't scale very well. CUDA "won" because a bigger GPU meant having an utterly massive amount of CUDA cores to delegate ALU work to. NPUs/Neural Engine has the opposite problem, where spending $10,000 on an M1 Ultra only gets you ~2x better NPU performance versus the baseline $600 M1 chip. NPUs and Neural Engines are essentially dark silicon on the majority of devices with them, their uses are few and far between. Considering Apple's refusal to sign Nvidia's ARM/CUDA drivers, it is pretty clear how Apple missed the boat here. Apple Silicon could have dominated the datacenter rollout if macOS supported CUDA properly. The Mac Pro would probably not have been cancelled if the PCI lanes could be used for normal datacenter GPUs and CUDA workloads. The excellent Thunderbolt bandwidth present on so many Macs is wasted supporting RDNA but not eGPU enclosures. There are several hardware features that Apple holds back for no good reason, handing Nvidia the lead in certain markets. The only thing that stopped Apple from riding AI to the top was their own petty grudge towards Nvidia. Simple changes to macOS would have destroyed Nvidia's Grace CPU sales and made Apple Silicon the crown prince of the AI boom, at zero risk to themselves. The only thing Apple really needed was their own Mellanox equivalent.

echelon 4d ago

It may not matter that they're "sleeping on it". AI is going to disrupt all tech companies. Clearly software and SaaS has no moat. I just saw someone vibe code a fully functional Photoshop in a week of Astra use. I'm thinking even big tech is not immune to this. Even the mighty infrastructure players. Vibe hardware is just getting started, and I'm hoping we see competitors to iPhone and Android and MacBook. It's less expensive to build and test things now, which might lead to a Cambrian explosion of new hardware startups. The world will greatly benefit from this. It's funny that American tech giants invented AI, because AI is going to unseat American tech giants.

tensor 4d ago
> It's funny that American tech giants invented AI, because AI is going to unseat American tech giants. Ah yes, let's ignore that 40 years of research done on neural networks and give all the credit instead to the tech giants that came in as soon as there was enough of a breakthrough to start making them useful in language tasks. Definitely the American tech giants and not the Canadian and international researchers who spent their lives building the foundations. /s
tpmoney 4d ago
SaaS' moat has always been someone else worrying about the hosting/upgrades/management part for you. Stores still sell sliced bread even though bread machines make bread for you and everyone already has a knife. People often buy convenience, even when they could do something themselves.

gleenn 4d ago

IMHO, I think Apple is poised extremely well. They didn't blow billions of dollars chasing models that are becoming commoditized. So many interesting models can now be run locally. All the big AI players have to pay even more to run those models when Apple will happily sell you the hardware, and you pay for the electricity. There will always be a place for some many-billion parameter model but as time progresses I think fast local models that keep data on site will always be a valuable, and Apple will happily sell you something to run them.

mathisfun123 4d ago

> but I think it's worth remembering that Apple added the Neural Engine to A-series chips in 2017 but missing the boat means you haven't succeeded not that you had some kind of initial foothold. haven't you seen the big short: "i may have been early but i'm not wrong" "it's the same thing" https://www.youtube.com/watch?v=pLLgNi5UmB0&t=116s https://www.youtube.com/watch?v=pLLgNi5UmB0&t=116s

bigyabai 4d ago

Apple's Neural Engine is equivalent to the "NPU" hardware you would find on many cheap ARM SBCs. It's a dedicated coprocessor with a very low performance target, not intended for giant transformers or LLM acceleration. These NPUs are not particularly hard to design, and their critical flaw is that they don't scale very well. CUDA "won" because a bigger GPU meant having an utterly massive amount of CUDA cores to delegate ALU work to. NPUs/Neural Engine has the opposite problem, where spending $10,000 on an M1 Ultra only gets you ~2x better NPU performance versus the baseline $600 M1 chip. NPUs and Neural Engines are essentially dark silicon on the majority of devices with them, their uses are few and far between. Considering Apple's refusal to sign Nvidia's ARM/CUDA drivers, it is pretty clear how Apple missed the boat here. Apple Silicon could have dominated the datacenter rollout if macOS supported CUDA properly. The Mac Pro would probably not have been cancelled if the PCI lanes could be used for normal datacenter GPUs and CUDA workloads. The excellent Thunderbolt bandwidth present on so many Macs is wasted supporting RDNA but not eGPU enclosures. There are several hardware features that Apple holds back for no good reason, handing Nvidia the lead in certain markets. The only thing that stopped Apple from riding AI to the top was their own petty grudge towards Nvidia. Simple changes to macOS would have destroyed Nvidia's Grace CPU sales and made Apple Silicon the crown prince of the AI boom, at zero risk to themselves. The only thing Apple really needed was their own Mellanox equivalent.