5 ms·
Do current models run on the NPU or GPU? Wondering if Apple will have something like a TPU.
by lifty 23d ago
Do current models run on the NPU or GPU? Wondering if Apple will have something like a TPU.
- bhouston 23d agoApple has a dedicated "neural engine" which is designed as an inference NPU. Where as Google's TPU has a dual focus, both inference and training, which is a more complex design.
- lifty 23d agoThat TPU training part I get, but from what I have seen the NPU is rarely used for inference by LLMs. They still use the GPU, no?