6 ms·
It's GPU performance. Spin up ollama and run some inference on your 5-year-old intel macbook. You won't see 4000x performance improvement (because performance
by condiment 11mo ago
It's GPU performance.
Spin up ollama and run some inference on your 5-year-old intel macbook. You won't see 4000x performance improvement (because performance is bottlenecked outside of the GPU), but you might be in the right order of magnitude.
- jandrese 11mo agoComparing GPU performance to some half decade old Intel IGP seems like lying with statistics. "Look how many times faster our car is![1]" [1] Compared to a paraplegic octogenarian in a broken wheelchair!"
- deleted 11mo ago[deleted]
- School-Cotton 11mo agoWell, Apple isn’t making that comparison, the OP was.
- blihp 11mo agoNot possible given the anemic memory bandwidth [1]... you can scale up the compute all you want but if the memory doesn't scale up as well you're not going to see anywhere near those numbers. [1] The memory bandwidth is fine for CPU workloads, but not for GPU / NN workloads.