6 ms·

This is genuinely very helpful. I'm planning a MacBook pro purchase with local inference in mind and now see I'll have to aim for a slightly higher memory optio

This is genuinely very helpful. I'm planning a MacBook pro purchase with local inference in mind and now see I'll have to aim for a slightly higher memory option because the Gemma A4 26B MoE is not all that!

tomr75 5mo ago
pretty sure Nvidia GPU is better bang for buck because of usable inference speed..
egorfine 5mo ago
I have upgraded my M4 Pro 24GB to M5 Pro 48GB yesterday. The same Gemma 4 MoE model (4bit, don't remember which version) runs about 8x faster on M5 Pro and loads 2x times faster in memory. So yes, do purchase that new MacBook Pro.
- croemer 5mo ago
  You don't know if it's the newer model or the increase in RAM. If someone has already got 48GB it they might not benefit much. You changed 2 things at once.
  - egorfine 5mo ago
    Not really: it's the same model size and it fits 24GB entirely.
59nadir 5mo ago
If you're doing it specifically for inference (or in most other situations) a Mac(book) represents very low RoE.
- 59nadir 5mo ago
  s/RoE/RoI