7 ms·
M4 Pro has been a solid performer for iterating on smaller local models. Much more convenient than spinning up cloud instances for dev.
by miles_io 15d ago
M4 Pro has been a solid performer for iterating on smaller local models. Much more convenient than spinning up cloud instances for dev.
- whatsThisBtn4 15d agoIf you just want chat. Agents require at least DeepSeek pro and even that is the minimum. You might be able to get a good model to write instructions and run it in smaller models. Otherwise, cool your AI got the current weather.
- EagnaIonat 15d ago> Agents require at least DeepSeek pro and even that is the minimum. The Granite 4.2 models which are just recently out, are optimized to handle agentic workflows. For local models, it's about using the right model for the right job.
- whatsThisBtn4 15d agoWaste of time when I can pay $20 a month for sol.
- EagnaIonat 15d agoversus $0 with local models. There will always be a reason to run frontier models, but local models are well at levels that assist with stuff that don't need that level of complexity.
- whatsThisBtn4 15d agoYou could buy a $150 refurbished 16gb i5 and use that model until openAI does its IPO and has to become sane again. But I guess a $2000 Mac is probably better if you don't care about cost or quality.
- EagnaIonat 14d ago> You could buy a $150 refurbished 16gb i5 Or I can use the Mac I already have. Your example though, Ouch! ~8B Q4. That's around 5-10 tokens a second. Base M1 16GB mac would do 15-20 tokens a seconds. That's a 6 year old machine. You do get what you pay for it seems.
- whatsThisBtn4 12d agoOh go get a $700 Nvidia laptop. Sorry local models are basically useless outside chat, I didn't even consider it.
- EagnaIonat 11d agoYou seem to be ill informed and only talking to yourself at this point.