6 ms·
Why do you need mlx? Like your blog post by you never explain why things need to be used. Why isn't using localforge enough as it ties into models?
by crazymoka 1y ago
Why do you need mlx?
Like your blog post by you never explain why things need to be used.
Why isn't using localforge enough as it ties into models?
- turnsout 1y agoI believe mlx will allow you to run the models marginally faster (per a recent blog post by @simonw)
- simonw 1y agoYeah, you don't necessarily need it but it's optimized for Apple Silicon and in my experience feels like it gives slightly better performance than GGUFs. I really need to formally measure that so I'm not just running on vibes!
- indigodaddy 1y agoI for one, am willing to just trust you bro ;)
- turnsout 1y agoYeah I’ll go with Simon’s vibes over most people’s measurements!
- freeone3000 1y agomlx is an alternative model format to GGUF. It executes natively on apple silicon using Apple's AI accelerator, rather than through GGUF as a compute shader(!). It's faster and uses fewer resources on Apple devices.
- avetiszakharyan 1y agoI was just trying to make sure is maximally performant, and did it with MLX because i am running on mac hardware and wanted to be able to run 30b in reasonable time so it can actually autonomously code something. Otherwise there are many ways to do it!
- p0w3n3d 1y agoIt would be nice if you mentioned it's about apple silicon, and not apple intel computers. They're still ubiquitous nowadays
- Tokumei-no-hito 1y agowe're on the 4th generation of silicon now