5 ms·
I thought the main advantage of oMLX is it's less likely to invalidate the KV cache when working with coding agents, which is key when working on a Mac because
by MrScruff 1mo ago
I thought the main advantage of oMLX is it's less likely to invalidate the KV cache when working with coding agents, which is key when working on a Mac because of the slower prompt processing.
- quantumleaper 1mo agollama-server also supports saving the kv cache to SSD. I had no issues with cache invalidation using pi.
- gcr 1mo agoTIL! When was this functionality added? It wasn’t in llamacpp when I looked in June