marzukia

5 ms·

1.
Reusing KV cache from SSD to run concurrent 122B sessions on one 96GB Mac Studio(mrzk.io)
1 points by marzukia 2mo ago|0 comments
2.
by marzukia 2mo ago
Fair enough, nothing more infuriating than the sycophantic way LLMs write. But I’d definitely be dishonest if I said I’d stop using LLMs to tidy my writing (out of principle or otherwise), my hold on the English language has seriously degen
3.
by marzukia 2mo ago
Ahhh yeah, I’m not going to sit here and say I don’t use Claude to clean up my writing (ie make it actually coherently laid out). In all honestly I tend to write in a rambling stream of consciousness style across random scrap markdown files
4.
by marzukia 2mo ago
Thanks, good call. I've switched both throughput charts to a log y-axis (they were already log on x), so the sublinear taper is actually readable now instead of getting flattened by the big prefill numbers up top. On mlx-engine /
5.
by marzukia 2mo ago
The most counter-intuitive bug was that a unique message ID in the system prompt broke the entire KV cache. Since the cache requires byte-exact matches, that changing ID forced a full re-compute on every turn, turning warm contexts into col
6.
Fixed three bugs that made Qwen3.5-122B a daily driver on Mac Studio(mrzk.io)
65 points by marzukia 2mo ago|28 comments
7.
by marzukia 2mo ago
I spent three weeks debugging why my Qwen 122B setup on an M3 Ultra was taking 3–5 minutes to generate the first token on follow-up messages (despite having a "warm" context). The root cause wasn't the model, but three specif