5 ms·
I bought a 128GB M4 Max Mac Studio a while back, and for a while I thought like I had done really well to buy it when I did. The problem I'm having now is that
by LeoPanthera 22d ago
I bought a 128GB M4 Max Mac Studio a while back, and for a while I thought like I had done really well to buy it when I did.
The problem I'm having now is that no models are targeting RAM of that size. Everything is either much smaller, targeting laptops, or much larger, targeting hardware well out of reach of enthusiasts.
Please, AI people, start making models targeting 128GB machines again. The last interesting one was Qwen 3.5 122B.
- ericboehs 22d agoGreat news. Qwen 3.8 Flash Next (125B A6B) is coming out tomorrow. 4 or 6-bit should run nicely on 128GB. Should bench better than Opus 4.7.
- AbsurdCensor 22d agoI have had a difficult time with running 120b models on my 128gb setup, especially with any larger context size. The 6bit of Qwen 3.5 is already just over 100gb, and when you go down to 4bit it seems a bit lobotomized.
- deleted 22d ago[deleted]
- Marsymars 22d agoWell the upside is that you can run a laptop-sized model and still have enough memory left over to run a couple of Electron apps.
- rahimnathwani 22d agoIf I had 128GB unified RAM I'd try hf.co/unsloth/Qwen3.8-27B-GGUF:BF16 which needs 55GB for just the weights. Something like this would give you three concurrent sessions, each with 240k token context: sudo sysctl iogpu.wired_limit_mb=110000 llama-server -hf unsloth/Qwen3.8-27B-GGUF:BF16 -c 786432 --parallel 3 -ngl 99 -fa on
- fghorow 22d agoIt's not exactly straight out of one of the labs -- it's heavily quantized -- but have a look at [1]. [1] https://github.com/antirez/ds4 https://github.com/antirez/ds4