5 ms·
I have 4x r9700s as my coding daily drivers running qwen 3.8 27b at 80tps each. Zero complaints. Especially at $1200 each.
by lrvick 10d ago
I have 4x r9700s as my coding daily drivers running qwen 3.8 27b at 80tps each. Zero complaints. Especially at $1200 each.
- SomeHacker44 9d agoPlease share what operating system and model runtime you use? I have two and don't get close to that with AMD's own Lemonade. Thanks!
- lrvick 9d agoLemonade is one of the worst performing options. Run any modern Linux distro and ask your current LLM to setup llama.cpp with dflash2 for you as an unprivileged container running from a systemd user unit. Obviously only on a system you do not trust at all.
- pyrolistical 9d agoNo way you can get that much each without extremely quants For a single r9700 you have 637 GB/s and for qwen 3.8 27b q4_k_xl the maximum tg/s is 33 before mtp Now if you meant 4xr9700 tensor parallelism with mtp, 80 tg/s starts to make sense
- naasking 9d agoQ4 isn't an extreme quant, and I average 75 toks/s on code, 45 tok/s on prose with MTP.
- latentsea 9d agoYou can get those numbers with https://codeberg.org/ggz14/radiance-vllm-mxfp4 https://codeberg.org/ggz14/radiance-vllm-mxfp4. I also can get it on a single R9700 but the 75 ~ 80 t/s is only peak acceptance of very predictable tokens like coding or json, and averages lower for prose. It's still much faster than regular llama.cpp.
- julianlam 7d agoCan confirm. I've a single R9700 and have maxed out at 45 tokens/second on llama.cpp with Q4 Qwen 3.6 27B (with MTP)