5 ms·
Running the mxfp4 unsloth quant of qwen3.5-397b-a17b, I get 40 tps prefill, 20tps decode. AMD threadripper pro 9965WX, 256gb ddr5 5600, rtx 4090.
by ydj 5mo ago
Running the mxfp4 unsloth quant of qwen3.5-397b-a17b, I get 40 tps prefill, 20tps decode.
AMD threadripper pro 9965WX, 256gb ddr5 5600, rtx 4090.