8 ms·
1x MI300x , rocm/sglang , highly optimized image, FP8, FP16 Cache (this fixes looping issues with qwen3.6-35b ~100tok/s average
by yolo-auto 2mo ago
1x MI300x , rocm/sglang , highly optimized image, FP8, FP16 Cache (this fixes looping issues with qwen3.6-35b
~100tok/s average