5 ms·
I'm getting around 27 tokens per second decode using vLLM + Qwen 3.8 27b nvfp4 + DFlash 2 on the DGX Spark.
by adefa 29d ago
I'm getting around 27 tokens per second decode using vLLM + Qwen 3.8 27b nvfp4 + DFlash 2 on the DGX Spark.
- pdntspa 29d agoI am working this out with Fable right now, for getting this running on my DGX spark homelab; it mentioned that there might be issues with the 'optimized LM-head restrictions' that unsloth NVFP4 ships with. Have you had any issues here? Are you trying this with vLLM? Or a different engine? I am getting about 15 tok/s on my spark on my current setup using the 0.26 nvidia vLLM image and MTP.
- adefa 29d agoIf you are using the Unsloth nvfp4 checkpoint, you need to patch vLLM+DFlash 2 to accept the quant's FP8 `lm_head`.