Just wanted to share, I had a SaaS AI drop me a script to bench ninfer against llama-cpp and it is impressive. The place that it's doing better at than llama-cp

Just wanted to share, I had a SaaS AI drop me a script to bench ninfer against llama-cpp and it is impressive. The place that it's doing better at than llama-cpp seems to really be late in the context window. Initial results boiled down as follows. # lmstudio-community/qwen3.8-27b@q4_k_m decode falloff 104.3 tok/s @ 12,683 -> 55.8 tok/s @ 240,755 (53% retained) prefill falloff 3,274 tok/s -> 1,059 tok/s (32% retained) # qwen3_8_27b_nvfp4.ninfer decode falloff 173.3 tok/s @ 11,867 -> 139.3 tok/s @ 225,710 (80% retained) prefill falloff 8,726 tok/s -> 2,816 tok/s (32% retained) I should still have room for more performance on the table. I've not even touched the overclock settings on the GPU. This is a really cool project, I'm going to have to get into what those 3 guys are doing... assuming it can be done with what I got.