10 ms·
To add for me (and claude assisted but tuned myself), here is for 3060 12gb + 5950x 128 gb ram without image support: https://pastebin.com/HsAUBGSY https://past
by tyfon 1mo ago
To add for me (and claude assisted but tuned myself), here is for 3060 12gb + 5950x 128 gb ram without image support: https://pastebin.com/HsAUBGSY https://pastebin.com/HsAUBGSY
Here is the chat template I used (and renamed to qwen.jinja):
https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/raw/main/chat_template.jinja https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/r...
For image support, drop to NGL 24 and add:
--mmproj-url https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/resolve/main/mmproj-F16.gguf \
--no-mmproj-offload \
--image-min-tokens 1024 \
And drop the --no-mmproj
- tommica 1mo agoThanks for this! I got the same card, but not the same CPU nor RAM, but this is a great starting point to start tweaking
- acidburnNSA 1mo agoHow many tokens/second you getting? I have the same CPU but a 5070 and only 64 GB of ram. I just got llama.cpp built and am now hitting a whopping 5-6 tokens/s.