5 ms·
That's funny, because I just went through the opposite. I got llama.cpp working with qwen3.6 and qwen3.8 by Googling and manually adjusting things according to
by wccrawford 25d ago
That's funny, because I just went through the opposite.
I got llama.cpp working with qwen3.6 and qwen3.8 by Googling and manually adjusting things according to reddit posts and Google not-really-helpful AI suggestions.
I tried settings up per-model stuff in settings.json, but again Google got in my way, and llama.cpp having 2 different settings.json (and Google lying about where 1 goes) made it far too difficulty to figure out. I spent hours on it.
Then I got fed up and asked Claude.
Immediately, it told me that the winget version of llama.cpp is for Vulkan, and I needed a different one and pointed at it. It doubled my speed.
Then it figured out what I was doing wrong with settings.json (wrong spot, global settings can't go in the per-model file, etc etc) and fixed all that, and got it working.
Then it tuned it somewhat.
Then I showed it the official settings pages for both models, and it undid the tuning and all the damage I had done with my tinkering, and got everything working.
In 30 minutes.
It was absolutely amazing.
Every time I see people recommending Qwen locally with llama.cpp, they just say "download it" and act like anyone that can't get it running is an idiot. But if there's a "using this settings.json" tutorial somewhere, I didn't find it, and neither did Google over a week of searching.
But Claude got it done for me.
Now, I admit, I haven't played with it much. Just before all this, I ran out of Claude on the $20 plan and bumped up to $100, and It's been so amazing that it's really hard to work on the local. Especially since it feels like Qwen3.8 35b a3b is probably around the corner, and why mess with 3.6 when 3.8 will probably release soon?
- ohyes 24d agoAm I the only one who just downloads directly from LM Studio and just runs the server there? It’s trivial.
- reverius42 23d agoYou're not the only one, I'm running Qwen3.8-27B in LM Studio and it seems to be going great. Was very easy to set up.