7 ms·
That's funny, I downloaded the same model on my 48GB M4 Pro and gave it a problem to solve in an existing codebase, it spun its wheels for twenty minutes and th
by velcrovan 25d ago
That's funny, I downloaded the same model on my 48GB M4 Pro and gave it a problem to solve in an existing codebase, it spun its wheels for twenty minutes and then fell over dead. This was using LMStudio and pi as a harness; I never use pi for anything else, so maybe I'm holding it wrong.
- w-ll 25d agoits all still somewhat of a dice roll
- NamlchakKhandro 25d ago[flagged]
- cellularmitosis 25d agoWe don’t know what quantization level was used for the weights or the kv cache for you or for parent poster, so this is probably an apples to oranges comparison.
- spacebacon 25d ago[dead]
- s1gsegv 25d agoThey made a kind of strange decision with Qwen3.8 27B, the template defaults the reasoning_effort to xhigh. I found if you set it to medium it doesn’t just sit there churning forever.
- ekianjo 25d agoxhigh gives better results
- dofm 25d agoNot necessarily. I have seen xhigh go down several rabbit holes, dwell on edge cases and write worse code as a result; it literally distracted itself into writing a complex chain of functions ignoring my prompt, when on “low” reasoning it gets it right on a prompt that requires a few lines of code in the right places. Simon Willison’s blog has another example (SVG of a circle). It’s a bit like how giving LLMs access to web search tools can cause them to go down a blind alley based on their first “reasoning” output that then leaves them unable to solve a puzzle correctly that they can fully solve on their own.
- freehorse 25d agoWith qwen 27b, setting the right reasoning effort for the specific task is important. With xhigh it has a chance at hard problems that bigger models may even fail. But for many everyday tasks, I have found that no reasoning and a system prompt instructing it to be brief is good enough. Note that even with thinking disabled, it may still get into long "chain of thought" reasoning state (out of thinking blocks) if the task is hard and you do not give further instructions, esp with access to tools etc.
- chmod775 25d agoNot if it fills up its entire context with "But wait..."
- dofm 24d agoxhigh tends not to do that. Uses caveman-ish language. But the reasoning trace does tend to obsess about stuff that it should just ask you about.
- apothegm 24d agoIs there an easy way for a n00b with LMStudio to switch it to medium? Asking for a friend… XD
- velcrovan 24d agoI had heard of this and actually did set the reasoning to medium ahead of time…
- deleted 25d ago[deleted]
- noman-land 25d agoI've recently learned and then observed that oMLX serves local models much, much faster than LM Studio.
- jonplackett 25d agoSet its thinking lower. This is a known issue. It still thinks A LOT with lower reasoning levels
- phront 25d agoMaybe giving pi more output by setting higher value to maxTokens will resolve his issue
- jonplackett 24d agoI’ve been using the mlx version with orb studio an opencode
- drdaeman 24d agoCheck out this: https://news.ycombinator.com/item?id=49402232 https://news.ycombinator.com/item?id=49402232 both article and comments. There are a lot of knobs to tweak, and some are pretty impactful.