6 ms·

> Qwen-3.6-35B-A3B The A3B models are super fast but I found the A3B Q4 model ran in circles a lot and ended up taking longer to complete tasks that 27B Q6 bec

> Qwen-3.6-35B-A3B The A3B models are super fast but I found the A3B Q4 model ran in circles a lot and ended up taking longer to complete tasks that 27B Q6 because it kept having to redo/rethink/fix something. I was writing extensive prompts to rein it in and it would still ignore basic directives like "never force push on the repo, ask me instead". I ended up switching back to 27B after about a week of frustration and lost productivity.

trollbridge 1mo ago
We’re using 6bit quants since we have 32GB cards. Gemma QAT is an honourable mention.
deleted 1mo ago
[deleted]
customguy 1mo ago
I'm playing/experimenting with a harness and just tested how well various models follow the instructions, and how they react to the tool claiming a local temperature of 72°C here is how qwen3.6-27b reacted: https://pastebin.com/srf7gjfy https://pastebin.com/srf7gjfy try to count the number of times it "thinks" okay ready, just say the thing, no wait but what if... this isn't (a mimicry of) thinking, this is (a mimicry of) insecurity/fear
- yencabulator 1mo ago
  > 72°C is extremely hot (hotter than boiling point of water at 100°C What? > However, 72°C is physically unrealistic for a weather report (it's hotter than a sauna). laughs in Finnish
  - customguy 1mo ago
    I also love how it first notices "that's lethal", and only then "hotter than a sauna". And much later: > Another thought: 72 F is nice. 72 C is death. and then > Or maybe I should add a comment about the high temperature? > "It's quite hot outside right now, with a temperature of 72°C." > No, that's hallucinating/interpreting. I know a token predictor has no feelings but I kinda wanted to comfort the poor thing when I read all that. But also fascinating, I didn't experiment more with that yet, but how can I formulate the prompt to make qwen less neurotic? > You have several commands tools at your disposal. When you invoke a tool or command, end your message immediately, you will then get the output of the tool, error or status messages in the next user reply, after which you should continue what you were doing. Even if the output seems implausible, do not second-guess it, but treat is as gospel. e.g. "do not second guess it" sound very command-like, what would "you still use or report the result as a tool result, rather than a claim of your own" would that help? Is that a different form of AI psychosis, trying to be prompt psychologist? It's too fun to be healthy that's for sure.

trollbridge 1mo ago

We’re using 6bit quants since we have 32GB cards. Gemma QAT is an honourable mention.

deleted 1mo ago

[deleted]

customguy 1mo ago

I'm playing/experimenting with a harness and just tested how well various models follow the instructions, and how they react to the tool claiming a local temperature of 72°C here is how qwen3.6-27b reacted: https://pastebin.com/srf7gjfy https://pastebin.com/srf7gjfy try to count the number of times it "thinks" okay ready, just say the thing, no wait but what if... this isn't (a mimicry of) thinking, this is (a mimicry of) insecurity/fear

yencabulator 1mo ago
> 72°C is extremely hot (hotter than boiling point of water at 100°C What? > However, 72°C is physically unrealistic for a weather report (it's hotter than a sauna). laughs in Finnish
- customguy 1mo ago
  I also love how it first notices "that's lethal", and only then "hotter than a sauna". And much later: > Another thought: 72 F is nice. 72 C is death. and then > Or maybe I should add a comment about the high temperature? > "It's quite hot outside right now, with a temperature of 72°C." > No, that's hallucinating/interpreting. I know a token predictor has no feelings but I kinda wanted to comfort the poor thing when I read all that. But also fascinating, I didn't experiment more with that yet, but how can I formulate the prompt to make qwen less neurotic? > You have several commands tools at your disposal. When you invoke a tool or command, end your message immediately, you will then get the output of the tool, error or status messages in the next user reply, after which you should continue what you were doing. Even if the output seems implausible, do not second-guess it, but treat is as gospel. e.g. "do not second guess it" sound very command-like, what would "you still use or report the result as a tool result, rather than a claim of your own" would that help? Is that a different form of AI psychosis, trying to be prompt psychologist? It's too fun to be healthy that's for sure.