6 ms·

Any recommendations?

sigmoid10 4d ago
GLM 5.3 is probably the best open weight model for cybersecurity/exploit development right now. Though it is still significantly behind the proprietary ones and you probably need your own datacenter to run it effectively. Same goes for the full Qwen 3.8 model. You can try the smaller versions, but even more capability will get left on the table that way.
- jnwatson 4d ago
  I run an abliterated distillation of Qwen 3.8 27B, slightly quantized to fit on my 4090, and I've been evaluating it to use as a worker bee for research directed by a smarter model. Much like in the article, abliterated Qwen will not obey restrictions on its behavior encoded in the prompt. If you want something not to happen, it better be enforced in the harness or environment (e.g. sandbox). It is much different than the Anthropic models I'm used to, which will, the vast majority of time, follow rules (before auto mode, I used to always run them in "yolo" mode). I am curious whether there's a connection between abliteration and rule following. These abliterated models are the ones you most want to follow your rules.
  - sroussey 4d ago
    Language models have always had an issue with negatives. A negative like do “not” xyz is just not encoded the same as spelling out what you want vs what you don’t want. Harder to write though.
    - sigmoid10 4d ago
      I would say in this case abliteration is the likely culprit. To uncensor a model this way, you literally deactivate the parts that would enact refusals. As in things it was told not to do. But the real process is more like brain surgery performed by a alchemist according to an ancient religious book where noone involved really understands what is actually happening in the model.
    - AndyNemmity 4d ago
      Exactly, I wrote a blog post in what feels like a long time ago on this topic. https://vexjoy.com/posts/positive-framing-agents-skills/ https://vexjoy.com/posts/positive-framing-agents-skills/
- barbazoo 4d ago
  Efficiently at scale or even as an individual?
  - sigmoid10 4d ago
    You'll need tons of tokens for these to churn through heavy reasoning and feedback driven tasks. You might even want to run many agents in parallel. Combine that with GPU memory requirements in the terabyte range just to hold the weights and you've pretty much left local consumer setups entirely.
- Terretta 4d ago
  Note that Mac Studio Ultra M3 (or upcoming M5) with 512GB is effective. You don't have to do this work fast, overnight is fine. Unless trying to use it interactively and adversarially, in which case it's not fast enough plus would be why those of us without our own datacenters will get told we can't have nice things.
cyanydeez 4d ago
Qwen3.8

GLM 5.3 is probably the best open weight model for cybersecurity/exploit development right now. Though it is still significantly behind the proprietary ones and you probably need your own datacenter to run it effectively. Same goes for the full Qwen 3.8 model. You can try the smaller versions, but even more capability will get left on the table that way.

jnwatson 4d ago
I run an abliterated distillation of Qwen 3.8 27B, slightly quantized to fit on my 4090, and I've been evaluating it to use as a worker bee for research directed by a smarter model. Much like in the article, abliterated Qwen will not obey restrictions on its behavior encoded in the prompt. If you want something not to happen, it better be enforced in the harness or environment (e.g. sandbox). It is much different than the Anthropic models I'm used to, which will, the vast majority of time, follow rules (before auto mode, I used to always run them in "yolo" mode). I am curious whether there's a connection between abliteration and rule following. These abliterated models are the ones you most want to follow your rules.
- sroussey 4d ago
  Language models have always had an issue with negatives. A negative like do “not” xyz is just not encoded the same as spelling out what you want vs what you don’t want. Harder to write though.
  - sigmoid10 4d ago
    I would say in this case abliteration is the likely culprit. To uncensor a model this way, you literally deactivate the parts that would enact refusals. As in things it was told not to do. But the real process is more like brain surgery performed by a alchemist according to an ancient religious book where noone involved really understands what is actually happening in the model.
  - AndyNemmity 4d ago
    Exactly, I wrote a blog post in what feels like a long time ago on this topic. https://vexjoy.com/posts/positive-framing-agents-skills/ https://vexjoy.com/posts/positive-framing-agents-skills/
barbazoo 4d ago
Efficiently at scale or even as an individual?
- sigmoid10 4d ago
  You'll need tons of tokens for these to churn through heavy reasoning and feedback driven tasks. You might even want to run many agents in parallel. Combine that with GPU memory requirements in the terabyte range just to hold the weights and you've pretty much left local consumer setups entirely.
Terretta 4d ago
Note that Mac Studio Ultra M3 (or upcoming M5) with 512GB is effective. You don't have to do this work fast, overnight is fine. Unless trying to use it interactively and adversarially, in which case it's not fast enough plus would be why those of us without our own datacenters will get told we can't have nice things.

cyanydeez 4d ago

Qwen3.8