5 ms·
> What makes this build different is the word before FP8: uncensored. We applied abliteration — orthogonalizing the refusal direction out of the residual stream
by radlad 25d ago
> What makes this build different is the word before FP8: uncensored. We applied abliteration — orthogonalizing the refusal direction out of the residual stream — to remove the model's safety-alignment refusals. The result is a model that will comply with requests the original would refuse.
Surely this has unintended side effects on output quality?
- andsoitis 25d ago> > What makes this build different is the word before FP8: uncensored. We applied abliteration — orthogonalizing the refusal direction out of the residual stream — to remove the model's safety-alignment refusals. The result is a model that will comply with requests the original would refuse. > Surely this has unintended side effects on output quality? Can you help me understand why that's the case?
- willy_k 25d agoBecause deleting model weights after training is likely to cause knock-on effects in model knowledge and/or behavior. Targetting it might mitigate this but it’s a) not guaranteed that only censor-ey parameters get removed, and b) likely that removing those parameters still has effects on the effectiveness of related parameters.
- jszymborski 25d agoThe weights aren't deleted, it's just additional fine tuning, is my understanding.
- naasking 25d agoThere is no question model quality is degraded by this though.
- InvertedRhodium 24d agoIt's altered, sure. I think inherent degradation is a step too far though.
- badsectoracula 25d agoConsidering these are essentially document completion engines[0], can't you just start the task with the version that doesn't refuse and then continue with the version that would refuse but now has to keep going after it accepted the task? :-P [0] in the sense that the "discussion" is basically a turn based game between you and the LLM filling a chat transcript document
- edg5000 24d agoThe article proves your point. It's eventually proceeded once there was the right history. If he already had the exploit and wanted the model to write an implementation, faking the history could have tripped the model over the edge. So the guardrails aren't unsurmountable.
- DiabloD3 25d agoIt does depending on the technique.
- miroljub 25d agoA bit worse quality is a fine trade off when the alternative is no output (zero quality).
- timmmmmmay 25d agoEarly attempts at this sort of thing definitely did, but these days the impact is minimal