5 ms·
>Just remove hacking (bio-weapon, etc.) data from the training dataset and you're done. Reasoning about how to write secure software uses the same knowledge as
by sigmar 3d ago
>Just remove hacking (bio-weapon, etc.) data from the training dataset and you're done.
Reasoning about how to write secure software uses the same knowledge as reasoning about how to break/hack it.
- amluto 3d agoI don’t buy it. Reasonable about building secure software can take the form “this memory access might be out of bounds — that MUST be fixed” or “this process has access to an inappropriate privilege — this is a serious weakness”. Exploiting things and the capabilities that the labs call “cyber” are about the ability to (a) find the issues mentioned above and then (b) string issues together and avoid all the imperfect mitigations to actually compromise something. That latter part was IMO not actually necessary to train extensively, and I’d be quite happy to use a model that has no special skills in this regard but that would do (a) without complaining.
- win311fwg 3d agoJust remove anything software-related from the training dataset. Which also solves the alignment problem with those who do not enjoy seeing LLMs write software. But that brings us back to: Aligned to whom?