7 ms·
The asymmetry part at the end is the frustrating part to me. I've been using Sol for code review in the last week or two. A couple of times during review it's e
by onionisafruit 2mo ago
The asymmetry part at the end is the frustrating part to me. I've been using Sol for code review in the last week or two. A couple of times during review it's errored out with the cybersecurity message. So it's found something but won't tell me what it is because I'm not on OpenAI's besties list.
- IshKebab 2mo agoYeah that's what I don't get. How can they possibly distinguish between good guys trying to secure code they wrote and bad guys trying to attack code they didn't?
- ClarityJones 2mo agoAnd... now it's a vulnerability that OpenAI has for your system, which you paid to provide.
- verdverm 2mo agoAnd... given OpenAI's secops, likely something others may have in due time
- didibus 2mo agoI agree, but I will say, I think both Mythos and these OpenAI model find exploits by examining and trying things against the running system, not from looking at the code. I think you'd have to do the same to catch the real vulnerabilities.
- emtel 2mo agoIn my experience fable can absolutely find very subtle bugs just by reading code and thinking. Last example I saw was a very subtle race condition that neither I not the author noticed. It wasn’t a security issue, but it could have been.
- paxys 2mo agoThe safety classifiers aren't all that advanced. It's more likely that something in your code triggered a random chain of thought that had the word "pentest" or "malware" or something of that sort in it and it automatically shut down.