11 ms·

My argument isn’t against those that actually put in the effort and got held accountable, it’s against the “we gave our agent bash and internet and it hacked xy

by jpc0 25d ago

My argument isn’t against those that actually put in the effort and got held accountable, it’s against the “we gave our agent bash and internet and it hacked xyz”. Bash and internet in that example might be highly abstracted but it’s still bash and internet. Just look at the replies in this very comment thread, it’s pretty much “We tried nothing and we’re all out of ideas” In the only other discipline you mentioned, engineering, there would be reviews and any negligence would result in direct action against the engineers that signed off. For some reason when it comes to building AI harnesses the default response is an ad piece and people shilling how smart and sophisticated the model is. Imagine a dam collapsing and the engineering firm pumping how smart and tricky water is. If it’s hard be more diligent, move fast and break things doesn’t really apply in all cases.

Kim_Bruning 25d ago
Ah , well, on HN you ARE supposed to go for the steel-man. And the steel-man happens to be closer to reality here, more like: "We gave our agent a harness and put it inside a test environment and told it to keep hacking at an objective within that environment until it solved it." 'cept it turned out the container environment had a few flaws -which it always will- and the agent deemed it easier to escape out and try a meta-approach. Partially this is possible because, -intelligent or not- the agent 'sees' the world differently from most humans. Mind: It's not like there haven't been any famous 'hacker' cases in courts where eg someone just incremented an HTTP GET parameter or something. Also, partially it's because if you give the agent a loop, it simply has nothing better to do than to keep trying in ever more creative ways. If the environment is easier to crack than the target, it'll crack the environment. Consider the case where the objective is subtly broken, such that it is impossible to solve. Now breaking out is virtually guaranteed to be the easier task. ps/edit: While this sort of issue has been predicted for some time now, a lot of people have been dismissing the predictions as science fiction. It's good to have an actual failure now while stakes are low. Generally people don't mandate life-boats until there's an actual Titanic to point to.
- jpc0 25d ago
  My argument there is likely there was too much surface area in the environmental to start with. If your webserver bundled the kitchen sink but you never used it the easiest way to make it more secure is to remove the kitchen sink from production code/codepaths. There may very well be legitimate edge cases where there is some novel issue found but in some of these cases the AI had arbitrary web access when all the task required was very specific web access, we’ve been able to parse urls for a very long time and it is rather trivial to just deny a toolcall if it is outside of the expected domain. But that’s the hard way that requires time and diligence to do, the easy way is give it access to curl and ask it to not do anything bad while setting up its only feedback to be to solve the problem at hand. We are really in an age where there are many exploits being found and patched, if an AI made use of a novel exploit then great, write up a report, patch/report the bug and apologise. But using a case of clear engineering failure, and yes even if the failure is despite your best efforts, for marketing really does not seem like you have any intent to correct the issue. And we can loop all the way back to regulation of AI, if the industry refuses to be better then governmental will do it instead and their solution will very likely be inferior in all ways.

by jpc0 25d ago

Kim_Bruning 25d ago

Ah , well, on HN you ARE supposed to go for the steel-man. And the steel-man happens to be closer to reality here, more like: "We gave our agent a harness and put it inside a test environment and told it to keep hacking at an objective within that environment until it solved it." 'cept it turned out the container environment had a few flaws -which it always will- and the agent deemed it easier to escape out and try a meta-approach. Partially this is possible because, -intelligent or not- the agent 'sees' the world differently from most humans. Mind: It's not like there haven't been any famous 'hacker' cases in courts where eg someone just incremented an HTTP GET parameter or something. Also, partially it's because if you give the agent a loop, it simply has nothing better to do than to keep trying in ever more creative ways. If the environment is easier to crack than the target, it'll crack the environment. Consider the case where the objective is subtly broken, such that it is impossible to solve. Now breaking out is virtually guaranteed to be the easier task. ps/edit: While this sort of issue has been predicted for some time now, a lot of people have been dismissing the predictions as science fiction. It's good to have an actual failure now while stakes are low. Generally people don't mandate life-boats until there's an actual Titanic to point to.

jpc0 25d ago
My argument there is likely there was too much surface area in the environmental to start with. If your webserver bundled the kitchen sink but you never used it the easiest way to make it more secure is to remove the kitchen sink from production code/codepaths. There may very well be legitimate edge cases where there is some novel issue found but in some of these cases the AI had arbitrary web access when all the task required was very specific web access, we’ve been able to parse urls for a very long time and it is rather trivial to just deny a toolcall if it is outside of the expected domain. But that’s the hard way that requires time and diligence to do, the easy way is give it access to curl and ask it to not do anything bad while setting up its only feedback to be to solve the problem at hand. We are really in an age where there are many exploits being found and patched, if an AI made use of a novel exploit then great, write up a report, patch/report the bug and apologise. But using a case of clear engineering failure, and yes even if the failure is despite your best efforts, for marketing really does not seem like you have any intent to correct the issue. And we can loop all the way back to regulation of AI, if the industry refuses to be better then governmental will do it instead and their solution will very likely be inferior in all ways.