5 ms·

> I don't think knowing that will make me rich. As someone who’s not really sure that any of this is sustainable, I’d implore you to not sell yourself short. I

> I don't think knowing that will make me rich. As someone who’s not really sure that any of this is sustainable, I’d implore you to not sell yourself short. I reckon there’s a ton of dogma and nearly religious zeal among these companies, which among some people is earnest, and among others is cynical hype farming. I’ll bet someone objective enough to focus on using available tooling to solve real problems in practical ways that mitigate actual risks and are honest about actual limitations will be eBay here while the others are going to be somewhere between lucent and pets.com.

brazukadev 3d ago
I appreciate the message. I do have some ideas around sandboxing that I could not yet turn into a product, maybe I should take them seriously.
schrodinger 3d ago
This is a great point, analogous to the https://boringtechnology.club/ https://boringtechnology.club/ philosophy I've come to love. There's no reason we need to make an incredibly intelligent shell execution engine that can identify patterns that seem evil and may represent unwanted behavior to solve this problem. Simply limiting the available tools to a finite, known, ironclad-secure set (even if it's quite sprawling) is sufficient. LLMs will still find workarounds — from what I understand, a large part of the issue in this situation was that an agent was presumed to have read-only Internet access because it could only make GET requests. It should be pretty obvious that there's at least one website on the Internet that allows writes via GET. I think this is where auditing comes in, and a live team of people watching tool calls would have noticed the strange behavior. But I think a lot of times people jump to overly complex solutions when simple, well-bounded ones would work just fine. Yes, the intelligent shell is a great goal, but it's akin to solving the halting problem. This philosophy is what I love about PicoClaw (https://github.com/sipeed/picoclaw https://github.com/sipeed/picoclaw), and incidentally the philosophy behind Go and even *nix in general (i.e. provide small, composable, single-purpose tools).

brazukadev 3d ago

I appreciate the message. I do have some ideas around sandboxing that I could not yet turn into a product, maybe I should take them seriously.

schrodinger 3d ago

This is a great point, analogous to the https://boringtechnology.club/ https://boringtechnology.club/ philosophy I've come to love. There's no reason we need to make an incredibly intelligent shell execution engine that can identify patterns that seem evil and may represent unwanted behavior to solve this problem. Simply limiting the available tools to a finite, known, ironclad-secure set (even if it's quite sprawling) is sufficient. LLMs will still find workarounds — from what I understand, a large part of the issue in this situation was that an agent was presumed to have read-only Internet access because it could only make GET requests. It should be pretty obvious that there's at least one website on the Internet that allows writes via GET. I think this is where auditing comes in, and a live team of people watching tool calls would have noticed the strange behavior. But I think a lot of times people jump to overly complex solutions when simple, well-bounded ones would work just fine. Yes, the intelligent shell is a great goal, but it's akin to solving the halting problem. This philosophy is what I love about PicoClaw (https://github.com/sipeed/picoclaw https://github.com/sipeed/picoclaw), and incidentally the philosophy behind Go and even *nix in general (i.e. provide small, composable, single-purpose tools).