6 ms·
Security nightmare inbound... The example given by Anthropic of tools filling valuable context space is a result of bad design. If you pass the tools below to
by michaelanckaert 10mo ago
Security nightmare inbound...
The example given by Anthropic of tools filling valuable context space is a result of bad design.
If you pass the tools below to your agent, you don't need "search tool" tool, you need good old fashion architecture: limit your tools based on the state of your agent, custom tool wrappers to limit MCP tools, routing to sub-agents, etc.
Ref:
GitHub: 35 tools (~26K tokens)
Slack: 11 tools (~21K tokens)
Sentry: 5 tools (~3K tokens)
Grafana: 5 tools (~3K tokens)
Splunk: 2 tools (~2K tokens)
- mkagenius 10mo agoDon't see whats wrong in letting llm decide which tool to call based on a search on long list of tools (or a binary tree of lists in case the list becomes too long, which is essentially what you eluded to with sub-agents)
- michaelanckaert 10mo agoI was referring to letting LLM's search github and run tools from there. That's like randomly searching the internet for code snippets and blindly running them on your production machine.
- mkagenius 10mo agoFor that, we need sandboxes to run the code in an isolated environment.
- michaelanckaert 10mo agoSure to protect your machine, but what about data security? Do I want to allow unknown code to be run on my private/corporate data? Sandbox all you want but sooner or later your data can be exfiltrated. My point is giving an LLM unrestricted access to random code that can be run is a bad idea. Curate carefully is my approach.
- mkagenius 10mo agoFor data security, you can run sandbox locally too. See https://github.com/instavm/coderunner https://github.com/instavm/coderunner