pipejosh

7 ms·

1.
by pipejosh 7mo ago
$2 million a year to run 244,000 searches that advanced 361 cases... That's about $5,500 per useful search. Meanwhile every car that drove past one of those cameras got logged, timestamped, and stored. These things aren't not law
2.
by pipejosh 7mo ago
I settled on a similar workflow but across two agents instead of one session. One agent writes task specs. The other implements them. Handoff files bridge the gap. The spec IS the session artifact because it captures intent, scope, and cons
3.
by pipejosh 7mo ago
Tried self-hosting with Mattermost to get around Slacks 90 day free tier history but my team didn't care for it much. Ended up back on Slack's free tier. This may solve that issue for me, will check it out.
4.
by pipejosh 7mo ago
Agreed. The non-determinism makes traditional testing basically useless here. You can't write a test suite for "the agent decided to do something unexpected this time." Logging and runtime checks are the only way to catch the
5.
by pipejosh 7mo ago
The automated audit only covers static analysis. When the agent actually runs, hitting MCP servers, making HTTP calls, getting responses back, that's where the real problems show up. Prompt injection through tool responses, malicious l
6.
by pipejosh 7mo ago
The maintenance burden is real but I think security is the bigger gap. People vibing out code with AI aren't thinking about input validation or dependency vulnerabilities. They build it, it works, they ship it. Then they're runnin
7.
by pipejosh 7mo ago
The part that worries me about agentic everything is the security model hasn't caught up. We're handing agents more and more access (shell, network, APIs, file systems) and the security story is still basically "the model pro
8.
by pipejosh 7mo ago
Circuit breakers for cost control is smart. The security equivalent is rate limiting and DLP on the egress side. If your agent suddenly starts making a bunch of requests to domains it's never hit before, or starts including high-entrop
9.
by pipejosh 7mo ago
Sandboxing the filesystem is one layer but egress scanning is where it gets interesting. An agent inside a sandbox can still exfiltrate secrets through any HTTP request it's allowed to make. The request looks totally legitimate from th
10.
by pipejosh 7mo ago
This is cool for testing the model side, but the real scary part is what happens after the injection succeeds. Even if your agent fails 3 out of 10 tests, that's a 30% chance it exfiltrates whatever secrets are in its environment. The
11.
by pipejosh 7mo ago
Everyone's talking about how productive agents are but nobody's talking about what happens when one gets prompt injected. Your agent has shell access, your API keys in env vars, and unrestricted internet. That's one bad depen