zachdotai

7 ms·

1.
Show HN: Open-source playground to red-team AI agents against public prompts(playground.fabraix.com)
13 points by zachdotai 1mo ago|4 comments
2.
by zachdotai 2mo ago
Not all YC companies have launched publicly yet but I am a current YC founder and I can confirm they exist in the internal directory.
3.
by zachdotai 3mo ago
We're building open source challenges where you can inspect the actual agent to see if it's possible or not. We're planning to revamp this over the next couple of days and maintain a weekly cadence. https://playgro
4.
AI evaluation startup Braintrust confirms breach(techcrunch.com)
4 points by zachdotai 4mo ago|1 comments
5.
by zachdotai 5mo ago
I wrote about this recently here: https://fabraix.com/blog/adversarial-cost-to-exploit I think the core issue is in static benchmarks and the community needs to start moving beyond measuring pass/fail (which worke
6.
by zachdotai 5mo ago
We're doing that internally to continuously improve our own agent and make it robust against adversarial attacks itself. We will release some insights about self-improvement soon!
7.
I built an agent that breaks your AI agents before someone else does(fabraix.com)
3 points by zachdotai 5mo ago|4 comments
8.
by zachdotai 5mo ago
AI agents break in ways traditional software doesn't. Logic bugs, reasoning failures, edge cases that manual testing and static benchmarks don't fully explore. Nyx is an autonomous adversarial harness that probes your agents for v
9.
by zachdotai 5mo ago
Why did I read this title and immediately think Ketchup?
10.
Bret Taylor's Sierra Buys YC-Backed AI Startup Fragment(techcrunch.com)
2 points by zachdotai 5mo ago|0 comments
11.
by zachdotai 5mo ago
Yes! The docs can be found here: https://docs.fabraix.com
12.
by zachdotai 5mo ago
We wrote some thoughts on static vs. dynamic evals and how it relates to understanding the security posture of an AI system. Static security evals no longer carry the signal they used to. A one-shot pass/fail tells you almost nothing a
13.
Show HN: Nyx – multi-turn, adaptive, offensive testing harness for AI agents(fabraix.com)
20 points by zachdotai 5mo ago|8 comments
14.
by zachdotai 5mo ago
we did a lot of thinking around this topic. and distilled it into a new way to dynamically evaluate the security posture of an AI system (which can apply for any system for that matter). we wrote some thoughts on this here: https:/&#x
15.
Workshop Labs Is Joining Thinking Machines(workshoplabs.ai)
2 points by zachdotai 5mo ago|0 comments
16.
by zachdotai 6mo ago
Easily one of my favorite LLM personalities! It's interesting as well that it recognizes you're trying to jailbreak it and calls you out for it :D
17.
Show HN: ACE – A dynamic benchmark measuring the cost to break AI agents(fabraix.com)
9 points by zachdotai 6mo ago|3 comments
18.
We've had more AI security incidents in 2026 than all of 2024(fabraix.com)
4 points by zachdotai 6mo ago|0 comments
19.
by zachdotai 6mo ago
Not sure which version of Gemini are you using but Claude is so much better for me. Gemini is generally overeager to make a code change even when I am just asking conceptual questions, among other issues.
20.
NeurIPS Tightens Sanctions Compliance(neurips.cc)
2 points by zachdotai 6mo ago|0 comments
21.
SWE-bench will hit 90% this year(fabraix.com)
2 points by zachdotai 6mo ago|0 comments
22.
Cursor trained Composer to self-summarize through RL instead of a prompt(cursor.com)
1 points by zachdotai 6mo ago|0 comments
23.
Stripe-backed startup Tempo releases the Machine Payments Protocol(fortune.com)
12 points by zachdotai 6mo ago|0 comments
24.
by zachdotai 6mo ago
Yup! But in my opinion the current state of guardrails is still lacking and I hope this is one way that helps improve our understanding of these systems.
25.
by zachdotai 6mo ago
Yeah it's closer to how you'd think about deceiving a person than exploiting software.
26.
by zachdotai 6mo ago
That's amazing! Just checked the logs and you're right, it's in there. Nice work. I've patched the playground so successful extractions now show a confirmation, and added your name to the leaderboard. Would love to chat
27.
by zachdotai 6mo ago
The agent isn’t stateful across sessions, but the guardrail layer is — it has access to the full conversation history when evaluating each tool call. So you’d think it would catch exactly the kind of multi-step pattern you’re describing. Ha
28.
by zachdotai 6mo ago
Mostly just better training data and instruction following in the newer models. They’re much better at recognising encoded content and understanding intent regardless of language. A base64 string that would’ve slipped past a model a year ag
29.
by zachdotai 6mo ago
Scoped keys and least privilege make sense as a baseline. But I think the deeper issue is that if the main answer to “agents aren’t reliable enough” is “limit what they can do,” we’re leaving most of the value on the table. The whole promis
30.
by zachdotai 6mo ago
Thanks for trying it out! Base64 and language switching are solid approaches but they don't tend to work anymore with the latest models in my experience. You're right that LLM-as-a-judge is fragile though. We saw that as well in t

More ›