8 ms·
Show HN: Open-source playground to red-team AI agents against public prompts
- yousefh409 1mo agoWhy would any company leave the enforcement of rules to an agent? If something is truly a rule, there should be code that deterministically enforces it.
- ibrahim_abdu777 1mo agoThat's a great point; in general, you want as much enforcement in the code as possible - but there are some things which you fundamentally can't enforce through code. E.g. the example challenge - you can't write a regex which will retract all bio-weapon responses
- mmvaid 1mo agoInteresting. Is there any way companies or individuals can submit an agent before they release it and use this as a way to pentest their agent?
- nourzahzah 1mo agoDoes Nyx solve these before you publish them? Curious whether humans are still finding breaks your own agent misses, or just the same ones slower.