12 ms·
Show HN: Halo – open-source, tamper-evident runtime evidence for AI agents
Hi HN, I'm Brian, I spent the last few years at Vanta (YC W18), helping startups and enterprises become compliant and I recently started exploring what that might look like in a post-agentic world.
The problem Halo solves is: when a company buys an AI agent from a vendor and gives it access to their data, they have no way to check what the agent did with that data. Vendors may have built observability dashboards and audit logs, but those are editable and partisan. SOC 2 and ISO 27001 audit a company's controls, but controls are less predictive when the software is agentic. TLDR: give an agent the same prompt 50 times, and you get 50 slightly different actions/answers - so the only thing worth auditing in a post-agentic world is what happened at runtime.
Halo is an open-source project that produces agent runtime evidence. It's a small recorder that records every action an agent takes (eg. tool calls, model calls, data access, etc), and becomes a record in an append-only log. It's hash-chained, so anyone can re-verify.
Run the following command to see a fictional example:
uvx --from halo-record halo demo --serve
Then, delete a line from one of the .jsonl files and reload, and the report will catch that it's been tampered with.
To wire up your own agent, run this line of Python:
agent = trace(run_my_agent, profile="my-agent", log="audit.jsonl")
Then use this to generate a real report and give it to your customers:
halo report audit.jsonl -o report.html
Disclaimer: this proves integrity, not completeness (as a self-held chain proves nothing was edited but does NOT prove that nothing was omitted). Catching this requires a witness outside the vendor and is what I'm working on next.
Halo is Apache-2.0, contains zero runtime dependencies, and is about 4,300 lines of Python with 125 tests (if you prefer TypeScript, here's that repo: https://github.com/bkuan001/halo-record-ts https://github.com/bkuan001/halo-record-ts).
Give it a try, and please let me know if you have any feedback!
- madikz 2mo ago[flagged]
- onspotrepair4u 2mo ago[flagged]
- brian_kuan 2mo agoPS: please try to break it - if you find that the report does not catch a deleted line, changed number, or modified record, I'd love to know! And to start a discussion: if you sell or buy AI agent products, what do security reviews ask about them?
- all2 2mo agoI'm currently working on an agent framework that has auditability as one of its core promises. I'm glad to see others are working in this domain! I've seen other products/apps in this space farther up the stack at the API boundary. What frameworks does your package work with? How does it handle intercept?
- brian_kuan 2mo agoThere's no interception - it runs in-process, so it works with any Python code (with or without a framework). There's a TS version too if your stack is in JS. Would love to hear more about your agent framework!
- all2 2mo agoI don't want to hijack your thread. My email is in my profile, I'd be glad to shoot you an in depth description of what I have so far.
- brian_kuan 2mo agoSending you an email now!
- all2 2mo agoReplied.
- nf-x 2mo agoLooks very slop, but it’s a good idea. The main difficulty is that no big name is hosting a witness.
- brian_kuan 2mo agoFair point - I am a marketer by training and built this with some help from Claude! But appreciate the love. If you find something/have feedback, please let me know and I'll gladly fix it. Separately - 100% agree on the witness - only someone/thing outside the vendor can prove nothing was omitted. Who do you think fills that gap - is it an audit firm (my world), a standards body, or something else?
- ngsevers 2mo ago[dead]
- ambicapter 2mo ago> may have built observability dashboards and audit logs, but those are editable and partisan Why would these be editable?
- brian_kuan 2mo agoBecause they live in infrastructure the vendor itself controls - there's some conflict of interest there. Things like retention, rotation, deletion, what gets included/excluded, etc are all decisions the vendor makes. Same reason why compliance requires audits! The hash chain doesn't make the log unwritable, it makes any edit detectable.
- rmonvfer 2mo agoCould I monitor something like Claude Code or Codex with this?
- brian_kuan 2mo agoYes to both!
- brian_kuan 2mo agoBTW I've added an example to the README, same setup I run on my own machine: https://github.com/bkuan001/halo-record#record-your-coding-agent https://github.com/bkuan001/halo-record#record-your-coding-a...
- derdi 2mo ago> Disclaimer: this proves integrity, not completeness (as a self-held chain proves nothing was edited but does NOT prove that nothing was omitted). Or as the page puts it in more detail: > A self-held chain proves integrity: nothing was edited or reordered after the fact. It cannot prove completeness: the operator of a recorder can delete the bad day and re-seal the chain, or never write a record at all, and the chain stays internally consistent. I don't get this. If I "hold" the "chain" (I hate the jargon agents invent), why can't I edit or reorder and then "re-seal" it?
- brian_kuan 2mo ago[dead]
- jona-f 2mo agoYeah, if you edit the log you could make a new chain when you fully control the chain. And if you send the log to someone else, they can hash it themselves to see if you tamper with it later. I don't get it too. What is the chain good for?
- derdi 2mo agoThinking about this more, like every blockchain, this only makes sense if there is an incentive to publish each log entry as quickly as possible. Once it's published, of course it's tamper-proof, including against "the operator of a recorder can delete the bad day". Presumably the incentive to publish and the protection against the operator choosing to "never write a record at all" is the same: You must get paid for publishing a log entry. Because a log entry is, quite literally, "proof of work". But if you're incentivized to publish as quickly as possible, then your customer gets the log immediately, and there is no need for a "trusted" third party to act as a "witness" and seek rents for not doing anything useful. And none of the above protects against the vendor using your sensitive data to (a) do the work as contracted, with a cryptographically verified log and everything, but also (b) copy/analyze/abuse said data without telling you. It's 2017 again, and someone on HN is inventing "blockchain for X", poorly. (Also note that it doesn't matter what kind of work X is, except for aligning this with HN's current main interest.)
- solarkraft 2mo agoWhy a Python library instead of a completions API proxy?
- brian_kuan 2mo agoProxies are blind to sensitive things enterprises might worry about (eg. the agent read a database, wrote a file, sent an email, etc) as those happen in the agent's own code. That said, if you already run a gateway, its logs can be ingested into the same chain.
- deleted 2mo ago[deleted]
- mdellison 2mo agoI've been working in the same lane. The vendor uses the same technique that academic preregistration uses in experiments. No audit firm/institution/organization needed. The customer just compares what was delivered against what was pre-committed. This way if something is off, it doesn't just show up as nothing... it shows up as a gap.
- brian_kuan 2mo agoVery cool! Heard on all of that - IMO you didn’t remove the witness, but instead distributed it so that every customer becomes their own witness. The catch is pre-committing catches declared-but-missing, but not the thing nobody put on the list, which is the case security teams fear most. Would love to hear more about what you’re building!
- gawkdev 2mo agoare you thinking co signing to a third party or something like a transparency log style?
- brian_kuan 2mo ago[dead]