5 ms·

Show HN: I built a tool to assist AI agents to know when a PR is good to go

45 points by dsifry 8mo ago

I've been using Claude Code heavily, and kept hitting the same issue: the agent would push changes, respond to reviews, wait for CI... but never really know when it was done. It would poll CI in loops. Miss actionable comments buried among 15 CodeRabbit suggestions. Or declare victory while threads were still unresolved. The core problem: no deterministic way for an agent to know a PR is ready to merge. So I built gtg (Good To Go). One command, one answer: $ gtg 123 OK PR #123: READY CI: success (5/5 passed) Threads: 3/3 resolved It aggregates CI status, classifies review comments (actionable vs. noise), and tracks thread resolution. Returns JSON for agents or human-readable text. The comment classification is the interesting part — it understands CodeRabbit severity markers, Greptile patterns, Claude's blocking/approval language. "Critical: SQL injection" gets flagged; "Nice refactor!" doesn't. MIT licensed, pure Python. I use this daily in a larger agent orchestration system — would love feedback from others building similar workflows.

joshribakoff 8mo ago
I dislike the idea of coupling my workflow to saas platforms like github or code rabbit. The fact that you still have to create local tools is a selling point for just doing it all “locally”.
mcolley 8mo ago
Super interesting, any particular reason you didn't try to solve these prior to pushing with hooks and subagents?
- dsifry 8mo ago
  I did! The issue however, is having a clear, deterministic method of defining when the code review was 'done'. So the hooks can fire off subagents, but they are non-deterministic and often miss vital code review comments - especially ones that are marked in an inline comment, or are marked as 'Out of PR Scope' or 'Out of range of the file' - which are often the MOST important comments to address! So gtg builds all of that in and deterministically determines whether or not there are any actionable comments, and thus you can block the agent from moving forward until all actionable comments are thoroughly reviewed, acted upon or acknowledged, at which point it will change state and allow the PR to be merged.
  - blutoot 8mo ago
    I thought hooks are always fired if you use it as a PreToolUse event. Wouldn’t that work for the GitHub action tools from the GitHub mcp?
    - dsifry 8mo ago
      Sure, but that mcp still missed actionable comments that are marked as Out of Scope or Outside the PR - and this doesn't require having the context window loss of having another mcp instantiated, either. Anyway, give gtg a competitive look against the mcp - you should be able to see the difference
    - dsifry 8mo ago
      Just to be clear - the hook is deterministic, but the subagent running with an mcp server loaded is not - and for medium/large PRs, it can run out of context window or just forget what it is trying to do and get lazy and say 'Everything is good, ready to merge!' when in fact tests are failing or there are still unaddressed PR comments.
rootnod3 8mo ago
Sorry, so the tool is now even circumventing human review? Is that the goal? So the agent can now merge shit by itself? Just the let damn thing push nto prod by itself at this point.
- ljm 8mo ago
  Someone’s gonna think about wiring all this up to Linear or Jira, and there’ll be a whole new set of vulnerabilities created from malicious bug reports.
  - dsifry 8mo ago
    That's why I intentionally don't have this hooked into an ingest flow - you still get control over what issues/stories you want the agent swarm to work on... Just now, I can know that the code that was written has been reviewed and all comments have been fully addressed!
- literalAardvark 8mo ago
  In some workflows it's helpful for the full loop to be automated so that the agent can test if what's done works. And you can do a more exhaustive test later, after the agents are done running amok to merge various things.
  - dsifry 8mo ago
    Exactly right!
- danenania 8mo ago
  I don’t think “ready to merge” necessarily means the agent actually merges. Just that it’s gone as far as it can automatically. It’s up to you whether to review at that point or merge, depending on the project and the stakes. If there are CI failures or obvious issues that another AI can identify, why not have the agent keep going until those are resolved? This tool just makes that process more token efficient. Seems pretty useful to me.
joshuanapoli 8mo ago
This looks nice! I like the idea of providing more deterministic feedback and more or less forcing the assistant to follow a particular development process. Do you have evidence that gtg improves the overall workflow? I think that there is a trade-off between risk of getting stuck (iteration without reaching gtg-green) versus reaching perfect 100% completion.
- dsifry 8mo ago
  I found that it has improved overall code quality significantly, at the cost of somewhat slower velocity. But it has meant fewer interruptions where the ai is just waiting for me, or saying "Everything is ready!" only to find that ci/cd failed or there were clearly existing comments/issues.
philipp-gayret 8mo ago
Very interesting! This has a gem in the documentation: Using the tool itself as a CI check. I hadn't considered unresolved comments by say a person, or CodeRabbit or similar tool being a CI status failure. That's an excellent idea for AI driven PR's. On a personal note; I hate LLM output to advertise a project. If you have something to share have the decency to type it out yourself or at least redact the nonsense from it.
- dsifry 8mo ago
  Lol, I thought it did a reasonably good job, but to each their own - this was the difference between releasing the project so others could use it with decent documentation, or not releasing and just using it internally. :)
furyofantares 8mo ago
Then you had the LLM write the blog post as well as your post on HN.
nyc1983 8mo ago
I don’t understand how this provides anything above using GitHub status checks and branch protections to require conversations to be resolved before merging. Combined with the GitHub CLI, this gives agents everything they need to achieve the same result. More AI slop on top of AI slop. At this point when seeing these kinds of posts I feel like Edward Norton in front of the copy machine.
- dsifry 8mo ago
  Some github comments are marked as sctionable, some have threads and suggestions, some are suggestions or are nitpicks. This provides you with a deterministic, reliable red/green approach that you cn use to enforce your policy. Give it a try and you will see how it is much more reliable than using a nondeterministic agent, especially for complex reviews!
Agent_Builder 8mo ago
[dead]
aaronbrethorst 8mo ago
“The problem no one talks about” is a bit of breathless LLM spew, and an even better tell than an emdash
forgotpwd16 8mo ago
That repo is quintessentially surreal. AI-written code, published in AI-made PRs, reviewed by multiple AI bots (one of which being same model that wrote code & made the PR, maybe others too just accessible via 3rd vendor), merged by AI (assuming dogfooding).

45 points by dsifry 8mo ago

joshribakoff 8mo ago

I dislike the idea of coupling my workflow to saas platforms like github or code rabbit. The fact that you still have to create local tools is a selling point for just doing it all “locally”.

mcolley 8mo ago

Super interesting, any particular reason you didn't try to solve these prior to pushing with hooks and subagents?

dsifry 8mo ago
I did! The issue however, is having a clear, deterministic method of defining when the code review was 'done'. So the hooks can fire off subagents, but they are non-deterministic and often miss vital code review comments - especially ones that are marked in an inline comment, or are marked as 'Out of PR Scope' or 'Out of range of the file' - which are often the MOST important comments to address! So gtg builds all of that in and deterministically determines whether or not there are any actionable comments, and thus you can block the agent from moving forward until all actionable comments are thoroughly reviewed, acted upon or acknowledged, at which point it will change state and allow the PR to be merged.
- blutoot 8mo ago
  I thought hooks are always fired if you use it as a PreToolUse event. Wouldn’t that work for the GitHub action tools from the GitHub mcp?
  - dsifry 8mo ago
    Sure, but that mcp still missed actionable comments that are marked as Out of Scope or Outside the PR - and this doesn't require having the context window loss of having another mcp instantiated, either. Anyway, give gtg a competitive look against the mcp - you should be able to see the difference
  - dsifry 8mo ago
    Just to be clear - the hook is deterministic, but the subagent running with an mcp server loaded is not - and for medium/large PRs, it can run out of context window or just forget what it is trying to do and get lazy and say 'Everything is good, ready to merge!' when in fact tests are failing or there are still unaddressed PR comments.

rootnod3 8mo ago

Sorry, so the tool is now even circumventing human review? Is that the goal? So the agent can now merge shit by itself? Just the let damn thing push nto prod by itself at this point.

ljm 8mo ago
Someone’s gonna think about wiring all this up to Linear or Jira, and there’ll be a whole new set of vulnerabilities created from malicious bug reports.
- dsifry 8mo ago
  That's why I intentionally don't have this hooked into an ingest flow - you still get control over what issues/stories you want the agent swarm to work on... Just now, I can know that the code that was written has been reviewed and all comments have been fully addressed!
literalAardvark 8mo ago
In some workflows it's helpful for the full loop to be automated so that the agent can test if what's done works. And you can do a more exhaustive test later, after the agents are done running amok to merge various things.
- dsifry 8mo ago
  Exactly right!
danenania 8mo ago
I don’t think “ready to merge” necessarily means the agent actually merges. Just that it’s gone as far as it can automatically. It’s up to you whether to review at that point or merge, depending on the project and the stakes. If there are CI failures or obvious issues that another AI can identify, why not have the agent keep going until those are resolved? This tool just makes that process more token efficient. Seems pretty useful to me.

joshuanapoli 8mo ago

This looks nice! I like the idea of providing more deterministic feedback and more or less forcing the assistant to follow a particular development process. Do you have evidence that gtg improves the overall workflow? I think that there is a trade-off between risk of getting stuck (iteration without reaching gtg-green) versus reaching perfect 100% completion.

dsifry 8mo ago
I found that it has improved overall code quality significantly, at the cost of somewhat slower velocity. But it has meant fewer interruptions where the ai is just waiting for me, or saying "Everything is ready!" only to find that ci/cd failed or there were clearly existing comments/issues.

philipp-gayret 8mo ago

Very interesting! This has a gem in the documentation: Using the tool itself as a CI check. I hadn't considered unresolved comments by say a person, or CodeRabbit or similar tool being a CI status failure. That's an excellent idea for AI driven PR's. On a personal note; I hate LLM output to advertise a project. If you have something to share have the decency to type it out yourself or at least redact the nonsense from it.

dsifry 8mo ago
Lol, I thought it did a reasonably good job, but to each their own - this was the difference between releasing the project so others could use it with decent documentation, or not releasing and just using it internally. :)

furyofantares 8mo ago

Then you had the LLM write the blog post as well as your post on HN.

nyc1983 8mo ago

I don’t understand how this provides anything above using GitHub status checks and branch protections to require conversations to be resolved before merging. Combined with the GitHub CLI, this gives agents everything they need to achieve the same result. More AI slop on top of AI slop. At this point when seeing these kinds of posts I feel like Edward Norton in front of the copy machine.

dsifry 8mo ago
Some github comments are marked as sctionable, some have threads and suggestions, some are suggestions or are nitpicks. This provides you with a deterministic, reliable red/green approach that you cn use to enforce your policy. Give it a try and you will see how it is much more reliable than using a nondeterministic agent, especially for complex reviews!

Agent_Builder 8mo ago

[dead]

aaronbrethorst 8mo ago

“The problem no one talks about” is a bit of breathless LLM spew, and an even better tell than an emdash

forgotpwd16 8mo ago

That repo is quintessentially surreal. AI-written code, published in AI-made PRs, reviewed by multiple AI bots (one of which being same model that wrote code & made the PR, maybe others too just accessible via 3rd vendor), merged by AI (assuming dogfooding).