6 ms·

Show HN: Nit – I rebuilt Git in Zig to save AI agents 71% on tokens

23 points by fielding 6mo ago

stingraycharles 6mo ago
Why a complete rewrite rather than a wrapper around the cli commands? Also, aren’t LLMs RLHFd a lot with using tools like git and as such have a better time interacting with it than custom tools?
- yodon 6mo ago
  It defaults to being a wrapper around git when it's not custom implemented, and it's recommended that you alias nit as git so the agent can work the way it normally would, just faster and cheaper.
- Brian_K_White 6mo ago
  TFA says there is no difference since it falls through and does act as just a wrapper for anything that it doesn't implement itself.
  - stingraycharles 6mo ago
    But that’s my problem, I don’t want it to implement anything itself, I just want it to transform git output.
    - Brian_K_White 6mo ago
      It's using libgit. IE it is basically still just a wrapper for official reference git implementation. I think you are worried anbout nothing.
      - 1718627440 6mo ago
        So he didn't 'rebuilt Git in Zig'.
        Brian_K_White 6mo ago
        The command "git" is a front-end for a lot of other seperate lower level functions, which he reimplimented in zig. He did some stuff, he didn't do some other stuff. Nothing about this story is automatically invalid or pointless or misleading or unwholesome.
      - fielding 6mo ago
        correct
antileet 6mo ago
Maybe I'm missing something. If I use `git status --short` or `git log --oneline`, I see output similar to your tool's.
- fielding 6mo ago
  nit's defaults go beyond what --short does. The token savings come from stripping headers, padding, instructional text, etc. Headers and decorative text ends up tokenizing poorly, so it helps quite a bit there.
spullara 6mo ago
check out rtk that does this for a bunch of commands
- aeonfox 6mo ago
  Do the larger LLM platforms just do this for you? Or perhaps they do this behinds the scenes, and charge you for the same amount of tokens?
  - spullara 6mo ago
    no because tool calls are all client side generally. unless you mean using a remote environment where Claude Code is running separately but usually those aren't being charged by the token.
  - fielding 6mo ago
    The tokens still land in the context window either way. Prompt caching gives you a discount on repeated input, but only for stable prefixes like system prompts. Git output changes every call, so it's always uncached, always full price. Nit reduces what goes into the window in the first place.
    - aeonfox 6mo ago
      I was thinking more if you write a prompt into an IDE that has first-party integration with an LLM platform (e.g. VS Code with Github Copilot), it would make sense on their end to reduce and remove redundant input before ingesting the token into their models, just to increase throughput (increase customers) and decrease latency (reduce costs). They would be foolish not to do this kind of optimisation, so surely they must be doing it. Whether they would pass on those token savings to the user, I couldn't say.
- fielding 6mo ago
  this is awesome! thanks for sharing rtk.. going to check it out.
slopinthebag 6mo ago
Was this entirely created by an LLM? The article clearly was.
- tempest_ 6mo ago
  Every "I created <xyz>" or "I rewrote x (y lang) into z lang" should really read "I prompted Claude Code to <insert thing>" "I" create stuff all the time with AI Agents but am real uncomfortable claiming ownership over them. Others don't seem to have this problem though so /shrug p.s. - in this case the commits are claude commits, even if they tell it not attribute itself you can tell because good commit messages were incredibly rare (even my own) until the last year or so when they started to look like entire pull requests
  - slopinthebag 6mo ago
    Yeah, like whatever I prompt I'm fine sharing it, but I'm not gonna claim I made something. It's like claiming I'm an artist because I paid a guy to paint someone.
    - ofrzeta 6mo ago
      There are certainly artists that have helpers that do the actual execution.
    - fielding 6mo ago
      Is it though? The person who commissions a painting doesn't design the composition, validate every brushstroke, and run the output through an automated test suite. The analogy breaks down pretty fast.
      - slopinthebag 6mo ago
        > The person who commissions a painting doesn't design the composition They often do! Of course the artist has creative liberty to make it work, similar to how LLM's will deviate from the spec. Was your automated test suite also AI generated? You probably could have avoided all criticism by simply writing the article yourself instead of publishing raw LLM output. If someone isn't willing to write about a project they made, it's usually an indicator that they spent an equal amount of effort on the code as well. And why did you make a commit to remove em dashes? That seems odd.
warwickmcintosh 6mo ago
The 71% reduction is interesting but I'd want to see where those tokens are actually going in a typical agent session. In my experience running multi-step coding agents, the git output itself is rarely the bottleneck...
- fielding 6mo ago
  Not saying it is the bottleneck. It's bloat. 7.4% of all shell tokens across 3,156 sessions is a lot of unnecessary context. It won't make or break a session, but it adds up across thousands of calls.
mpalmer 6mo ago
You should expect to be gently ridiculed if you submit this as a serious project. Do not use LLMs to claim knowledge or abilities you don't actually have. It's not impressive, and it's obvious to anyone with more than a year or two of actual programming experience.
- mpalmer 6mo ago
  To elaborate: - The claim of "71% token savings" is misleading. You've already told us that git output is 7% of the "bash commands", so your token savings is really ~5% of bash command output. Also, the "200K tokens saved" refers the total saved across more than 3000 agent sessions. So it's really just ~60 tokens saved per session. That's a fraction of a fraction of a penny. Did you check your math here? Or just trust the model? - You have not rebuilt git, you have replaced a small part of its porcelain CLI. Most of what makes git git is in the library code. - Zig is a high-performance systems programming language that requires the programmer to manage memory manually. You've used it here to handle command line options (and yes, to call into libgit). That is overkill, and that matters, because: - The performance difference (e.g. 8ms vs 16ms) might be real, but it still doesn't matter at all. For one, there's no way you yourself could tell the difference. The slower time is still a single frame of 60fps. For another, this is not a hot path. These git commands - however fast - are swallowed up by the time it takes the agent to do inference. There's no question that inference itself will take orders of magnitude more time than a 16ms shell command.
000ooo000 6mo ago
So the big token saving was changing a setting that Git already allows you to change? And does 8ms really matter when you're shunting bulk crap back and forth from the cloud?
- fielding 6mo ago
  It goes beyond what I was able to do with git settings alone. Specifically stripping the headers/padding/decorative and it doees it across all output (or well a lot of it).

23 points by fielding 6mo ago

Why a complete rewrite rather than a wrapper around the cli commands? Also, aren’t LLMs RLHFd a lot with using tools like git and as such have a better time interacting with it than custom tools?

yodon 6mo ago
It defaults to being a wrapper around git when it's not custom implemented, and it's recommended that you alias nit as git so the agent can work the way it normally would, just faster and cheaper.
Brian_K_White 6mo ago
TFA says there is no difference since it falls through and does act as just a wrapper for anything that it doesn't implement itself.
- stingraycharles 6mo ago
  But that’s my problem, I don’t want it to implement anything itself, I just want it to transform git output.
  - Brian_K_White 6mo ago
    It's using libgit. IE it is basically still just a wrapper for official reference git implementation. I think you are worried anbout nothing.
    - 1718627440 6mo ago
      So he didn't 'rebuilt Git in Zig'.
      - Brian_K_White 6mo ago
        The command "git" is a front-end for a lot of other seperate lower level functions, which he reimplimented in zig. He did some stuff, he didn't do some other stuff. Nothing about this story is automatically invalid or pointless or misleading or unwholesome.
    - fielding 6mo ago
      correct

antileet 6mo ago

Maybe I'm missing something. If I use `git status --short` or `git log --oneline`, I see output similar to your tool's.

fielding 6mo ago
nit's defaults go beyond what --short does. The token savings come from stripping headers, padding, instructional text, etc. Headers and decorative text ends up tokenizing poorly, so it helps quite a bit there.

spullara 6mo ago

check out rtk that does this for a bunch of commands

aeonfox 6mo ago
Do the larger LLM platforms just do this for you? Or perhaps they do this behinds the scenes, and charge you for the same amount of tokens?
- spullara 6mo ago
  no because tool calls are all client side generally. unless you mean using a remote environment where Claude Code is running separately but usually those aren't being charged by the token.
- fielding 6mo ago
  The tokens still land in the context window either way. Prompt caching gives you a discount on repeated input, but only for stable prefixes like system prompts. Git output changes every call, so it's always uncached, always full price. Nit reduces what goes into the window in the first place.
  - aeonfox 6mo ago
    I was thinking more if you write a prompt into an IDE that has first-party integration with an LLM platform (e.g. VS Code with Github Copilot), it would make sense on their end to reduce and remove redundant input before ingesting the token into their models, just to increase throughput (increase customers) and decrease latency (reduce costs). They would be foolish not to do this kind of optimisation, so surely they must be doing it. Whether they would pass on those token savings to the user, I couldn't say.
fielding 6mo ago
this is awesome! thanks for sharing rtk.. going to check it out.

slopinthebag 6mo ago

Was this entirely created by an LLM? The article clearly was.

tempest_ 6mo ago
Every "I created <xyz>" or "I rewrote x (y lang) into z lang" should really read "I prompted Claude Code to <insert thing>" "I" create stuff all the time with AI Agents but am real uncomfortable claiming ownership over them. Others don't seem to have this problem though so /shrug p.s. - in this case the commits are claude commits, even if they tell it not attribute itself you can tell because good commit messages were incredibly rare (even my own) until the last year or so when they started to look like entire pull requests
- slopinthebag 6mo ago
  Yeah, like whatever I prompt I'm fine sharing it, but I'm not gonna claim I made something. It's like claiming I'm an artist because I paid a guy to paint someone.
  - ofrzeta 6mo ago
    There are certainly artists that have helpers that do the actual execution.
  - fielding 6mo ago
    Is it though? The person who commissions a painting doesn't design the composition, validate every brushstroke, and run the output through an automated test suite. The analogy breaks down pretty fast.
    - slopinthebag 6mo ago
      > The person who commissions a painting doesn't design the composition They often do! Of course the artist has creative liberty to make it work, similar to how LLM's will deviate from the spec. Was your automated test suite also AI generated? You probably could have avoided all criticism by simply writing the article yourself instead of publishing raw LLM output. If someone isn't willing to write about a project they made, it's usually an indicator that they spent an equal amount of effort on the code as well. And why did you make a commit to remove em dashes? That seems odd.

warwickmcintosh 6mo ago

The 71% reduction is interesting but I'd want to see where those tokens are actually going in a typical agent session. In my experience running multi-step coding agents, the git output itself is rarely the bottleneck...

fielding 6mo ago
Not saying it is the bottleneck. It's bloat. 7.4% of all shell tokens across 3,156 sessions is a lot of unnecessary context. It won't make or break a session, but it adds up across thousands of calls.

mpalmer 6mo ago

You should expect to be gently ridiculed if you submit this as a serious project. Do not use LLMs to claim knowledge or abilities you don't actually have. It's not impressive, and it's obvious to anyone with more than a year or two of actual programming experience.

mpalmer 6mo ago
To elaborate: - The claim of "71% token savings" is misleading. You've already told us that git output is 7% of the "bash commands", so your token savings is really ~5% of bash command output. Also, the "200K tokens saved" refers the total saved across more than 3000 agent sessions. So it's really just ~60 tokens saved per session. That's a fraction of a fraction of a penny. Did you check your math here? Or just trust the model? - You have not rebuilt git, you have replaced a small part of its porcelain CLI. Most of what makes git git is in the library code. - Zig is a high-performance systems programming language that requires the programmer to manage memory manually. You've used it here to handle command line options (and yes, to call into libgit). That is overkill, and that matters, because: - The performance difference (e.g. 8ms vs 16ms) might be real, but it still doesn't matter at all. For one, there's no way you yourself could tell the difference. The slower time is still a single frame of 60fps. For another, this is not a hot path. These git commands - however fast - are swallowed up by the time it takes the agent to do inference. There's no question that inference itself will take orders of magnitude more time than a 16ms shell command.

000ooo000 6mo ago

So the big token saving was changing a setting that Git already allows you to change? And does 8ms really matter when you're shunting bulk crap back and forth from the cloud?

fielding 6mo ago
It goes beyond what I was able to do with git settings alone. Specifically stripping the headers/padding/decorative and it doees it across all output (or well a lot of it).