7 ms·
Show HN: ctx – Search the coding agent history already on your machine
Coding agents don't have long-term memory.
But you do have months of full-fidelity agent transcripts stored on your machine.
A simple solution that goes a long way: ingest those transcripts and logs into a structured SQLite database, then search them with ranked text match. Everything is fully local and doesn't require anything fancy like a graph database or hosted memory service.
This is the idea behind ctx, a Rust CLI that handles the ingestion and searching.
We give our agents a skill that tells them to reference past sessions before working in an area. Usually we do this through an "Agent History Research Subagent" whose job is just to prepare a short brief covering any relevant history before the task begins.
A real example: sometimes our test suite runs would fail because disk was full on the runner. The correct approach was to run the cleanup runbook, but the root cause of the failure was not clear to the agents, so they would think it was a test regression and go down the wrong rabbit hole debugging. When the agent searched history, it realized this failure had been encountered before and found the right workaround immediately. That got the agent onto the right cleanup path, and later we improved the log output so the same failure would be clearer next time. It's a boring story, but it's real agent productivity.
Another nice use case is quickly generating session transcripts for sharing. You can exclude the noisy intermediate messages, so the transcript shows the important parts of the session more cleanly. Try attaching a session transcript to your next PR so your teammate and their agent can review the provenance and prompting behind the change.
If you're up for an additional challenge, ask your agent to "exhaustively review all agent history in this repo and find where the SDLC is struggling or isn't agent-native". Using past sessions to recursively improve the agentic SDLC is a loop that we're using a lot today.
If you try it out, please let us know what you think!
- mewsen 2mo agoyo this is crap. stop emailing people from the AUR using your stupid email handle to advertise your ai slop. the "we" you often mention is most likely you and that clanker producing this slop.
- luca-ctx 3mo agoBuilding this made it obvious that there should be a standard format / specification for agent transcripts and logs (similar to ACP for runtime events). If you're interested in discussing this, please reach out!
- scritty-dev 3mo ago[flagged]
- dang 3mo agoCan you please not post AI-generated or AI-edited comments to HN? It's not allowed here - see https://news.ycombinator.com/newsguidelines.html#generated https://news.ycombinator.com/newsguidelines.html#generated and https://news.ycombinator.com/item?id=47340079 https://news.ycombinator.com/item?id=47340079. Of course, it's impossible to know for sure what was LLM processed or not, but some of your posts (like this one) have been getting classified that way.
- scritty-dev 3mo agomy post are not AI generated...I apologize if my tone/vernacular comes off as generated, but that's just my own voice. down voting my comment based on unfounded assumptions is upsetting and discouraging to a new member such as myself.
- meowface 3mo agoYou did write the above comment with an LLM, though. Why are you now denying it?
- dang 3mo agoWe can't know for sure, so it's best not to attack like this. All we can really say is that the posts were getting classified that way. There's also a large fuzzy area these days where people are using tools to edit, "polish", etc., but do not think of it as using an LLM to write. This is particularly the case with non-native English speakers. A few recent cases where this sort of thing came up: https://news.ycombinator.com/item?id=48467726 https://news.ycombinator.com/item?id=48467726 https://news.ycombinator.com/item?id=48416592 https://news.ycombinator.com/item?id=48416592 https://news.ycombinator.com/item?id=48405497 https://news.ycombinator.com/item?id=48405497
- AM1010101 3mo agoI would love a service that I could upload these chats to (anonymously) so that those developing open models can have it as training data and not just the closed model companies. My understanding is that it’s very valuable, look what cursor have managed to train. Obviously some filtering so that only chats or projects you want to share get shared would need to be in order.
- adamsmark 3mo agoThe Chinese would love this.
- luca-ctx 3mo agoWe have a private beta for a secure cloud version of the service, although its more geared towards teams/enterprise who want to share their work internally, rather than donating to open model developers. But interesting idea! I'm not very knowledgeable about crypto things, but I believe this is what people have considered "microtransactions" to be useful for.
- matheusmoreira 3mo agoI love this idea. I would totally upload my sessions if doing so helped train the next generation of open weight models. I'm using Claude to work on my free software projects so there aren't many secrets there anyway.
- wrs 3mo agoI often tell Claude Code to look at previous sessions in ~/.claude and it’s happy to jq/grep its way through them with no special tool. But being more efficient is always good.
- CuriouslyC 3mo agoClaude has been heavily RL'd on using jq/grep, even if the tool is more efficient, Claude using it incorrectly or reading a book of examples in order to understand how to use it correctly is going to end up underperforming.
- luca-ctx 3mo agoThis is likely true, but ultimately this tool is just SQL, which I believe Claude and others must be heavily RL'd on. We try to not do anything "special" and make it boringly SQL representation of past sessions.
- luca-ctx 3mo agoYes this is how we started as well! At first I thought the main improvement would be that the search would be faster, but rg is already pretty freakin fast when the fs cache is warm. What really ended up being the big efficiency improvement is the token efficiency. When you structure all of the transcripts in a SQL table, the agent can retrieve exactly what is needed (such as "print me the lite transcript, without the intermediate messages").
- Terretta 3mo ago> Coding agents usually start from zero. They can inspect the current repo, but they often cannot recover the discussions, decisions, failed attempts, commands, and test results from earlier work. Sure they can. Just ask them. Some (like Claude Code) even have built in tools for it that work a treat. It'll happily rebuild an entire edit history diff by diff.
- luca-ctx 3mo agoThis is true, maybe we could reword it to be less absolute. The bigger point is that when they do go spelunking in the old session logs, it is extremely token inefficient, and you can often fill up an entire context window and force a compaction just by trying to put together a transcript or summary. The goal here is less of doing something previously impossible, but doing it in a way that makes it so efficient and cheap that you can have agents do it very often, like before they start on every single task.
- beaugunderson 3mo agoThis is rapidly becoming the "todo list demo" of the LLM era... I say this as someone who also built one because I was tired of all the others! (https://github.com/beaugunderson/obliscence https://github.com/beaugunderson/obliscence)
- luca-ctx 3mo agoLol fair enough! Great project btw. Interesting choice to trigger incremental refresh on SessionStart hook, that's nice. How have you enjoyed the semantic search?
- beaugunderson 3mo agosemantic search has been pretty good, it usually finds what it's looking for! a couple of times I was certain that there was a session that contained some word but in reality it was in my personal claude.ai web account, so needed to add the import functionality there. my favorite piece is the `corrections` command which surfaces all my frustrations/corrections in the last week for example... and I can then figure out if missing context would improve those scenarios going forward
- luca-ctx 3mo agoNice, yea I typically spend about 1/3 of my sessions on finding ways to improve the agents' SDLC. Lots of random audits and things. And yea on the import thing, there are quite a few instances when session records can live on other machines, like cloud agents, dev boxes, etc. Do you have any interest in sharing some transcripts with team members? I'm trying to figure out the shape of this solution because often times people I work with want to see what I did or fork one of my sessions, but I also don't necessarily just want unlimited dumping because I'm sure I have personal details in there too.
- beaugunderson 3mo agosometimes i'll share prompts but but never a whole transcript (have not had a reason to) if i do want to share context i'll use something like "give me a prompt $coworker can share with their claude to continue this work"
- meowface 3mo agoThe number of LLMs (and possibly very odd humans) replying to this thread is unusual.
- zaptheimpaler 3mo agoI've been running https://github.com/kenn-io/agentsview https://github.com/kenn-io/agentsview for this, works well.
- linggen 3mo ago[flagged]
- malandin 3mo agoVery interesting project! I guess it could be even better if you didn't have to ingest the session data into a database but just build an index on top. I have an idea how to do it
- luca-ctx 3mo agoThanks and this is a very interesting idea! We considered this, but the main thing you gain from this tradeoff is some disk space and cleaner retention semantics from not having to duplicate all of the searchable text. But you still have to do the parsing and ingestion work to build the index in the first place, so CPU time does not go away. And you still have to store the indexes and enough metadata to map results back to the raw session files, which bounds the benefit of not duplicating the data. The main downside is flexibility (you would lose the ability to do arbitrary SQL queries, semantic search on top of structured corpus, etc) But I would love to see if I can be proven wrong on this!
- mkornaukhov 3mo agoWe've been working on remote search indexing in our project and it works pretty good. since we are building a Postgres-compatible database, everything is pure SQL. I'd say we could join forces if you're up for it.
- luca-ctx 3mo agoVery cool! Do you have any of it OSS? Or drop me an email: luca@ctx.rs
- alex_hirner 3mo agoI like the CLI interface, very comprehensive. However, I'm puzzled by pi support: https://github.com/ctxrs/ctx/issues/40 https://github.com/ctxrs/ctx/issues/40
- luca-ctx 3mo ago[dead]
- indigodaddy 3mo agoOr you could just use a great agent/harness like Shelley that already uses sqlite to store converians is searchable from the UI as well. https://github.com/boldsoftware/shelley https://github.com/boldsoftware/shelley
- gb2d_hn 3mo agoI worked on this problem with https://www.agentkanban.io https://www.agentkanban.io - this is a human in the loop integration for VS code that stores the context in the kanban task so it lives with the task (can also be forked, split etc).
- luca-ctx 3mo agoWe’re working on adding native support for Shelley! The idea is that even with native recall from Shelley, ctx results are more accurate, ergonomic, and token efficient For example search can retrieve a specific message and then window for trailing and leading N messages, in just a few hundred tokens
- sinisha_djukic 3mo agoReally nice idea - there is certainly gold in the history of agent actions. But how do you keep the ctx from trusting stale history?
- uikdjsah 3mo ago[flagged]
- meowface 3mo agoSeek help.
- luca-ctx 3mo agoIt's all in the skill / instruction that you give to the agent. The agent should treat the history as anthropology - a record of what happened, not necessarily the ground truth. Creating ground truth is an orthogonal problem - I try to work hard to put it into specs and docs and regularly update those. Searching history is closer to "super git blame" or like looking through logs. We should expect a lot of stuff went wrong in there.
- JeelVankhede 2mo ago[flagged]