10 ms·

You should write an agent

1070 points by tabletcorry 11mo ago

sibeliuss 11mo ago
Its easy to create a toy, but much harder to make something right! Like anything, so much weird polish stuff creeps in at the 90% mark.
- sumedh 11mo ago
  > so much weird polish stuff creeps in at the 90% mark. That is where the human in the loop needs to focus on for now :)
litestartup_com 11mo ago
[dead]
tlarkworthy 11mo ago
Yeah I was inspired after https://news.ycombinator.com/item?id=43998472 https://news.ycombinator.com/item?id=43998472 which is also very concrete
- tptacek 11mo ago
  I love everything they've written and also Sketch is really good.
manishsharan 11mo ago
How.. please don't say use langxxx library I am looking for a language or library agnostic pattern like we have MVC etc. for web applications. Or Gang of Four patterns but for building agents.
- tptacek 11mo ago
  The whole post is about not using frameworks; all you need is the LLM API. You could do it with plain HTTP without much trouble.
  - manishsharan 11mo ago
    When I ask for Patterns, I am seeking help for recurring problems that I have encountered. Context management .. small llms ( ones with small context size) break and get confused and forget work they have done or the original goal.
    - skeledrew 11mo ago
      That's why you want to use sub-agents which handle smaller tasks and return results to a delegating agent. So all agents have their own very specialized context window.
      - tptacek 11mo ago
        That's one legit answer. But if you're not stuck in Claude's context model, you can do other things. One extremely stupid simple thing you can do, which is very handy when you're doing large-scale data processing (like log analysis): just don't save the bulky tool responses in your context window once the LLM has generated a real response to them. My own dumb TUI agent, I gave a built in `lobotomize` tool, which dumps a text list of everything in the context window (short summary text plus token count), and then lets it Eternal Sunshine of the Spotless Agent things out of the window. It works! The models know how to drive that tool. It'll do a series of giant ass log queries, filling up the context window, and then you can watch as it zaps things out of the window to make space for more queries. This is like 20 lines of code.
        adiasg 11mo ago
        Did something similar - added `summarize` and `restore` tools to maximize/minimize messages. Haven't gotten it to behave like I want. Hoping that some fiddling with the prompt will do it.
        lbotos 11mo ago
        FYI -- I vouched for you to undead this comment. It felt like a fine comment? I don't think you are shadowbanned but consider emailing the mods if you think you might me.
- oooyay 11mo ago
  I'm not going to link my blog again but I have a reply on this post where I link to my blog post where I talk about how I built mine. Most agents fit nicely into a finite state machine or a directed acyclic graph that responds to an event loop. I do use provider SDKs to interact with models but mostly because it saves me a lot of boilerplate. MCP clients and servers are also widely available as SDKs. The biggest thing to remember, imo, is to keep the relationship between prompts, resources, and tools in mind. They make up a sort of dynamic workflow engine.
behnamoh 11mo ago
> nobody knows anything yet that sums up my experience in AI over the past three years. so many projects reinvent the same thing, so much spaghetti thrown at the wall to see what sticks, so much excitement followed by disappointment when a new model drops, so many people grifting, and so many hacks and workarounds like RAG with no evidence of them actually working other than "trust me bro" and trial and error.
- w_for_wumbo 11mo ago
  I think we'd get better results if we thought of it as a conscious agent. If we recognized that it was going to mirror back or unconscious biases and try to complete the task as we define it, instead of how we think it should behave. Then we'd at least get our own ignorance out of the way when writing prompts. Being able to recognize that 'make this code better' provides no direction, it should make sense that the output is directionless. But on more subtle levels, whatever subtle goals that we have and hold in the workplace will be reflected back by the agents. If you're trying to optimise costs, and increase profits as your north star. Having layoffs and unsustainable practices is a logical result, when you haven't balanced this with any incentives to abide by human values.
- sumedh 11mo ago
  That is because for the people for whom AI is actually working/making money they would prefer to keep it a secret on what and how they are doing it, why attract competition?
  - nylonstrung 11mo ago
    Who would you say it's working for? What products or companies are the gold standard of agent implementation right now?
oooyay 11mo ago
Heh, the bit about context engineering is palpable. I'm writing a personal assistant which, imo, is distinct from an agent in that it has a lot of capabilities a regular agent wouldn't necessarily need such as memory, task tracking, broad solutioning capabilities, etc... I ended up writing agents that talk to other agents which have MCP prompts, resources, and tools to guide them as general problem solvers. The first agent that it hits is a supervisor that specializes in task management and as a result writes a custom context and tool selection for the react agent it tasks. All that to say, the farther you go down this rabbit hole the more "engineering" it becomes. I wrote a bit on it here: https://ooo-yay.com/blog/building-my-own-personal-assistant/ https://ooo-yay.com/blog/building-my-own-personal-assistant/
- qwertox 11mo ago
  This sounds really great.
- cantor_S_drug 11mo ago
  https://github.com/mem0ai/mem0?tab=readme-ov-file https://github.com/mem0ai/mem0?tab=readme-ov-file Is this useful for you?
  - oooyay 11mo ago
    Could be! I'll give it a shot
esafak 11mo ago
What's wrong with the OWASP Top Ten?
- kennethallen 11mo ago
  Author on Twitter a few years ago: https://x.com/tqbf/status/851466178535055362 https://x.com/tqbf/status/851466178535055362
riskable 11mo ago
It's interesting how much this makes you want to write Unix-style tools that do one thing and only one thing really well. Not just because it makes coding an agent simpler, but because it's much more secure!
- chemotaxis 11mo ago
  You could even imagine a world in which we create an entire suite of deterministic, limited-purpose tools and then expose it directly to humans!
  - layer8 11mo ago
    I wonder if we could develop a language with well-defined semantics to interact with and wire up those tools.
    - chubot 11mo ago
      > language with well-defined semantics That would certainly be nice! That's why we have been overhauling shell with https://oils.pub https://oils.pub , because shell can't be described as that right now It's in extremely poor shape e.g. some things found from building several thousand packages with OSH recently (decades of accumulated shell scripts) - bugs caused by the differing behavior of 'echo hi | read x; echo x=$x' in shells, i.e. shopt -s lastpipe in bash. - 'set -' is an archaic shortcut for 'set +v +x' - Almquist shell is technically a separate dialact of shell -- namely it supports 'chdir /tmp' as well as cd /tmp. So bash and other shells can't run any Alpine builds. I used to maintain this page, but there are so many problems with shell that I haven't kept up ... https://github.com/oils-for-unix/oils/wiki/Shell-WTFs https://github.com/oils-for-unix/oils/wiki/Shell-WTFs OSH is the most bash-compatible shell, and it's also now Almquist shell compatible: https://pages.oils.pub/spec-compat/2025-11-02/renamed-tmp/spec/compat/TOP.html https://pages.oils.pub/spec-compat/2025-11-02/renamed-tmp/sp... It's more POSIX-compatible than the default /bin/sh on Debian, which is dash The bigger issue is not just bugs, but lack of understanding among people who write foundational shell programs. e.g. the lastpipe issue, using () as grouping instead of {}, etc. --- It is often treated like an "unknowable" language Any reasonable person would use LLMs to write shell/bash, and I think that is a problem. You should be able to know the language, and read shell programs that others have written
teiferer 11mo ago
Write an agent, it's easy! You will learn so much! ... let's see ... client = OpenAI() Um right. That's like saying you should implement a web server, you will learn so much, and then you go and import http (in golang). Yeah well, sure, but that brings you like 98% of the way there, doesn't it? What am I missing?
- victorbjorklund 11mo ago
  maybe more like “let’s write a web server but let’s use a library for the low level networking stack”. That can still teach you a lot.
- tptacek 11mo ago
  That OpenAI() is a wrapper around a POST to a single HTTP endpoint: POST https://api.openai.com/v1/responses
  - tabletcorry 11mo ago
    Plus a few other endpoints, but it is pretty exclusively an HTTP/REST wrapper. OpenAI does have an agents library, but it is separate in https://github.com/openai/openai-agents-python https://github.com/openai/openai-agents-python
- bootwoot 11mo ago
  That's not an agent, it's an LLM. An agent is an LLM that takes real-world actions
- MeetingsBrowser 11mo ago
  I think you might be conflating an agent with an LLM. The term "agent" isn't really defined, but its generally a wrapper around an LLM designed to do some task better than the LLM would on its own. Think Claude vs Claude Code. The latter wraps the former, but with extra prompts and tooling specific to software engineering.
- munchbunny 11mo ago
  An agent is more like a web service in your metaphor. Yes, building a web server is instructive, but almost nobody has a reason to do it instead of using an out of the box implementation once it’s time to build a production web service.
ATechGuy 11mo ago
Maybe we should write an agent that writes an agent that writes an agent...
chrisweekly 11mo ago
There's something(s) about @tptacek's writing style that has always made me want to root for fly.io.
qwertox 11mo ago
I've found it much more useful to create an MCP server, and this is where Claude really shines. You would just say to Claude on web, mobile or CLI that it should "describe our connectivity to google" either via one of the three interfaces, or via `claude -p "describe our connectivity to google"`, and it will just use your tool without you needing to do anything special. It's like custom-added intelligence to Claude.
- tptacek 11mo ago
  You can do this. Claude Code can do everything the toy agent this post shows, and much more. But you shouldn't, because doing that (1) doesn't teach you as much as the toy agent does, (2) isn't saving you that much time, and (3) locks you into Claude Code's context structure, which is just one of a zillion different structures you can use. That's what the post is about, not automating ping.
- mattmanser 11mo ago
  Honest question, as your comment confuses me. Did you get to the part where he said MCP is pointless and are saying he's wrong? Or did you just read the start of the article and not get to that bit?
  - vidarh 11mo ago
    I'd second the article on this, but also add to it that the biggest reason MCP servers don't really matter much any more is that the models are so capable of working with APIs, that most of the time you can just point them at an API and give them a spec instead. And the times that doesn't work, just give them a CLI tool with a good --help option. Now you have a CLI tool you can use yourself, and the agent has a tool to use. Anthropic itself have made MCP server increasingly pointless: With agents + skills you have a more composeable model that can use the model capabilities to do all an MCP server can with or without CLI tools to augment them.
    - simplesagar 11mo ago
      I feel the CLI vs MCP debate is an apples to oranges framing. When you're using claude you can watch it using CLI's, running brew, mise, lots of jq but what about when you've built an agent that needs to work through a complicated API? You don't want to make 5 CRUD calls to get the right answer. A curated MCP tool ensures it can determinism where it matters most.. when interacting with customer data
zkmon 11mo ago
A very good blog article that I have read in a while. Maybe MCP could have been involved as well?
_pdp_ 11mo ago
It is also very simple to be a programmer.. see, print "Hello world!" so easy...
- dan_can_code 11mo ago
  But that didn't use the H100 I just bought to put me out of my own job!
robot-wrangler 11mo ago
> Another thing to notice: we didn’t need MCP at all. That’s because MCP isn’t a fundamental enabling technology. The amount of coverage it gets is frustrating. It’s barely a technology at all. MCP is just a plugin interface for Claude Code and Cursor, a way of getting your own tools into code you don’t control. Write your own agent. Be a programmer. Deal in APIs, not plugins. Hold up. These are all the right concerns but with the wrong conclusion. You don't need MCP if you're making one agent, in one language, in one framework. But the open coding and research assistants that we really want will be composed of several. MCP is the only thing out there that's moving in a good direction in terms of enabling us to "just be programmers" and "use APIs", and maybe even test things in fairly isolated and reproducible contexts. Compare this to skills.md, which is actually defacto proprietary as of now, does not compose, has opaque run-times and dispatch, is pushing us towards certain models, languages and certain SDKs, etc. MCP isn't a plugin interface for Claude, it's just JSON-RPC.
- tptacek 11mo ago
  I think my thing about MCP, besides the outsized press coverage it gets, is the implicit presumption it smuggles in that agents will be built around the context architecture of Claude Code --- that is to say, a single context window (maybe with sub-agents) with a single set of tools. That straitjacket is really most of the subtext of this post. I get that you can use MCP with any agent architecture. I debated whether I wanted to hedge and point out that, even if you build your own agent, you might want to do an MCP tool-call feature just so you can use tool definitions other people have built (though: if you build your own, you'd probably be better off just implementing Claude Code's "skill" pattern). But I decided to keep the thrust of that section clearer. My argument is: MCP is a sideshow.
  - robot-wrangler 11mo ago
    I still don't really get it, but would like to hear more. Just to get it out of the way, there's obvious bad aspects. Re: press coverage, everything in AI is bound to be frustrating this way. The MCP ecosystem is currently still a lot of garbage. It feels like a very shitty app-store, lots of abandonware, things that are shipped without testing, the usual band-wagoning. For example instead of a single obvious RAG tool there's 200 different specific tools for ${language} docs The core MCP tech though is not only directionally correct, but even the implementation seems to have made lots of good and forward-looking choices, even if those are still under-utilized. For example besides tools, it allows for sharing prompts/resources between agents. In time, I'm also expecting the idea of "many agents, one generic model in the background" is going to die off. For both costs and performance, agents will use special-purpose models but they still need a place and a way to collaborate. If some agents coordinate other agents, how do they talk? AFAIK without MCP the answer for this would be.. do all your work in the same framework and language, or to give all agents access to the same database or the same filesystem, reinventing ad-hoc protocols and comms for every system.
solomonb 11mo ago
This work predates agents as we know them now and was intended for building chat bots (as in irc chat bots) but when auto-gpt I realized I could formalize it super nicely with this library: https://blog.cofree.coffee/2025-03-05-chat-bots-revisited/ https://blog.cofree.coffee/2025-03-05-chat-bots-revisited/ I did some light integration experiments with the OpenAI API but I never got around to building a full agent. Alas..
vkou 11mo ago
> It’s Incredibly Easy client = OpenAI() context_good, context_bad = [{ "role": "system", "content": "you're Alph and you only tell the truth" }], [{ "role": "system", "content": "you're Ralph and you only tell lies" }] ... And this will work great until next week's update when Ralph responses will consist of "I'm sorry, it would be unethical for me to respond with lies, unless you pay for the Premium-Super-Deluxe subscription, only available to state actors and firms with a six-figure contract." You're building on quicksand. You're delegating everything important to someone who has no responsibility to you.
- tptacek 11mo ago
  I love that the thing you singled out as not safe to run long term, because (apparently) of woke, was my weird deep-cut Labyrinth joke.
- sumedh 11mo ago
  Its easy to switch to an open source model
nowittyusername 11mo ago
I agree with the sentiment but I also recommend you build a local only agent. Something that runs on llama.cpp or vllm, whatever... This way you can better grasp the more fundamental nature of what LLM's really are and how they work under the hood. That experience will also make you realize how much control you are giving up when using cloud based api providers like OpenAI and why so mane engineers feel that LLM's are a "black box". Well duh buddy you been working with apis this whole time, of course you wont understand much working just with that.
- 8note 11mo ago
  ive been trying this for a few week, but i dont at all currently own hardware good enough to be useful for local inference. ill be trying again once i have written my own agent, but i dont expect to get any useful results compared to using some claude or gemini tokens
  - nowittyusername 11mo ago
    My man, we now have llms that are anywhere between 130 million to 1 trillion parameters available for us to run locally, I can guarantee there is a model for you there that even your toaster can run. I have a RTX 4090 but for most of my fiddling i use small models like Qwen 3 4b and they work amazing so there's no excuse :P.
    - 8note 11mo ago
      well, i got some gemini models running on my phone, but if i switch apps, android kills it, so the call to the server always hangs... and then the screen goes black the new laptop only has 16GB of memory total, with another 7 dedicated to the NPU. i tried pulling up Qwen 3 4B on it, but the max context i can get loaded is about 12k before the laptop crashes. my next attempt is gonna be a 0.5B one, but i think ill still end up having to compress the context every call, which is my real challenge
      - nowittyusername 11mo ago
zahlman 11mo ago
> Imagine what it’ll do if you give it bash. You could find out in less than 10 minutes. Spoiler: you’d be surprisingly close to having a working coding agent. Okay, but what if I'd prefer not to have to trust a remote service not to send me { "output": [ { "type": "function_call", "command": "rm -rf / --no-preserve-root" } ] } ?
- tptacek 11mo ago
  Obviously if you're concerned about that, which is very reasonable, don't run it in an environment where `rm -rf` can cause you a real problem.
  - awayto 11mo ago
    Also if you're doing function calls you can just have the command as one response param, and arguments array as another response param. Then just black/white list commands you either don't want to run or which should require a human to say ok.
    - aidenn0 11mo ago
      blacklist is going to be a bad idea since so many commands can be made to run other commands with their arguments.
      - awayto 11mo ago
        Yeah I agree. Ultimately I would suggest not having any kind of function call which returns an arbitrary command. Instead, think of it as if you were enabling capabilities for AppArmor, by making a function call definition for just 1 command. Then over time suss out what commands you need your agent do to and nothing more.
- worldsayshi 11mo ago
  There are MCP configured virtualization solutions that is supposed to be safe for letting LLM go wild. Like this one: https://github.com/zerocore-ai/microsandbox https://github.com/zerocore-ai/microsandbox I haven't tried it.
dagss 11mo ago
I realize now what I need in Cursor: A button for "fork context". I believe that would be a powerful tool solving many things there are now separate techniques for.
- all2 11mo ago
  crush-cli has this. I think the google gemini chat app also has this now.
ericd 11mo ago
Absolutely, especially the part about just rolling your own alternative to Claude Code - build your own lightsaber. Having your coding agent improve itself is a pretty magical experience. And then you can trivially swap in whatever model you want (Cerebras is crazy fast, for example, which makes a big difference for these many-turn tool call conversations with big lumps of context, though gpt-oss 120b is obviously not as good as one of the frontier models). Add note-taking/memory, and ask it to remember key facts to that. Add voice transcription so that you can reply much faster (LLMs are amazing at taking in imperfect transcriptions and understanding what you meant). Each of these things takes on the order of a few minutes, and it's super fun.
- anonym29 11mo ago
  Cerebras now has glm 4.6. Still obscenely fast, and now obscenely smart, too.
  - ericd 11mo ago
    Ooh thanks for the heads up!
  - DeathArrow 11mo ago
    Aren't there cheaper providers of GLM 4.6 on Openrouter? What are the advantages of using Cerebras? Is it much faster?
    - simonw 11mo ago
      It's astonishingly fast.
    - meeq 11mo ago
      You know how sometimes when you send a prompt to Claude, you just know it’s gonna take a while, so you go grab a coffee, come back, and it’s still working? With Cerebras it’s not even worth switching tabs, because it’ll finish the same task in like three seconds.
    - anonym29 10mo ago
threecheese 11mo ago
Does anyone have an understanding - or intuition - of what the agentic loop looks like in the popular coding agents? Is it purely a “while 1: call_llm(system, assistant)”, or is there complex orchestration? I’m trying to understand if the value for Claude Code (for example) is purely in Sonnet/Haiku + the tool system prompt, or if there’s more secret sauce - beyond the “sugar” of instruction file inclusion via commands, tools, skills etc.
- CraftThatBlock 11mo ago
  Generally, that's pretty much it. More advanced tools like Claude Code will also have context compaction (which sometimes isn't very good), or possibly RAG on code (unsure about this, I haven't used any tools that did this). Context compaction, to my understanding, is just passing all the previous context into a call which summarizes it, then that becomes to new context starting point.
- colonCapitalDee 11mo ago
  I thought this was informative: https://minusx.ai/blog/decoding-claude-code/ https://minusx.ai/blog/decoding-claude-code/
- mrkurt 11mo ago
  Claude Code is an obfuscated javascript app. You can point Claude Code at it's own package and it will pretty reliably tell you how it works. I think Claude Code's magic is that Anthropic is happy to burn tokens. The loop itself is not all that interesting. What is interesting is how they manage the context window over a long chat. And I think a fair amount of that is serverside.
  - AdieuToLogic 11mo ago
    > Claude Code is an obfuscated javascript app. You can point Claude Code at it's own package and it will pretty reliably tell you how it works. This is why I keep coming back to Hacker News. If the above is not a quintessential "hack", then I've never seen one. Bravo!
fsndz 11mo ago
I did that, burned 2.6B tokens in the process and learned a lot: https://transitions.substack.com/p/what-burning-26-billion-prompt-tokens https://transitions.substack.com/p/what-burning-26-billion-p...
whyryoulikethis 11mo ago
[dead]
Zak 11mo ago
> You only think you understand how a bicycle works, until you learn to ride one. I bet a majority of people who can ride a bicycle don't know how they steer, and would describe the physical movements they use to initiate and terminate a turn inaccurately. https://en.wikipedia.org/wiki/Countersteering https://en.wikipedia.org/wiki/Countersteering
- captainkrtek 11mo ago
  Relevant interesting tangent: “Most People Don't Know How Bikes Work” https://www.youtube.com/watch?v=9cNmUNHSBac https://www.youtube.com/watch?v=9cNmUNHSBac
- itsmemattchung 11mo ago
  Reminds me of this YouTube video (below) on how difficult it is (nearly impossible) to re-learn how to ride a bicycle when you have the handles are reversed (i.e. pulling left handle bar towards you, the wheel goes to the right) https://www.youtube.com/watch?v=MFzDaBzBlL0 https://www.youtube.com/watch?v=MFzDaBzBlL0
- vinhnx 11mo ago
  A Brief History of Bicycle Engineering https://www.youtube.com/watch?v=EcRlDCsZM20 https://www.youtube.com/watch?v=EcRlDCsZM20
rbren 11mo ago
Spoiler: it's not actually that easy. Compaction, security, sandboxing, planning, custom tools--all this is really hard to get right. We're about to launch an SDK that gives devs all these building blocks, specifically oriented around software agents. Would love feedback if anyone wants to look: https://github.com/OpenHands/software-agent-sdk https://github.com/OpenHands/software-agent-sdk
- solarkraft 11mo ago
  How autonomous/controllable are the agents with this SDK? When I build an agent my standard is Cursor, which updates the UI at every reportable step of the way, and gives you a ton of control opportunities, which I find creates a lot of confidence. Is this level of detail and control possible with the OpenHands SDK? I’m asking because the last SDK that was simple to get into lacked that kind of control.
  - rbren 11mo ago
    That's the idea! We have a confirmation_mode that can interrupt at any step in the process.
- olingern 11mo ago
  Only on HN is there a “well, actually” with little substance followed by a comment about a launch. The article isn’t about writing production ready agents, so it does appear to be that easy
dave1010uk 11mo ago
Two years ago I wrote an agent in 25 lines of PHP [0]. It was surprisingly effective, even back then before tool calling was a thing and you had to coax the LLM into returning structured output. I think it even worked with GPT-3.5 for trivial things. In my mind LLMs are just UNIX strong manipulation tools like `sed` or `awk`: you give them an input and command and they give you an output. This is especially true if you use something like `llm` [1]. It then seems logical that you can compose calls to LLMs, loop and branch and combine them with other functions. [0] https://github.com/dave1010/hubcap https://github.com/dave1010/hubcap [1] https://github.com/simonw/llm https://github.com/simonw/llm
- simonw 11mo ago
  I love hubcap so much. It was a real eye-opener for me at the time, really impressive result for so little code. https://simonwillison.net/2023/Sep/6/hubcap/ https://simonwillison.net/2023/Sep/6/hubcap/
  - dingnuts 11mo ago
    You're posting too fast please slow down
    - rolymath 10mo ago
      I agree. I'm getting too much simonw in my feed. Getting too saturated.
  - dave1010uk 11mo ago
    Thanks Simon! It only worked because of your LLM tool. Standing on the shoulders of giants.
- keyle 11mo ago
  > a small Autobot that you can't trust That gave me a hearty chuckle!
  - nativeit 11mo ago
imiric 11mo ago
> Give each call different tools. Make sub-agents talk to each other, summarize each other, collate and aggregate. Build tree structures out of them. Feed them back through the LLM to summarize them as a form of on-the-fly compression, whatever you like. You propose increasing the complexity of interactions of these tools, and giving them access to external tools that have real-world impact? As a security researcher, I'm not sure how you can suggest that with a straight face, unless your goal is to have more vulnerable systems. Most people can't manage to build robust and secure software using SOTA hosted "agents". Building their own may be a fun learning experience, but relying on a Rube Goldberg assembly of disparate "agents" communicating with each other and external tools is a recipe for disaster. Any token could trigger a cascade of hallucinations, wild tangents, ignored prompts, poisoned contexts, and similar issues that have plagued this tech since the beginning. Except that now you've wired them up to external tools, so maybe the system chooses to wipe your home directory for whatever reason. People nonchalantly trusting nondeterministic tech with increasingly more real-world tasks should concern everyone. Today it's executing `ping` and `rm`; tomorrow it's managing nuclear launch systems.
8note 11mo ago
> A subtler thing to notice: we just had a multi-turn conversation with an LLM. To do that, we remembered everything we said, and everything the LLM said back, and played it back with every LLM call. The LLM itself is a stateless black box. The conversation we’re having is an illusion we cast, on ourselves. the illusion was broken for me by Cline context overflows/summaries, but i think its very easy to miss if you never push the LLM hard or build you own agent. I really like this wording, amd the simple description is missing from how science communicators tend to talk about agents and LLMs imo
a-dub 11mo ago
they kinda feel like the cgi perl scripts of the mid 2020s.
- indeyets 11mo ago
  You mean late 1990’s? :)
  - a-dub 11mo ago
    no i mean, back in the 90's cgi perl scripts were the easy it thing for interacting with the big tech wave and now in the mid-2020s llm python agent scripts with tool extensions are the easy it thing for interacting with the big tech wave.
    - oblio 10mo ago
      Now we need PHP and Ruby or Rails, somewhere down the line :-))
wayy 11mo ago
everybody loves building agents, nobody likes debugging them. agents hit the classic llm app lifecycle problem: at first it feels magical. it nails the first few tasks, doing things you didn’t even think were possible. you get excited, start pushing it further. you run it and then it fails on step 17, then 41, then step 9. now you can’t reproduce it because it’s probabilistic. each step takes half a second, so you sit there for 10–20 minutes just waiting for a chance to see what went wrong
- furyofantares 11mo ago
  That's why you build extensive tooling to run your change hundreds of times in parallel against the context you're trying to fix, and then re-run hundreds of past scenarios in parallel to verify none of them breaks.
  - ht96 11mo ago
    Do you use a tool for this? Is there some sort of tool which collects evals from live inferences (especially those which fail)
    - AdieuToLogic 11mo ago
      There is no way to prove the correctness of non-deterministic (a.k.a. probabilistic) results for any interesting generative algorithm. All one can do is validate against a known set of tests, with the understanding that the set is unbounded over time.
    - aenis 11mo ago
      For sure, for instance Google has ADK Eval framework. You write tests, and you can easily run them against given input. I'd say its a bit unpolished, as is the rest of the rapidly developing ADK framework, but it does exist.
    - saturatedfat 11mo ago
      heya, building this. been used in prod for a month now, has saved my customer’s ass while building general workflow automation agents. happy to chat if ur interested. darin@mcptesting.com (gist: evals as a service)
hoppp 11mo ago
I should? what problems can I solve, that can be only done with an agent? As long as every AI provider is operating at a loss starting a sustainably monetizable project doesn't feel that realistic.
- throwaway6977 11mo ago
  You can be your own AI provider.
  - bilbo0s 11mo ago
    >starting a sustainably monetizable project doesn't feel that realistic. and >You can be your own AI provider. Not sure that being your own AI provider is "sustainably monetizable"?
  - hoppp 11mo ago
    For internal software maybe, but for a client facing service the incentives are not right when the norm is to operate at a loss.
- furyofantares 11mo ago
  > As long as every AI provider is operating at a loss None of them are doing that. They need funding because the next model has always been much more expensive to train than the profits of the previous model. And many do offer a lot of free usage which is of course operated at a loss. But I don't think any are operating inference at a loss, I think their margins are actually rather large.
  - hoppp 11mo ago
    When comparing the cost of an H100 GPU per hour and calculating cost of tokens, it seems the OpenAI offering for the latest model is 5 times cheaper than renting the hardware. OpenAI balance sheet also shows an $11 billion loss . I can't see any profit on anything they create. The product is good but it relies on investors fueling the AI bubble.
andai 11mo ago
.text-gray-600 { color: black; }
8cvor6j844qw_d6 11mo ago
Question, how hard is it for someone new to agents to dip their toes into writing a simple agent to get data? (e.g., getting reviews from sites for sentiment analysis?) Forgive if I get someting wrong: From what I see, it seems fundamentally it is a LLM being ran each loop with information about tools provided to it. On each loop the LLM evaluates inputs/context (from tool calls, inputs, etc.) and decided which tool to call / text output.
- simonw 11mo ago
  You can prototype this without writing any code at all. Fire up "claude --dangerously-skip-permissions" in a fresh directory (ideally in a Docker container if you want to limit the chance of it breaking anything else) and prompt this: > Use Playwright to fetch ten reviews from http://www.example.com/ http://www.example.com/ then run sentiment analysis on them and write the results out as JSON files. Install any missing dependencies. Watch what it does. Be careful not to let it spider the site in a way that would justifiably upset the site owners.
  - sumedh 11mo ago
    Dont you need to setup Playwright MCP first?
    - simonw 11mo ago
      No. I don't use Playwright MCP at all - if the coding agent can run Python code it can use the Playwright Python library directly, if Node.js it can use the Playwright Node library.
      - sumedh 11mo ago
        Interesting, thanks for the info. I wanted to run claude headlessly (-p) and playwright headlessly to get some content. I was using Playwright MCP and for some reason claude in headless mode could not open playwright MCP in headless mode. I never realized i can just use playwright directly without the playwright MCP before your comment. Thanks once again.
vinhnx 11mo ago
> “You only think you understand how a bicycle works, until you learn to ride one.” This resonates deeply with me. That's why I built one myself [0], I really really love to truly understand how coding agents work. The learning has been immense for me, I now have working knowledge of ANSI escape codes, grapheme clusters, terminal emulators, Unicode normalization, VT protocols, PTY sessions, and filesystem operations - all the low-level details I would have never think about until I were implementing them. [0] https://github.com/vinhnx/vtcode https://github.com/vinhnx/vtcode
- dfex 11mo ago
  >> “You only think you understand how a bicycle works, until you learn to ride one.” > This resonates deeply with me. That's why I built one myself [0] I was hoping to see a home-made bike at that link.. Came away disappointed
  - vinhnx 11mo ago
    Good one! Sorry to disappoint you. But personally, that line strike deeply with me, honestly.
- lowbloodsugar 11mo ago
  It's conflating two issues though. Most people who can ride a bike can't explain the physics. They really don't know how it works. The bicycle lesson is about training the brain on a new task that cannot be taught in any other way. This case is more like a journeyman blacksmith who has to make his own tools before he can continue. In doing so, he gets tools of his own, but the real reward was learning what is required to handle the metal such that it makes a strong hammer. And like the blacksmith, you learn more if you use an existing agent to write your agent.
  - vinhnx 11mo ago
    Agree, to me, the wheel is the greatest invention of all. Everyone could have rode a bike, but the underlying physic and motion that came to `riding` is a whole another story.
hshdhdhehd 11mo ago
There is a lot of stuff I should do. From making my own CPU from a breadboard of nand gates to building a CDN in Rust. But aint got time for all the things. That said I built an LLM following Karpathy's tutorial. So I think it aims good to dabble a bit.
- coffeecoders 11mo ago
  Yeah, it’s a never-ending curve. I built an 8-bit computer on breadboards once, then went down the rabbit hole of flight training for a PPL. Every time I think I’m "done," the finish line moves a few miles further. Guess we nerds are never happy.
  - javchz 11mo ago
    One should be melting sand to get silicon, anything else it's too abstract to my taste.
    - tomcam 11mo ago
      Glad you’ve got all that time on your hands. I am still working on the fusion reactor portion of my supernova simulator, so that I can generate the silicon you so blithely refer to.
  - krsdcbl 11mo ago
    Given the premise, one could also say we nerds are forever happy.
  - ericmcer 11mo ago
    Seriously I feel like it's self-sabotage sometimes at work. Just fixing the thing getting tests to pass isn't enough. Until I fully have a mental model of what is happening I can't move on.
- qwertygnu 11mo ago
  Very early in TFA it explains how easy it is to do. That's the whole point of the post.
throwaway8xak92 11mo ago
[flagged]
- simonw 11mo ago
  I don't think "insane to not believe in vibe coding" is a fair summary of https://fly.io/blog/youre-all-nuts/ https://fly.io/blog/youre-all-nuts/ - that post wasn't about vibe coding (at least by its I-think-correct definition of prompt-driven coding where you don't pay any attention to the code that's being written), it was about AI-assisted engineering by professional software developers. It did have some swear words in - as did many of the previous posts on the Fly.io corporate blog.
  - AceJohnny2 11mo ago
    Worth highlighting that both OP article and the one Simon linked are by @tptacek, who is also one of the top commenters here on HN. His fly.io posts are very much in his style. I figure they let him post there, without corp-washing, because any publicity is good publicity.
    - tptacek 11mo ago
      This is the corp-washed version of this post.
      - AceJohnny2 11mo ago
        can I have access to the corp-unwashed version
rambojohnson 11mo ago
The bravado posturing in this article is nauseating. Sure, there are a few serious points buried in there, but damn...dial it down, please.
wahnfrieden 11mo ago
The Codex agent has an official TypeScript SDK now. Why would Fly.io advocate using the vanilla GPT API to write an agent, instead of the official agent?
- tptacek 11mo ago
  Because you won't learn as much using an agent framework, and, as you can see from the post, you absolutely don't need one.
azimux 11mo ago
I wrote an agent from scratch in Ruby several months back. Was fun! These 4 lines wound up being the heart of it, which is surprisingly simple, conceptually. until mission_accomplished? or given_up? or killed? determine_next_command_and_inputs run_next_command end
jbmsf 11mo ago
I agree. I find LLMs a bit overblown. I don't think most people want to use chat as their primary interface. But writing a few agents was incredibly informative.
zb3 11mo ago
No, because I know that "agents" are token burning machines - for me they're less efficient than the chat interface, slower and burning much more tokens. I'm not surprised that AI companies would want me to use them though.. I know what you're doing there :)
byronic 11mo ago
The author shoulda written a REPL
rmoriz 11mo ago
Side note: While the example uses GPT-5, the query interface is already some kind of industry standard. For example you could easily connect OpenRouter.ai and switch models and providers during runtime as needed. OpenRouter also has free models like some of the DeepSeek. While they are slow/rate limited and quantized, they are great for examples and playing around with it. https://openrouter.ai/models?fmt=cards&order=pricing-low-to-high https://openrouter.ai/models?fmt=cards&order=pricing-low-to-...
almaight 11mo ago
So I wrote an MCP using your code: https://gurddy-mcp.fly.dev https://gurddy-mcp.fly.dev. You can get the source code from https://github.com/novvoo/gurddy-mcp https://github.com/novvoo/gurddy-mcp.
aaronblohowiak 11mo ago
THEY SEND THE WHOLE CONTEXT EVERY TIME? Man that seems... not great. sometimes it will go off and spin on something.. seems like it would be a LOT better to roll back than to send a corrective message. hmmm...... this article is nerd-sniping on a massive scale ;D
- tptacek 11mo ago
  In the Responses API, you can implicitly chain messages with `previous_response_id` (I'm not sure how old a conversation you can resurrect that way). But I think Codex CLI actually sends the full context every time? And keep in mind, sending the whole context gives you fine-grained control over what does and doesn't appear in your context window. Anyways, if it nerd sniped you, I succeeded. :)
  - aaronblohowiak 11mo ago
    Yes indeed you did succeed. I totally want to try gaslighting an LLM now! Ah to find the time…
- cantor_S_drug 11mo ago
  There is context caching in many models. It is less expensive if you enable that.
- michaelanckaert 11mo ago
  Sending the whole context on each user message is essentially what the model remembers of this conversation. ie: it is entirely stateless. I've written some agents that have their context altered by another llm to get it back on track. Let's say the agent is going off rails, then a supervisor agent will spot this and remove messages from the context where it went off rails, or alter those with correct information. Really fun stuff but yeah, we're essentially still inventing this as we go along.
larusso 11mo ago
Just the other day we tried to explain inner workings of cursor etc to a bunch of colleagues who had a very complicated view how these agents achieve what they do. Awesome post. Makes it easier for me the next time. The options are so big. But one should say that an agent with file access etc, is easy to write but hard to control. If you want to build yourself a general coding agent a bit more thought needs to be put into the whole thing. Otherwise you might end up with a “dd -if=/dev/random -of=/“ or something ^^ and happily execute it.
fatesblind 11mo ago
[dead]
gloosx 11mo ago
Didn't see such a bad piece of writing for a long time. Serously guys, is it just me? It's hard to read for some reason.
jq_2023 11mo ago
the point around MCPs is spot on
psychoslave 11mo ago
It really reads to me like, "you should build a running water circuit", then presenting you how easy it is to phone a plumber and let them free ride on the matter, but beware to not use a project manager as real people implement project management of plumbery themselves."
- tptacek 11mo ago
  You're going to have to explain that analogy to me, sorry.
  - psychoslave 11mo ago
    Sure, phone call to plumber is remote call to turn key API, and manager layer is MVP. Hope that makes it more clear.
p0w3n3d 11mo ago
Actually tool "ping 8.8.8.8" never quits unless running on windows. This can spawn many processes that kill the server. This is one of the first production grade errors I've made when I started my programming. I had a widget that would ping the network, but every time someone went on the page, a new ping process would spawn
- sanxiyn 11mo ago
  If you look at the actual code, it runs ping -c 5. I agree ping without options doesn't terminate.
DeathArrow 11mo ago
You should write agents if you want to learn how agents work, if the problem you are trying to solve is not solved yet or if you are convinced that you will do much better job solving the problem again. Otherwise is just reinventing the wheel.
DeathArrow 11mo ago
I am thinking of building agents that can partly replace manual testing using a headless browser.
worldsayshi 11mo ago
I feel like one small piece is missing to call it an agent? The ability to iterate in multiple steps until it feels like it's "done". What is the canonical way to do that? I suspect that implementing that in the wrong way could make it spiral.
- cornel_io 11mo ago
  When a tool call completes the result is sent back to the LLM to decide what to do next, that's where it can decide to go do other stuff before returning a final answer. Sometimes people use structured outputs or tool calls to explicitly have the LLM decide when it's done, or allow it to send intermediate messages for logging to the user. But the simple loop there lets the LLM do plenty of it has good tools.
  - worldsayshi 11mo ago
    So it returns a tool call for "continue" every time it wants to continue working? Do people implement this in different ways? It would be nice what method it has been trained on if any.
    - tptacek 11mo ago
      The model will quickly stop tool calling on its own; in fact, I've had more trouble getting GPT5 to tool call enough. The "real" loop is driven, at each iteration, by a prompt from the "user" (which might be human or might be human-mediated code that keeps supplying new prompts). In my personal agent, I have a system prompt that tells the model to generate responses (after absorbing tool responses) with <1>...</1> <2>...</2> <3>...</3> delimited suggestions for next steps; my TUI presents those, parsed out of the output, as a selector, which is how I drive it.
DeathArrow 11mo ago
I would like an LLM to be integrated in the shell so I don't have to learn all the Unix tools arguments and write Bash scripts.
deleted 11mo ago
[deleted]
globular-toast 11mo ago
The formatting of the code is messed up on my phone. I was looking at the first bit thinking `call` was a function returning `None`. I thought initially it was doing some clever functional programming stuff but, no, just a linebreak that shouldn't be there.
otsaloma 11mo ago
Agreed! It's easy understand "LLM with tools in a loop" at a high-level, but once you actually design the architecture and implement the code in full, you'll have proper understanding of how it all fits and works together. I did the same exercise. My implementation is at around 300 lines with two tools: web search and web page fetch with a command line chat interface and Python package. And it could have been a lot less lines if I didn't want to write a usable, extensible package interface. As the agent setup itself is simple, majority of the work to make this useful would in the tools themselves and context management for the tools.
lazy_afternoons 11mo ago
Seriously, what is the advantage of tools at all. Why not implement custom string based triggers. First of all, the call accuracy is much higher. Second, you get more consistent results across models.
joelthelion 11mo ago
If you want to play with this stuff without spending a lot of money, what are your best options?
- thatscot 11mo ago
  Most cloud providers, like Azure have free credits at the start. On azure you can deploy your own model and pay with the free credits.
- thatscot 11mo ago
  You can just stick a tenner in OpenAI though and it won't charge anymore than the credit you've put in
- thatscot 11mo ago
  and sorry, forgot you can also run local models aswell :)
- beklein 11mo ago
  I love OpenRouter, since it is a simple way to get started and provides a wide range of available models. You can buy credits and set usage limits for safe testing per API key to gain access from many AI models through one simple and unified API from all popular model providers (OpenAI, Anthropic, Google, xAI, DeepSeek, Z.AI, Qwen, ...) Ten dollars is plenty to get started... experiments like in the post will cost you cents, not dollars.
- simonw 11mo ago
  Gemini has a generous free tier (2500 prompts per day), all you need is a Google account to get an API key.
amelius 11mo ago
Why write an agent when you can just ask the LLM to write one?
TYPE_FASTER 11mo ago
The Google Agent Development Kit (https://google.github.io/adk-docs/ https://google.github.io/adk-docs/) is really fun to play with. It's open source and supports both using a LLM in the cloud and running locally.
novoreorx 11mo ago
Reminds of this one [1] that I read half a year ago, which I used to develop my first agent. But what fly wrotes is definitely easier to understand, how I wish it was written a year earlier. [1]: https://ampcode.com/how-to-build-an-agent https://ampcode.com/how-to-build-an-agent
fauria 11mo ago
> I’m not even going to bother explaining what an agent is. Does anyone actually know what exactly an agent is?
- tptacek 11mo ago
  Yes, and the post says what it is about 100 words later. It's an LLM running in a loop that can access tool calls.
MinimalAction 11mo ago
Do we need an agent? I get the point of this post: have fun building one because it's easy. But every time I see one of these takes, I keep wondering why do we encourage a tool that would potentially replace us. Why help it build better that could eventually take away what was fun and sustainable income-wise?
- AlecSchueler 11mo ago
  Interesting to think that this question could have been asked of almost all software work up until this point, except the "us" was always "someone else "
  - DrewADesign 11mo ago
    It’s generally been true, but not close to the scale we’re looking at now. The implied/assumed hypocrisy also doesn’t stop it from it sucking, or make it immune to criticism.
    - AlecSchueler 11mo ago
      Indeed it probably sucks even more in a "you reap what you sow" kind of way :(
- tptacek 11mo ago
  Easy answer: so you can more sharply criticize them, rather than falling into the rhetorical traps of people who don't understand how they work well enough to sound credible. It's so little effort to get to that point!
- richardlblair 11mo ago
  I've been building tools for stuff I don't want to do. Any task where I need to take some amount of data, structured or unstructured, and need a specific outcome is perfect. That way I can spend more time on the thing I do want to do (including building these little tools).
  - 11mo ago
thedangler 11mo ago
Cool, can you make it use local free models because I'm broke and can't afford AI's crazy costs.
- Spivak 11mo ago
  Yep, change nothing in the code in the article but spin up an Ollama server and use the OpenAI API https://docs.ollama.com/api/openai-compatibility https://docs.ollama.com/api/openai-compatibility.
artursapek 11mo ago
I've been having so much fun writing the agent loop for https://revise.io https://revise.io, most fun I've had programming in a long time.
losvedir 11mo ago
I appreciate the goal of demystifying agents by writing one yourself, but for me the key part is still a little obscured by using OpenAI APIs in the examples. A lot of the magic has to do with tool calls, which the API helpfully wraps for you, with a format for defining tools and parsed responses helpfully telling you the tools it wants to call. I kind of am missing the bridge between that, and the fundamental knowledge that everything is token based in and out. Is it fair to say that the tool abstraction the library provides you is essentially some niceties around a prompt something like "Defined below are certain 'tools' you can use to gather data or perform actions. If you want to use one, please return the tool call you want and it's arguments, delimited before and after with '###', and stop. I will invoke the tool call and then reply with the output delimited by '==='". Basically, telling the model how to use tools, earlier in the context window. I already don't totally understand how a model knows when to stop generating tokens, but presumably those instructions will get it to output the request for a tool call in a certain way and stop. Then the agent harness knows to look for those delimiters and extract out the tool call to execute, and then add to the context with the response so the LLM keeps going. Is that basically it? Or is there more magic there? Are the tool call instructions in some sort of permanent context, or could the interaction demonstrated in a fine tuning step, and inferred by the model and just in its weights?
- JoshMandel 11mo ago
  I think that it's basically fair and I often write simple agents using exactly the technique that you describe. I typically provide a TypeScript interface for the available tools and just ask the model to respond with a JSON block and it works fine. That said, it is worth understanding that the current generation of models is extensively RL-trained on how to make tool calls... so they may in fact be better at issuing tool calls in the specific format that their training has focused on (using specific internal tokens to demarcate and indicate when a tool call begins/ends, etc). Intuitively, there's probably a lot of transfer learning between this format and any ad-hoc format that you might request inline your prompt. There may be recent literature quantifying the performance gap here. And certainly if you're doing anything performance-sensitive you will want to characterize this for your use case, with benchmarks. But conceptually, I think your model is spot on.
- 11mo ago
deadbabe 11mo ago
The more I use agents, the more I find agents to be pointless, any tasks an agent performs regularly in high volume should be turned into classical deterministic code. The number one feature of agents is to be disambiguation for tool selectors and pretty printers.
lbeurerkellner 11mo ago
Everybody should try. It helps a ton to demystify the relatively simple but powerful underpinning of how modern agents work. You can get quite far quite quickly. My toy implementation [1] is <600 LOC and even supports MCP. [1] https://github.com/lbeurerkellner/agent.py https://github.com/lbeurerkellner/agent.py
khazhoux 11mo ago
Agree 100% with premise of the article. I feel like the big secret of the recent advances in LLM tooling is that these are all just variations of “send a chat request and process the output.” Even Tool Calling is just wrapping one chat request with another hidden one that is asking which of N tools applies and what the parameters should be. RAG is simply pre-loading a bunch of extra text into the chat request, etc. My main point being, though: for anyone intimidated by the recent tooling advances… you can most definitely do all this yourself.
nitwit005 11mo ago
> You only think you understand how a bicycle works, until you learn to ride one. I realize this is just for motivation in a subtitle, but people generally don't grasp how bicycles work, even after having ridden one. Veritasium has a quite good video on the subject: https://www.youtube.com/watch?v=9cNmUNHSBac https://www.youtube.com/watch?v=9cNmUNHSBac
scrubs 10mo ago
"You don’t have to like them, but you should want to be right about them. To be the best hater (or stan) you can be." The op has a point - a good one
gtukmanov 10mo ago
The evolution of software agents leveraging LLMs as versatile building blocks is exciting. It underscores the shift towards modular, composable AI workflows that can integrate deterministic functions with generative intelligence. Great food for thought on how these tools might transform productivity and automation across industries

1070 points by tabletcorry 11mo ago

Its easy to create a toy, but much harder to make something right! Like anything, so much weird polish stuff creeps in at the 90% mark.

sumedh 11mo ago
> so much weird polish stuff creeps in at the 90% mark. That is where the human in the loop needs to focus on for now :)

litestartup_com 11mo ago

[dead]

tlarkworthy 11mo ago

Yeah I was inspired after https://news.ycombinator.com/item?id=43998472 https://news.ycombinator.com/item?id=43998472 which is also very concrete

tptacek 11mo ago
I love everything they've written and also Sketch is really good.

manishsharan 11mo ago

How.. please don't say use langxxx library I am looking for a language or library agnostic pattern like we have MVC etc. for web applications. Or Gang of Four patterns but for building agents.

tptacek 11mo ago
The whole post is about not using frameworks; all you need is the LLM API. You could do it with plain HTTP without much trouble.
- manishsharan 11mo ago
  When I ask for Patterns, I am seeking help for recurring problems that I have encountered. Context management .. small llms ( ones with small context size) break and get confused and forget work they have done or the original goal.
  - skeledrew 11mo ago
    That's why you want to use sub-agents which handle smaller tasks and return results to a delegating agent. So all agents have their own very specialized context window.
    - tptacek 11mo ago
      That's one legit answer. But if you're not stuck in Claude's context model, you can do other things. One extremely stupid simple thing you can do, which is very handy when you're doing large-scale data processing (like log analysis): just don't save the bulky tool responses in your context window once the LLM has generated a real response to them. My own dumb TUI agent, I gave a built in `lobotomize` tool, which dumps a text list of everything in the context window (short summary text plus token count), and then lets it Eternal Sunshine of the Spotless Agent things out of the window. It works! The models know how to drive that tool. It'll do a series of giant ass log queries, filling up the context window, and then you can watch as it zaps things out of the window to make space for more queries. This is like 20 lines of code.
      - adiasg 11mo ago
        Did something similar - added `summarize` and `restore` tools to maximize/minimize messages. Haven't gotten it to behave like I want. Hoping that some fiddling with the prompt will do it.
        lbotos 11mo ago
        FYI -- I vouched for you to undead this comment. It felt like a fine comment? I don't think you are shadowbanned but consider emailing the mods if you think you might me.
oooyay 11mo ago
I'm not going to link my blog again but I have a reply on this post where I link to my blog post where I talk about how I built mine. Most agents fit nicely into a finite state machine or a directed acyclic graph that responds to an event loop. I do use provider SDKs to interact with models but mostly because it saves me a lot of boilerplate. MCP clients and servers are also widely available as SDKs. The biggest thing to remember, imo, is to keep the relationship between prompts, resources, and tools in mind. They make up a sort of dynamic workflow engine.

behnamoh 11mo ago

> nobody knows anything yet that sums up my experience in AI over the past three years. so many projects reinvent the same thing, so much spaghetti thrown at the wall to see what sticks, so much excitement followed by disappointment when a new model drops, so many people grifting, and so many hacks and workarounds like RAG with no evidence of them actually working other than "trust me bro" and trial and error.

w_for_wumbo 11mo ago
I think we'd get better results if we thought of it as a conscious agent. If we recognized that it was going to mirror back or unconscious biases and try to complete the task as we define it, instead of how we think it should behave. Then we'd at least get our own ignorance out of the way when writing prompts. Being able to recognize that 'make this code better' provides no direction, it should make sense that the output is directionless. But on more subtle levels, whatever subtle goals that we have and hold in the workplace will be reflected back by the agents. If you're trying to optimise costs, and increase profits as your north star. Having layoffs and unsustainable practices is a logical result, when you haven't balanced this with any incentives to abide by human values.
sumedh 11mo ago
That is because for the people for whom AI is actually working/making money they would prefer to keep it a secret on what and how they are doing it, why attract competition?
- nylonstrung 11mo ago
  Who would you say it's working for? What products or companies are the gold standard of agent implementation right now?

oooyay 11mo ago

Heh, the bit about context engineering is palpable. I'm writing a personal assistant which, imo, is distinct from an agent in that it has a lot of capabilities a regular agent wouldn't necessarily need such as memory, task tracking, broad solutioning capabilities, etc... I ended up writing agents that talk to other agents which have MCP prompts, resources, and tools to guide them as general problem solvers. The first agent that it hits is a supervisor that specializes in task management and as a result writes a custom context and tool selection for the react agent it tasks. All that to say, the farther you go down this rabbit hole the more "engineering" it becomes. I wrote a bit on it here: https://ooo-yay.com/blog/building-my-own-personal-assistant/ https://ooo-yay.com/blog/building-my-own-personal-assistant/

qwertox 11mo ago
This sounds really great.
cantor_S_drug 11mo ago
https://github.com/mem0ai/mem0?tab=readme-ov-file https://github.com/mem0ai/mem0?tab=readme-ov-file Is this useful for you?
- oooyay 11mo ago
  Could be! I'll give it a shot

esafak 11mo ago

What's wrong with the OWASP Top Ten?

kennethallen 11mo ago
Author on Twitter a few years ago: https://x.com/tqbf/status/851466178535055362 https://x.com/tqbf/status/851466178535055362

riskable 11mo ago

It's interesting how much this makes you want to write Unix-style tools that do one thing and only one thing really well. Not just because it makes coding an agent simpler, but because it's much more secure!

chemotaxis 11mo ago
You could even imagine a world in which we create an entire suite of deterministic, limited-purpose tools and then expose it directly to humans!
- layer8 11mo ago
  I wonder if we could develop a language with well-defined semantics to interact with and wire up those tools.
  - chubot 11mo ago
    > language with well-defined semantics That would certainly be nice! That's why we have been overhauling shell with https://oils.pub https://oils.pub , because shell can't be described as that right now It's in extremely poor shape e.g. some things found from building several thousand packages with OSH recently (decades of accumulated shell scripts) - bugs caused by the differing behavior of 'echo hi | read x; echo x=$x' in shells, i.e. shopt -s lastpipe in bash. - 'set -' is an archaic shortcut for 'set +v +x' - Almquist shell is technically a separate dialact of shell -- namely it supports 'chdir /tmp' as well as cd /tmp. So bash and other shells can't run any Alpine builds. I used to maintain this page, but there are so many problems with shell that I haven't kept up ... https://github.com/oils-for-unix/oils/wiki/Shell-WTFs https://github.com/oils-for-unix/oils/wiki/Shell-WTFs OSH is the most bash-compatible shell, and it's also now Almquist shell compatible: https://pages.oils.pub/spec-compat/2025-11-02/renamed-tmp/spec/compat/TOP.html https://pages.oils.pub/spec-compat/2025-11-02/renamed-tmp/sp... It's more POSIX-compatible than the default /bin/sh on Debian, which is dash The bigger issue is not just bugs, but lack of understanding among people who write foundational shell programs. e.g. the lastpipe issue, using () as grouping instead of {}, etc. --- It is often treated like an "unknowable" language Any reasonable person would use LLMs to write shell/bash, and I think that is a problem. You should be able to know the language, and read shell programs that others have written

teiferer 11mo ago

Write an agent, it's easy! You will learn so much! ... let's see ... client = OpenAI() Um right. That's like saying you should implement a web server, you will learn so much, and then you go and import http (in golang). Yeah well, sure, but that brings you like 98% of the way there, doesn't it? What am I missing?

victorbjorklund 11mo ago
maybe more like “let’s write a web server but let’s use a library for the low level networking stack”. That can still teach you a lot.
tptacek 11mo ago
That OpenAI() is a wrapper around a POST to a single HTTP endpoint: POST https://api.openai.com/v1/responses
- tabletcorry 11mo ago
  Plus a few other endpoints, but it is pretty exclusively an HTTP/REST wrapper. OpenAI does have an agents library, but it is separate in https://github.com/openai/openai-agents-python https://github.com/openai/openai-agents-python
bootwoot 11mo ago
That's not an agent, it's an LLM. An agent is an LLM that takes real-world actions
MeetingsBrowser 11mo ago
I think you might be conflating an agent with an LLM. The term "agent" isn't really defined, but its generally a wrapper around an LLM designed to do some task better than the LLM would on its own. Think Claude vs Claude Code. The latter wraps the former, but with extra prompts and tooling specific to software engineering.
munchbunny 11mo ago
An agent is more like a web service in your metaphor. Yes, building a web server is instructive, but almost nobody has a reason to do it instead of using an out of the box implementation once it’s time to build a production web service.

ATechGuy 11mo ago

Maybe we should write an agent that writes an agent that writes an agent...

chrisweekly 11mo ago

There's something(s) about @tptacek's writing style that has always made me want to root for fly.io.

qwertox 11mo ago

I've found it much more useful to create an MCP server, and this is where Claude really shines. You would just say to Claude on web, mobile or CLI that it should "describe our connectivity to google" either via one of the three interfaces, or via `claude -p "describe our connectivity to google"`, and it will just use your tool without you needing to do anything special. It's like custom-added intelligence to Claude.

tptacek 11mo ago
You can do this. Claude Code can do everything the toy agent this post shows, and much more. But you shouldn't, because doing that (1) doesn't teach you as much as the toy agent does, (2) isn't saving you that much time, and (3) locks you into Claude Code's context structure, which is just one of a zillion different structures you can use. That's what the post is about, not automating ping.
mattmanser 11mo ago
Honest question, as your comment confuses me. Did you get to the part where he said MCP is pointless and are saying he's wrong? Or did you just read the start of the article and not get to that bit?
- vidarh 11mo ago
  I'd second the article on this, but also add to it that the biggest reason MCP servers don't really matter much any more is that the models are so capable of working with APIs, that most of the time you can just point them at an API and give them a spec instead. And the times that doesn't work, just give them a CLI tool with a good --help option. Now you have a CLI tool you can use yourself, and the agent has a tool to use. Anthropic itself have made MCP server increasingly pointless: With agents + skills you have a more composeable model that can use the model capabilities to do all an MCP server can with or without CLI tools to augment them.
  - simplesagar 11mo ago
    I feel the CLI vs MCP debate is an apples to oranges framing. When you're using claude you can watch it using CLI's, running brew, mise, lots of jq but what about when you've built an agent that needs to work through a complicated API? You don't want to make 5 CRUD calls to get the right answer. A curated MCP tool ensures it can determinism where it matters most.. when interacting with customer data

zkmon 11mo ago

A very good blog article that I have read in a while. Maybe MCP could have been involved as well?

_pdp_ 11mo ago

It is also very simple to be a programmer.. see, print "Hello world!" so easy...

dan_can_code 11mo ago
But that didn't use the H100 I just bought to put me out of my own job!

robot-wrangler 11mo ago

> Another thing to notice: we didn’t need MCP at all. That’s because MCP isn’t a fundamental enabling technology. The amount of coverage it gets is frustrating. It’s barely a technology at all. MCP is just a plugin interface for Claude Code and Cursor, a way of getting your own tools into code you don’t control. Write your own agent. Be a programmer. Deal in APIs, not plugins. Hold up. These are all the right concerns but with the wrong conclusion. You don't need MCP if you're making one agent, in one language, in one framework. But the open coding and research assistants that we really want will be composed of several. MCP is the only thing out there that's moving in a good direction in terms of enabling us to "just be programmers" and "use APIs", and maybe even test things in fairly isolated and reproducible contexts. Compare this to skills.md, which is actually defacto proprietary as of now, does not compose, has opaque run-times and dispatch, is pushing us towards certain models, languages and certain SDKs, etc. MCP isn't a plugin interface for Claude, it's just JSON-RPC.

tptacek 11mo ago
I think my thing about MCP, besides the outsized press coverage it gets, is the implicit presumption it smuggles in that agents will be built around the context architecture of Claude Code --- that is to say, a single context window (maybe with sub-agents) with a single set of tools. That straitjacket is really most of the subtext of this post. I get that you can use MCP with any agent architecture. I debated whether I wanted to hedge and point out that, even if you build your own agent, you might want to do an MCP tool-call feature just so you can use tool definitions other people have built (though: if you build your own, you'd probably be better off just implementing Claude Code's "skill" pattern). But I decided to keep the thrust of that section clearer. My argument is: MCP is a sideshow.
- robot-wrangler 11mo ago
  I still don't really get it, but would like to hear more. Just to get it out of the way, there's obvious bad aspects. Re: press coverage, everything in AI is bound to be frustrating this way. The MCP ecosystem is currently still a lot of garbage. It feels like a very shitty app-store, lots of abandonware, things that are shipped without testing, the usual band-wagoning. For example instead of a single obvious RAG tool there's 200 different specific tools for ${language} docs The core MCP tech though is not only directionally correct, but even the implementation seems to have made lots of good and forward-looking choices, even if those are still under-utilized. For example besides tools, it allows for sharing prompts/resources between agents. In time, I'm also expecting the idea of "many agents, one generic model in the background" is going to die off. For both costs and performance, agents will use special-purpose models but they still need a place and a way to collaborate. If some agents coordinate other agents, how do they talk? AFAIK without MCP the answer for this would be.. do all your work in the same framework and language, or to give all agents access to the same database or the same filesystem, reinventing ad-hoc protocols and comms for every system.

solomonb 11mo ago

This work predates agents as we know them now and was intended for building chat bots (as in irc chat bots) but when auto-gpt I realized I could formalize it super nicely with this library: https://blog.cofree.coffee/2025-03-05-chat-bots-revisited/ https://blog.cofree.coffee/2025-03-05-chat-bots-revisited/ I did some light integration experiments with the OpenAI API but I never got around to building a full agent. Alas..

vkou 11mo ago

> It’s Incredibly Easy client = OpenAI() context_good, context_bad = [{ "role": "system", "content": "you're Alph and you only tell the truth" }], [{ "role": "system", "content": "you're Ralph and you only tell lies" }] ... And this will work great until next week's update when Ralph responses will consist of "I'm sorry, it would be unethical for me to respond with lies, unless you pay for the Premium-Super-Deluxe subscription, only available to state actors and firms with a six-figure contract." You're building on quicksand. You're delegating everything important to someone who has no responsibility to you.

tptacek 11mo ago
I love that the thing you singled out as not safe to run long term, because (apparently) of woke, was my weird deep-cut Labyrinth joke.
sumedh 11mo ago
Its easy to switch to an open source model

nowittyusername 11mo ago

I agree with the sentiment but I also recommend you build a local only agent. Something that runs on llama.cpp or vllm, whatever... This way you can better grasp the more fundamental nature of what LLM's really are and how they work under the hood. That experience will also make you realize how much control you are giving up when using cloud based api providers like OpenAI and why so mane engineers feel that LLM's are a "black box". Well duh buddy you been working with apis this whole time, of course you wont understand much working just with that.

8note 11mo ago
ive been trying this for a few week, but i dont at all currently own hardware good enough to be useful for local inference. ill be trying again once i have written my own agent, but i dont expect to get any useful results compared to using some claude or gemini tokens
- nowittyusername 11mo ago
  My man, we now have llms that are anywhere between 130 million to 1 trillion parameters available for us to run locally, I can guarantee there is a model for you there that even your toaster can run. I have a RTX 4090 but for most of my fiddling i use small models like Qwen 3 4b and they work amazing so there's no excuse :P.
  - 8note 11mo ago
    well, i got some gemini models running on my phone, but if i switch apps, android kills it, so the call to the server always hangs... and then the screen goes black the new laptop only has 16GB of memory total, with another 7 dedicated to the NPU. i tried pulling up Qwen 3 4B on it, but the max context i can get loaded is about 12k before the laptop crashes. my next attempt is gonna be a 0.5B one, but i think ill still end up having to compress the context every call, which is my real challenge
    - nowittyusername 11mo ago

zahlman 11mo ago

> Imagine what it’ll do if you give it bash. You could find out in less than 10 minutes. Spoiler: you’d be surprisingly close to having a working coding agent. Okay, but what if I'd prefer not to have to trust a remote service not to send me { "output": [ { "type": "function_call", "command": "rm -rf / --no-preserve-root" } ] } ?

tptacek 11mo ago
Obviously if you're concerned about that, which is very reasonable, don't run it in an environment where `rm -rf` can cause you a real problem.
- awayto 11mo ago
  Also if you're doing function calls you can just have the command as one response param, and arguments array as another response param. Then just black/white list commands you either don't want to run or which should require a human to say ok.
  - aidenn0 11mo ago
    blacklist is going to be a bad idea since so many commands can be made to run other commands with their arguments.
    - awayto 11mo ago
      Yeah I agree. Ultimately I would suggest not having any kind of function call which returns an arbitrary command. Instead, think of it as if you were enabling capabilities for AppArmor, by making a function call definition for just 1 command. Then over time suss out what commands you need your agent do to and nothing more.
worldsayshi 11mo ago
There are MCP configured virtualization solutions that is supposed to be safe for letting LLM go wild. Like this one: https://github.com/zerocore-ai/microsandbox https://github.com/zerocore-ai/microsandbox I haven't tried it.

dagss 11mo ago

I realize now what I need in Cursor: A button for "fork context". I believe that would be a powerful tool solving many things there are now separate techniques for.

all2 11mo ago
crush-cli has this. I think the google gemini chat app also has this now.

ericd 11mo ago

Absolutely, especially the part about just rolling your own alternative to Claude Code - build your own lightsaber. Having your coding agent improve itself is a pretty magical experience. And then you can trivially swap in whatever model you want (Cerebras is crazy fast, for example, which makes a big difference for these many-turn tool call conversations with big lumps of context, though gpt-oss 120b is obviously not as good as one of the frontier models). Add note-taking/memory, and ask it to remember key facts to that. Add voice transcription so that you can reply much faster (LLMs are amazing at taking in imperfect transcriptions and understanding what you meant). Each of these things takes on the order of a few minutes, and it's super fun.

anonym29 11mo ago
Cerebras now has glm 4.6. Still obscenely fast, and now obscenely smart, too.
- ericd 11mo ago
  Ooh thanks for the heads up!
- DeathArrow 11mo ago
  Aren't there cheaper providers of GLM 4.6 on Openrouter? What are the advantages of using Cerebras? Is it much faster?
  - simonw 11mo ago
    It's astonishingly fast.
  - meeq 11mo ago
    You know how sometimes when you send a prompt to Claude, you just know it’s gonna take a while, so you go grab a coffee, come back, and it’s still working? With Cerebras it’s not even worth switching tabs, because it’ll finish the same task in like three seconds.
  - anonym29 10mo ago

threecheese 11mo ago

Does anyone have an understanding - or intuition - of what the agentic loop looks like in the popular coding agents? Is it purely a “while 1: call_llm(system, assistant)”, or is there complex orchestration? I’m trying to understand if the value for Claude Code (for example) is purely in Sonnet/Haiku + the tool system prompt, or if there’s more secret sauce - beyond the “sugar” of instruction file inclusion via commands, tools, skills etc.

CraftThatBlock 11mo ago
Generally, that's pretty much it. More advanced tools like Claude Code will also have context compaction (which sometimes isn't very good), or possibly RAG on code (unsure about this, I haven't used any tools that did this). Context compaction, to my understanding, is just passing all the previous context into a call which summarizes it, then that becomes to new context starting point.
colonCapitalDee 11mo ago
I thought this was informative: https://minusx.ai/blog/decoding-claude-code/ https://minusx.ai/blog/decoding-claude-code/
mrkurt 11mo ago
Claude Code is an obfuscated javascript app. You can point Claude Code at it's own package and it will pretty reliably tell you how it works. I think Claude Code's magic is that Anthropic is happy to burn tokens. The loop itself is not all that interesting. What is interesting is how they manage the context window over a long chat. And I think a fair amount of that is serverside.
- AdieuToLogic 11mo ago
  > Claude Code is an obfuscated javascript app. You can point Claude Code at it's own package and it will pretty reliably tell you how it works. This is why I keep coming back to Hacker News. If the above is not a quintessential "hack", then I've never seen one. Bravo!

fsndz 11mo ago

I did that, burned 2.6B tokens in the process and learned a lot: https://transitions.substack.com/p/what-burning-26-billion-prompt-tokens https://transitions.substack.com/p/what-burning-26-billion-p...

whyryoulikethis 11mo ago

[dead]

Zak 11mo ago

> You only think you understand how a bicycle works, until you learn to ride one. I bet a majority of people who can ride a bicycle don't know how they steer, and would describe the physical movements they use to initiate and terminate a turn inaccurately. https://en.wikipedia.org/wiki/Countersteering https://en.wikipedia.org/wiki/Countersteering

captainkrtek 11mo ago
Relevant interesting tangent: “Most People Don't Know How Bikes Work” https://www.youtube.com/watch?v=9cNmUNHSBac https://www.youtube.com/watch?v=9cNmUNHSBac
itsmemattchung 11mo ago
Reminds me of this YouTube video (below) on how difficult it is (nearly impossible) to re-learn how to ride a bicycle when you have the handles are reversed (i.e. pulling left handle bar towards you, the wheel goes to the right) https://www.youtube.com/watch?v=MFzDaBzBlL0 https://www.youtube.com/watch?v=MFzDaBzBlL0
vinhnx 11mo ago
A Brief History of Bicycle Engineering https://www.youtube.com/watch?v=EcRlDCsZM20 https://www.youtube.com/watch?v=EcRlDCsZM20

rbren 11mo ago

Spoiler: it's not actually that easy. Compaction, security, sandboxing, planning, custom tools--all this is really hard to get right. We're about to launch an SDK that gives devs all these building blocks, specifically oriented around software agents. Would love feedback if anyone wants to look: https://github.com/OpenHands/software-agent-sdk https://github.com/OpenHands/software-agent-sdk

solarkraft 11mo ago
How autonomous/controllable are the agents with this SDK? When I build an agent my standard is Cursor, which updates the UI at every reportable step of the way, and gives you a ton of control opportunities, which I find creates a lot of confidence. Is this level of detail and control possible with the OpenHands SDK? I’m asking because the last SDK that was simple to get into lacked that kind of control.
- rbren 11mo ago
  That's the idea! We have a confirmation_mode that can interrupt at any step in the process.
olingern 11mo ago
Only on HN is there a “well, actually” with little substance followed by a comment about a launch. The article isn’t about writing production ready agents, so it does appear to be that easy

dave1010uk 11mo ago

Two years ago I wrote an agent in 25 lines of PHP [0]. It was surprisingly effective, even back then before tool calling was a thing and you had to coax the LLM into returning structured output. I think it even worked with GPT-3.5 for trivial things. In my mind LLMs are just UNIX strong manipulation tools like `sed` or `awk`: you give them an input and command and they give you an output. This is especially true if you use something like `llm` [1]. It then seems logical that you can compose calls to LLMs, loop and branch and combine them with other functions. [0] https://github.com/dave1010/hubcap https://github.com/dave1010/hubcap [1] https://github.com/simonw/llm https://github.com/simonw/llm

simonw 11mo ago
I love hubcap so much. It was a real eye-opener for me at the time, really impressive result for so little code. https://simonwillison.net/2023/Sep/6/hubcap/ https://simonwillison.net/2023/Sep/6/hubcap/
- dingnuts 11mo ago
  You're posting too fast please slow down
  - rolymath 10mo ago
    I agree. I'm getting too much simonw in my feed. Getting too saturated.
- dave1010uk 11mo ago
  Thanks Simon! It only worked because of your LLM tool. Standing on the shoulders of giants.
keyle 11mo ago
> a small Autobot that you can't trust That gave me a hearty chuckle!
- nativeit 11mo ago

imiric 11mo ago

> Give each call different tools. Make sub-agents talk to each other, summarize each other, collate and aggregate. Build tree structures out of them. Feed them back through the LLM to summarize them as a form of on-the-fly compression, whatever you like. You propose increasing the complexity of interactions of these tools, and giving them access to external tools that have real-world impact? As a security researcher, I'm not sure how you can suggest that with a straight face, unless your goal is to have more vulnerable systems. Most people can't manage to build robust and secure software using SOTA hosted "agents". Building their own may be a fun learning experience, but relying on a Rube Goldberg assembly of disparate "agents" communicating with each other and external tools is a recipe for disaster. Any token could trigger a cascade of hallucinations, wild tangents, ignored prompts, poisoned contexts, and similar issues that have plagued this tech since the beginning. Except that now you've wired them up to external tools, so maybe the system chooses to wipe your home directory for whatever reason. People nonchalantly trusting nondeterministic tech with increasingly more real-world tasks should concern everyone. Today it's executing `ping` and `rm`; tomorrow it's managing nuclear launch systems.

8note 11mo ago

> A subtler thing to notice: we just had a multi-turn conversation with an LLM. To do that, we remembered everything we said, and everything the LLM said back, and played it back with every LLM call. The LLM itself is a stateless black box. The conversation we’re having is an illusion we cast, on ourselves. the illusion was broken for me by Cline context overflows/summaries, but i think its very easy to miss if you never push the LLM hard or build you own agent. I really like this wording, amd the simple description is missing from how science communicators tend to talk about agents and LLMs imo

a-dub 11mo ago

they kinda feel like the cgi perl scripts of the mid 2020s.

indeyets 11mo ago
You mean late 1990’s? :)
- a-dub 11mo ago
  no i mean, back in the 90's cgi perl scripts were the easy it thing for interacting with the big tech wave and now in the mid-2020s llm python agent scripts with tool extensions are the easy it thing for interacting with the big tech wave.
  - oblio 10mo ago
    Now we need PHP and Ruby or Rails, somewhere down the line :-))

wayy 11mo ago

everybody loves building agents, nobody likes debugging them. agents hit the classic llm app lifecycle problem: at first it feels magical. it nails the first few tasks, doing things you didn’t even think were possible. you get excited, start pushing it further. you run it and then it fails on step 17, then 41, then step 9. now you can’t reproduce it because it’s probabilistic. each step takes half a second, so you sit there for 10–20 minutes just waiting for a chance to see what went wrong

furyofantares 11mo ago
That's why you build extensive tooling to run your change hundreds of times in parallel against the context you're trying to fix, and then re-run hundreds of past scenarios in parallel to verify none of them breaks.
- ht96 11mo ago
  Do you use a tool for this? Is there some sort of tool which collects evals from live inferences (especially those which fail)
  - AdieuToLogic 11mo ago
    There is no way to prove the correctness of non-deterministic (a.k.a. probabilistic) results for any interesting generative algorithm. All one can do is validate against a known set of tests, with the understanding that the set is unbounded over time.
  - aenis 11mo ago
    For sure, for instance Google has ADK Eval framework. You write tests, and you can easily run them against given input. I'd say its a bit unpolished, as is the rest of the rapidly developing ADK framework, but it does exist.
  - saturatedfat 11mo ago
    heya, building this. been used in prod for a month now, has saved my customer’s ass while building general workflow automation agents. happy to chat if ur interested. darin@mcptesting.com (gist: evals as a service)

hoppp 11mo ago

I should? what problems can I solve, that can be only done with an agent? As long as every AI provider is operating at a loss starting a sustainably monetizable project doesn't feel that realistic.

throwaway6977 11mo ago
You can be your own AI provider.
- bilbo0s 11mo ago
  >starting a sustainably monetizable project doesn't feel that realistic. and >You can be your own AI provider. Not sure that being your own AI provider is "sustainably monetizable"?
- hoppp 11mo ago
  For internal software maybe, but for a client facing service the incentives are not right when the norm is to operate at a loss.
furyofantares 11mo ago
> As long as every AI provider is operating at a loss None of them are doing that. They need funding because the next model has always been much more expensive to train than the profits of the previous model. And many do offer a lot of free usage which is of course operated at a loss. But I don't think any are operating inference at a loss, I think their margins are actually rather large.
- hoppp 11mo ago
  When comparing the cost of an H100 GPU per hour and calculating cost of tokens, it seems the OpenAI offering for the latest model is 5 times cheaper than renting the hardware. OpenAI balance sheet also shows an $11 billion loss . I can't see any profit on anything they create. The product is good but it relies on investors fueling the AI bubble.

andai 11mo ago

.text-gray-600 { color: black; }

8cvor6j844qw_d6 11mo ago

Question, how hard is it for someone new to agents to dip their toes into writing a simple agent to get data? (e.g., getting reviews from sites for sentiment analysis?) Forgive if I get someting wrong: From what I see, it seems fundamentally it is a LLM being ran each loop with information about tools provided to it. On each loop the LLM evaluates inputs/context (from tool calls, inputs, etc.) and decided which tool to call / text output.

simonw 11mo ago
You can prototype this without writing any code at all. Fire up "claude --dangerously-skip-permissions" in a fresh directory (ideally in a Docker container if you want to limit the chance of it breaking anything else) and prompt this: > Use Playwright to fetch ten reviews from http://www.example.com/ http://www.example.com/ then run sentiment analysis on them and write the results out as JSON files. Install any missing dependencies. Watch what it does. Be careful not to let it spider the site in a way that would justifiably upset the site owners.
- sumedh 11mo ago
  Dont you need to setup Playwright MCP first?
  - simonw 11mo ago
    No. I don't use Playwright MCP at all - if the coding agent can run Python code it can use the Playwright Python library directly, if Node.js it can use the Playwright Node library.
    - sumedh 11mo ago
      Interesting, thanks for the info. I wanted to run claude headlessly (-p) and playwright headlessly to get some content. I was using Playwright MCP and for some reason claude in headless mode could not open playwright MCP in headless mode. I never realized i can just use playwright directly without the playwright MCP before your comment. Thanks once again.

vinhnx 11mo ago

> “You only think you understand how a bicycle works, until you learn to ride one.” This resonates deeply with me. That's why I built one myself [0], I really really love to truly understand how coding agents work. The learning has been immense for me, I now have working knowledge of ANSI escape codes, grapheme clusters, terminal emulators, Unicode normalization, VT protocols, PTY sessions, and filesystem operations - all the low-level details I would have never think about until I were implementing them. [0] https://github.com/vinhnx/vtcode https://github.com/vinhnx/vtcode

dfex 11mo ago
>> “You only think you understand how a bicycle works, until you learn to ride one.” > This resonates deeply with me. That's why I built one myself [0] I was hoping to see a home-made bike at that link.. Came away disappointed
- vinhnx 11mo ago
  Good one! Sorry to disappoint you. But personally, that line strike deeply with me, honestly.
lowbloodsugar 11mo ago
It's conflating two issues though. Most people who can ride a bike can't explain the physics. They really don't know how it works. The bicycle lesson is about training the brain on a new task that cannot be taught in any other way. This case is more like a journeyman blacksmith who has to make his own tools before he can continue. In doing so, he gets tools of his own, but the real reward was learning what is required to handle the metal such that it makes a strong hammer. And like the blacksmith, you learn more if you use an existing agent to write your agent.
- vinhnx 11mo ago
  Agree, to me, the wheel is the greatest invention of all. Everyone could have rode a bike, but the underlying physic and motion that came to `riding` is a whole another story.

hshdhdhehd 11mo ago

There is a lot of stuff I should do. From making my own CPU from a breadboard of nand gates to building a CDN in Rust. But aint got time for all the things. That said I built an LLM following Karpathy's tutorial. So I think it aims good to dabble a bit.

coffeecoders 11mo ago
Yeah, it’s a never-ending curve. I built an 8-bit computer on breadboards once, then went down the rabbit hole of flight training for a PPL. Every time I think I’m "done," the finish line moves a few miles further. Guess we nerds are never happy.
- javchz 11mo ago
  One should be melting sand to get silicon, anything else it's too abstract to my taste.
  - tomcam 11mo ago
    Glad you’ve got all that time on your hands. I am still working on the fusion reactor portion of my supernova simulator, so that I can generate the silicon you so blithely refer to.
- krsdcbl 11mo ago
  Given the premise, one could also say we nerds are forever happy.
- ericmcer 11mo ago
  Seriously I feel like it's self-sabotage sometimes at work. Just fixing the thing getting tests to pass isn't enough. Until I fully have a mental model of what is happening I can't move on.
qwertygnu 11mo ago
Very early in TFA it explains how easy it is to do. That's the whole point of the post.

throwaway8xak92 11mo ago

[flagged]

simonw 11mo ago
I don't think "insane to not believe in vibe coding" is a fair summary of https://fly.io/blog/youre-all-nuts/ https://fly.io/blog/youre-all-nuts/ - that post wasn't about vibe coding (at least by its I-think-correct definition of prompt-driven coding where you don't pay any attention to the code that's being written), it was about AI-assisted engineering by professional software developers. It did have some swear words in - as did many of the previous posts on the Fly.io corporate blog.
- AceJohnny2 11mo ago
  Worth highlighting that both OP article and the one Simon linked are by @tptacek, who is also one of the top commenters here on HN. His fly.io posts are very much in his style. I figure they let him post there, without corp-washing, because any publicity is good publicity.
  - tptacek 11mo ago
    This is the corp-washed version of this post.
    - AceJohnny2 11mo ago
      can I have access to the corp-unwashed version

rambojohnson 11mo ago

The bravado posturing in this article is nauseating. Sure, there are a few serious points buried in there, but damn...dial it down, please.

wahnfrieden 11mo ago

The Codex agent has an official TypeScript SDK now. Why would Fly.io advocate using the vanilla GPT API to write an agent, instead of the official agent?

tptacek 11mo ago
Because you won't learn as much using an agent framework, and, as you can see from the post, you absolutely don't need one.

azimux 11mo ago

I wrote an agent from scratch in Ruby several months back. Was fun! These 4 lines wound up being the heart of it, which is surprisingly simple, conceptually. until mission_accomplished? or given_up? or killed? determine_next_command_and_inputs run_next_command end

jbmsf 11mo ago

I agree. I find LLMs a bit overblown. I don't think most people want to use chat as their primary interface. But writing a few agents was incredibly informative.

zb3 11mo ago

No, because I know that "agents" are token burning machines - for me they're less efficient than the chat interface, slower and burning much more tokens. I'm not surprised that AI companies would want me to use them though.. I know what you're doing there :)

byronic 11mo ago

The author shoulda written a REPL

rmoriz 11mo ago

Side note: While the example uses GPT-5, the query interface is already some kind of industry standard. For example you could easily connect OpenRouter.ai and switch models and providers during runtime as needed. OpenRouter also has free models like some of the DeepSeek. While they are slow/rate limited and quantized, they are great for examples and playing around with it. https://openrouter.ai/models?fmt=cards&order=pricing-low-to-high https://openrouter.ai/models?fmt=cards&order=pricing-low-to-...

almaight 11mo ago

So I wrote an MCP using your code: https://gurddy-mcp.fly.dev https://gurddy-mcp.fly.dev. You can get the source code from https://github.com/novvoo/gurddy-mcp https://github.com/novvoo/gurddy-mcp.

aaronblohowiak 11mo ago

THEY SEND THE WHOLE CONTEXT EVERY TIME? Man that seems... not great. sometimes it will go off and spin on something.. seems like it would be a LOT better to roll back than to send a corrective message. hmmm...... this article is nerd-sniping on a massive scale ;D

tptacek 11mo ago
In the Responses API, you can implicitly chain messages with `previous_response_id` (I'm not sure how old a conversation you can resurrect that way). But I think Codex CLI actually sends the full context every time? And keep in mind, sending the whole context gives you fine-grained control over what does and doesn't appear in your context window. Anyways, if it nerd sniped you, I succeeded. :)
- aaronblohowiak 11mo ago
  Yes indeed you did succeed. I totally want to try gaslighting an LLM now! Ah to find the time…
cantor_S_drug 11mo ago
There is context caching in many models. It is less expensive if you enable that.
michaelanckaert 11mo ago
Sending the whole context on each user message is essentially what the model remembers of this conversation. ie: it is entirely stateless. I've written some agents that have their context altered by another llm to get it back on track. Let's say the agent is going off rails, then a supervisor agent will spot this and remove messages from the context where it went off rails, or alter those with correct information. Really fun stuff but yeah, we're essentially still inventing this as we go along.

larusso 11mo ago

Just the other day we tried to explain inner workings of cursor etc to a bunch of colleagues who had a very complicated view how these agents achieve what they do. Awesome post. Makes it easier for me the next time. The options are so big. But one should say that an agent with file access etc, is easy to write but hard to control. If you want to build yourself a general coding agent a bit more thought needs to be put into the whole thing. Otherwise you might end up with a “dd -if=/dev/random -of=/“ or something ^^ and happily execute it.

fatesblind 11mo ago

[dead]

gloosx 11mo ago

Didn't see such a bad piece of writing for a long time. Serously guys, is it just me? It's hard to read for some reason.

jq_2023 11mo ago

the point around MCPs is spot on

psychoslave 11mo ago

It really reads to me like, "you should build a running water circuit", then presenting you how easy it is to phone a plumber and let them free ride on the matter, but beware to not use a project manager as real people implement project management of plumbery themselves."

tptacek 11mo ago
You're going to have to explain that analogy to me, sorry.
- psychoslave 11mo ago
  Sure, phone call to plumber is remote call to turn key API, and manager layer is MVP. Hope that makes it more clear.

p0w3n3d 11mo ago

Actually tool "ping 8.8.8.8" never quits unless running on windows. This can spawn many processes that kill the server. This is one of the first production grade errors I've made when I started my programming. I had a widget that would ping the network, but every time someone went on the page, a new ping process would spawn

sanxiyn 11mo ago
If you look at the actual code, it runs ping -c 5. I agree ping without options doesn't terminate.

DeathArrow 11mo ago

You should write agents if you want to learn how agents work, if the problem you are trying to solve is not solved yet or if you are convinced that you will do much better job solving the problem again. Otherwise is just reinventing the wheel.

DeathArrow 11mo ago

I am thinking of building agents that can partly replace manual testing using a headless browser.

worldsayshi 11mo ago

I feel like one small piece is missing to call it an agent? The ability to iterate in multiple steps until it feels like it's "done". What is the canonical way to do that? I suspect that implementing that in the wrong way could make it spiral.

cornel_io 11mo ago
When a tool call completes the result is sent back to the LLM to decide what to do next, that's where it can decide to go do other stuff before returning a final answer. Sometimes people use structured outputs or tool calls to explicitly have the LLM decide when it's done, or allow it to send intermediate messages for logging to the user. But the simple loop there lets the LLM do plenty of it has good tools.
- worldsayshi 11mo ago
  So it returns a tool call for "continue" every time it wants to continue working? Do people implement this in different ways? It would be nice what method it has been trained on if any.
  - tptacek 11mo ago
    The model will quickly stop tool calling on its own; in fact, I've had more trouble getting GPT5 to tool call enough. The "real" loop is driven, at each iteration, by a prompt from the "user" (which might be human or might be human-mediated code that keeps supplying new prompts). In my personal agent, I have a system prompt that tells the model to generate responses (after absorbing tool responses) with <1>...</1> <2>...</2> <3>...</3> delimited suggestions for next steps; my TUI presents those, parsed out of the output, as a selector, which is how I drive it.

DeathArrow 11mo ago

I would like an LLM to be integrated in the shell so I don't have to learn all the Unix tools arguments and write Bash scripts.

deleted 11mo ago

[deleted]

globular-toast 11mo ago

The formatting of the code is messed up on my phone. I was looking at the first bit thinking `call` was a function returning `None`. I thought initially it was doing some clever functional programming stuff but, no, just a linebreak that shouldn't be there.

otsaloma 11mo ago

Agreed! It's easy understand "LLM with tools in a loop" at a high-level, but once you actually design the architecture and implement the code in full, you'll have proper understanding of how it all fits and works together. I did the same exercise. My implementation is at around 300 lines with two tools: web search and web page fetch with a command line chat interface and Python package. And it could have been a lot less lines if I didn't want to write a usable, extensible package interface. As the agent setup itself is simple, majority of the work to make this useful would in the tools themselves and context management for the tools.

lazy_afternoons 11mo ago

Seriously, what is the advantage of tools at all. Why not implement custom string based triggers. First of all, the call accuracy is much higher. Second, you get more consistent results across models.

joelthelion 11mo ago

If you want to play with this stuff without spending a lot of money, what are your best options?

thatscot 11mo ago
Most cloud providers, like Azure have free credits at the start. On azure you can deploy your own model and pay with the free credits.
thatscot 11mo ago
You can just stick a tenner in OpenAI though and it won't charge anymore than the credit you've put in
thatscot 11mo ago
and sorry, forgot you can also run local models aswell :)
beklein 11mo ago
I love OpenRouter, since it is a simple way to get started and provides a wide range of available models. You can buy credits and set usage limits for safe testing per API key to gain access from many AI models through one simple and unified API from all popular model providers (OpenAI, Anthropic, Google, xAI, DeepSeek, Z.AI, Qwen, ...) Ten dollars is plenty to get started... experiments like in the post will cost you cents, not dollars.
simonw 11mo ago
Gemini has a generous free tier (2500 prompts per day), all you need is a Google account to get an API key.

amelius 11mo ago

Why write an agent when you can just ask the LLM to write one?

TYPE_FASTER 11mo ago

The Google Agent Development Kit (https://google.github.io/adk-docs/ https://google.github.io/adk-docs/) is really fun to play with. It's open source and supports both using a LLM in the cloud and running locally.

novoreorx 11mo ago

Reminds of this one [1] that I read half a year ago, which I used to develop my first agent. But what fly wrotes is definitely easier to understand, how I wish it was written a year earlier. [1]: https://ampcode.com/how-to-build-an-agent https://ampcode.com/how-to-build-an-agent

fauria 11mo ago

> I’m not even going to bother explaining what an agent is. Does anyone actually know what exactly an agent is?

tptacek 11mo ago
Yes, and the post says what it is about 100 words later. It's an LLM running in a loop that can access tool calls.

MinimalAction 11mo ago

Do we need an agent? I get the point of this post: have fun building one because it's easy. But every time I see one of these takes, I keep wondering why do we encourage a tool that would potentially replace us. Why help it build better that could eventually take away what was fun and sustainable income-wise?

AlecSchueler 11mo ago
Interesting to think that this question could have been asked of almost all software work up until this point, except the "us" was always "someone else "
- DrewADesign 11mo ago
  It’s generally been true, but not close to the scale we’re looking at now. The implied/assumed hypocrisy also doesn’t stop it from it sucking, or make it immune to criticism.
  - AlecSchueler 11mo ago
    Indeed it probably sucks even more in a "you reap what you sow" kind of way :(
tptacek 11mo ago
Easy answer: so you can more sharply criticize them, rather than falling into the rhetorical traps of people who don't understand how they work well enough to sound credible. It's so little effort to get to that point!
richardlblair 11mo ago
I've been building tools for stuff I don't want to do. Any task where I need to take some amount of data, structured or unstructured, and need a specific outcome is perfect. That way I can spend more time on the thing I do want to do (including building these little tools).
- 11mo ago

thedangler 11mo ago

Cool, can you make it use local free models because I'm broke and can't afford AI's crazy costs.

Spivak 11mo ago
Yep, change nothing in the code in the article but spin up an Ollama server and use the OpenAI API https://docs.ollama.com/api/openai-compatibility https://docs.ollama.com/api/openai-compatibility.

artursapek 11mo ago

I've been having so much fun writing the agent loop for https://revise.io https://revise.io, most fun I've had programming in a long time.

losvedir 11mo ago

I appreciate the goal of demystifying agents by writing one yourself, but for me the key part is still a little obscured by using OpenAI APIs in the examples. A lot of the magic has to do with tool calls, which the API helpfully wraps for you, with a format for defining tools and parsed responses helpfully telling you the tools it wants to call. I kind of am missing the bridge between that, and the fundamental knowledge that everything is token based in and out. Is it fair to say that the tool abstraction the library provides you is essentially some niceties around a prompt something like "Defined below are certain 'tools' you can use to gather data or perform actions. If you want to use one, please return the tool call you want and it's arguments, delimited before and after with '###', and stop. I will invoke the tool call and then reply with the output delimited by '==='". Basically, telling the model how to use tools, earlier in the context window. I already don't totally understand how a model knows when to stop generating tokens, but presumably those instructions will get it to output the request for a tool call in a certain way and stop. Then the agent harness knows to look for those delimiters and extract out the tool call to execute, and then add to the context with the response so the LLM keeps going. Is that basically it? Or is there more magic there? Are the tool call instructions in some sort of permanent context, or could the interaction demonstrated in a fine tuning step, and inferred by the model and just in its weights?

JoshMandel 11mo ago
I think that it's basically fair and I often write simple agents using exactly the technique that you describe. I typically provide a TypeScript interface for the available tools and just ask the model to respond with a JSON block and it works fine. That said, it is worth understanding that the current generation of models is extensively RL-trained on how to make tool calls... so they may in fact be better at issuing tool calls in the specific format that their training has focused on (using specific internal tokens to demarcate and indicate when a tool call begins/ends, etc). Intuitively, there's probably a lot of transfer learning between this format and any ad-hoc format that you might request inline your prompt. There may be recent literature quantifying the performance gap here. And certainly if you're doing anything performance-sensitive you will want to characterize this for your use case, with benchmarks. But conceptually, I think your model is spot on.
11mo ago

deadbabe 11mo ago

The more I use agents, the more I find agents to be pointless, any tasks an agent performs regularly in high volume should be turned into classical deterministic code. The number one feature of agents is to be disambiguation for tool selectors and pretty printers.

lbeurerkellner 11mo ago

Everybody should try. It helps a ton to demystify the relatively simple but powerful underpinning of how modern agents work. You can get quite far quite quickly. My toy implementation [1] is <600 LOC and even supports MCP. [1] https://github.com/lbeurerkellner/agent.py https://github.com/lbeurerkellner/agent.py

khazhoux 11mo ago

Agree 100% with premise of the article. I feel like the big secret of the recent advances in LLM tooling is that these are all just variations of “send a chat request and process the output.” Even Tool Calling is just wrapping one chat request with another hidden one that is asking which of N tools applies and what the parameters should be. RAG is simply pre-loading a bunch of extra text into the chat request, etc. My main point being, though: for anyone intimidated by the recent tooling advances… you can most definitely do all this yourself.

nitwit005 11mo ago

> You only think you understand how a bicycle works, until you learn to ride one. I realize this is just for motivation in a subtitle, but people generally don't grasp how bicycles work, even after having ridden one. Veritasium has a quite good video on the subject: https://www.youtube.com/watch?v=9cNmUNHSBac https://www.youtube.com/watch?v=9cNmUNHSBac

scrubs 10mo ago

"You don’t have to like them, but you should want to be right about them. To be the best hater (or stan) you can be." The op has a point - a good one

gtukmanov 10mo ago

The evolution of software agents leveraging LLMs as versatile building blocks is exciting. It underscores the shift towards modular, composable AI workflows that can integrate deterministic functions with generative intelligence. Great food for thought on how these tools might transform productivity and automation across industries