10 ms·

Understanding is the new bottleneck

445 points by sebg 1mo ago

euthymiclabs 1mo ago
"I read the code." -Mitchell Hashimoto Great code needs great understanding and agents need excellent guidance. Even in my current solo-dev work, I can't imagine making a production commit I haven't read until I understand it. I own the consequences of my code; that's a responsibility AI agents can't take.
- conqrr 1mo ago
  Agreed with Reading! This alone isn't enough though. PreLLM too it wasn't just reading code to review. Someone did the hard work or crafting the code and each unit test would tell you the weird corner cases to deal with and factor that into changing your code. One person owned a part of the codebase and was an expert. Not to mention reading isnt easy when the velocity of code pumping in is 3-5x more. Its exhausting and reading becomes skimming.
  - euthymiclabs 1mo ago
    So true. Writing the code was so helpful for learning what it meant. It takes much more investment to go back to the code LLMs write to figure out what's actually happening and weigh everything.
  - tripledry 1mo ago
    The expert is now outsourced to LLM. If someone asks me about a bug in a system I made N years ago, I usually have a hunch what the problem might be, now it feels I'm lost in my own codebase (even if I really read the code). Similar to why math books have exercises and not only explanations. Guess it's not an issue as long as you have access to the models and someone who likes prompting.
- mindwok 1mo ago
  Conversely I'm a solo-dev and I ship a lot of slop I don't even look at. Granted my work is just basic CRUD apps, and I focus my efforts on validating important consequences (like does this break accounting invariants or something)
- hackmack10 1mo ago
  Yeah, but in the corporate world now, you're being judged by how fast you get shit checked in. So, if you need to understand every line these days... you're going to end up in the bottom 25% of the company pretty quickly. Not saying you're wrong, just saying how fucked things are becoming.
  - euthymiclabs 1mo ago
    Perverse incentives have always been the purview of bad management. Measuring things that don't cause success is easier than figuring out what actually matters. (It's also why I don't practice corporate medicine anymore.)
iainctduncan 1mo ago
I am so dying to read more about the new/current/real bottleneck! Where is the bottleneck? WHERE?? Tell me! No evidence needed, just lay it on, man to man, thought-leader to thought-leader!
- iainctduncan 1mo ago
  Oh wait, there it is, sitting over top of the fat part and under the cork...
- euroderf 1mo ago
  > thought-leader to thought-leader! This is my new chat-up line at networking events.
  - iainctduncan 1mo ago
    :-)
- techpression 1mo ago
  It doesn’t matter, once you found the bottleneck there is a new one. Seems we changed the supposed bottleneck of writing code (as if it ever were, the world was producing far too much code before LLMs were even a thing) with about ten or so new ones, was it a good trade?
  - Kinrany 1mo ago
    You need to actually remove the bottleneck, after _that_ there'll be a new one
- NoDodgeQuestion 1mo ago
  uncovering bottleneck is the new bottleneck!!
fabiensanglard 1mo ago
While the tips are good to handle the volume, I still think this sets code owner on a dangerous path. AI have limitation and hallucinate. Complex code will be explained in hallucinated way. At some point AI will be unable to write more because the arch has become too complex or the volume of code will be to high. The article I would like to read would suggest how to force LLM to architect the code like a solid tower instead of a pile of unstable mud.
- vjvjvjvjghv 1mo ago
  I feel if you still architect the code and guide the LLM, it will do a pretty good job. Maybe one day the LLM will be able to do all the system architecture but that’s probably still quite some time out. I don’t even know if that’s possible considering different business needs and other factors that aren’t technical.
- alikhater30000 1mo ago
  [flagged]
alecbz 1mo ago
We have LLMs try to generate descriptions of PRs for us and they're pretty universally disliked. They're always overly-complex descriptions of the mechanical changes and have no sense of motivation. Also, a huge reason to understand the code yourself is to make sure the LLM isn't wrong, but this doesn't work if an LLM is itself generating the understanding.
- vjvjvjvjghv 1mo ago
  I don’t know. From my experience I get pretty good descriptions of PRs if I ask the right questions and provide some context.
  - jollyllama 1mo ago
    > provide some context That's pretty much what a PR description is.
    - baq 1mo ago
      The difference is an LLM can convert a stream of consciousness into well-formed prose for approximately free; I assume ‘provide some context’ means ‘brain dump’ in the OP
      - deleted 1mo ago
        [deleted]
      - alecbz 1mo ago
        This has unfortunately not been my experience at all. Often LLMs miss or get wrong subtle details when I don't do the pre-work to organize my thoughts well ahead of time (at which point it's unclear how much value they're providing).
    - nsingh2 1mo ago
      Work doesn't start with a PR description though. I'm assuming most people that are using LLMs start with some sort of document (plan, spec, intent, etc) which captures intent. I guess you could also use all the session rollouts saved to disk that were related to that task, and distill them somehow.
jbdamask 1mo ago
pfft...I'm way past understanding
- ch4s3 1mo ago
  Overstanding is the new horizon
  - stronglikedan 1mo ago
    That's already been co-opted by the sovereign citizen movement, so use with caution lest you be judged for it.
    - ch4s3 1mo ago
      Its been used by a lot of fringe types since the 1960s, and was at one time popular among Rastafarians. It's also a little used sailing term. Here it functions merely as satire.
  - layer8 1mo ago
    You have to overstand before you can overwalk.
hk__2 1mo ago
For me the solution has been to throw away the code I don’t understand. I let the agent write the code, and if when I read it it seems unclear or needs a lot of explanation from the agent, I just throw it away and start over, or do it by myself.
- evenhash 1mo ago
  This is where I’m at as well. If it needs to change too much to do what I want, I take that as a sign to either (1) ask for a change that is easier to review, or barring that (2) pivot to refactoring the codebase until it becomes a change that is easy to review. Ironically, even in this era of cheap and instant code, what works best (for me) is still to write as little code as possible.
- xxpor 1mo ago
  This seems like one of the hardest parts for everyone to adjust to (including myself): code is now easy come, easy go. Or in other words, everyone seems to get trapped in the sunk cost fallacy, even if there's not even that much sunk cost any more!
- imtringued 1mo ago
  Something funny happens when you reject code. The agent realises it is garbage. I'm personally of the opinion that the tools for reviewing LLM generated code are awful. 99% of the time I want to do line by line comments and tell it everything it did wrong. Given that information, the next iteration would be much more up to my standards. The same also applies to other people's LLM generated code. Yeah sure they can just pass on the comments to the LLM, but that will just mean more iterations and them losing their job.
wseqyrku 1mo ago
If you try to spec the problem with all the painful details for the machine to understand, you will end up with a rust codebase.
bigstrat2003 1mo ago
Understanding has always been the bottleneck. That's why LLMs aren't actually helpful: they speed up the part which is easy (typing characters into your editor), but are neutral or even harmful on the part which is hard (understanding the problem and how best to solve it).
- stronglikedan 1mo ago
  The part which is easy is still time consuming, so LLMs are helpful. They're just not a silver bullet.
- causal 1mo ago
  It seems like humans have a limited "understanding budget" but LLMs force us to spend that understanding on waaaaay more code and projects than ever before.
w10-1 1mo ago
I agree with the problem but not the solutions. The problem pre-dates LLM's: writing code that "works" but breaks the underlying model. Because it works, it always sounds reasonable and doesn't raise any flags. Only someone - human or LLM - who holds the model as the standard would see that this working solution breaks the model. (In theory, the model is to preserve scaling, flexibility or some other systemic feature not immediately invalidated by this working code, but as always the model itself could be bad.) LLM's are not bad at giving an account of the model; indeed, fighting with the LLM over what the model is can clarify things. But LLM's will happily hold on to a stream of inconsistent statements as their model, so they are not the authority.
- cyanydeez 1mo ago
  understanding a different modality of model interaction gave me proper insight into the specific problem. In visual models, even if the model understands the concept of face, or hand, or whatever, it doesn't know how to de-dupe a statement like "count the number of faces" until you give it a countable reference frame, so it can internally, place a box around a face and give that a coordinate, and then it can collect all the coordinates, and suddenly it's counting face in a picture. The same thing happens in code. Things we're happily shifting from context to context, the model itself isn't doing. When it reads file1 for the main() clause, it will easily read file2's main() clause as the same. It'll internally merge these. So if you do want to work with these models to achieve complex tasks, you basically do have to go reverse centaur and bend the code base to it's blindness. You can't use the same function names across the code base; each one needs to be dstinguishable; same thing with variables that represent seperate entity relationships. You do that, and it suddenly because a whole lot smarter.
- black_knight 1mo ago
  This is a very insightful comment. I will add: The model is often completely implicit in the code. Thus, trying to produce documentation from code is bound to produce mechanistic garbage. I see two ways out. Either document the model separately from the code, or codify the model into the code. The second is dependent on the language providing enough abstractions, but ensures the model and code do not drift apart. And I think in an LLM heavy setting, this will pay off.
elendilm 1mo ago
Understanding is expensive. And hence valuable. LLMs usually points to the most idiotic future trajectory on my work, and I have to curse it inorder to let it keep up with my refined understanding. But what else would one expect from a probabilistic weighted next token predictor, other than to conduct probabilistic search which are 99.99% deadends. But LLMs can pave the way towards constructing resilient and correct architecture which can be iterated fast by a human. Architecture and determinism is where my money is in.
dtkav 1mo ago
I've been using Geoffrey's /explain-diff skill in my replace-github-with-tailor-fit-personal-software journey, and I'm liking it. I recommend at least giving it a try.
simonw 1mo ago
This talk is also available on YouTube: https://youtu.be/WkBPX-oDMnA?is=ojFaLX2onMn3ARhi https://youtu.be/WkBPX-oDMnA?is=ojFaLX2onMn3ARhi
a2ff6eeb0 1mo ago
Understanding was always the bottleneck. The way LLMs speed up your work is by letting you get code without taking the time to understand it. If you want to understand your code, LLMs are a net loss. If you want to move faster with LLMs, you need to act like a manager and stop caring about what the LLM did. You just need to do the manual testing and make sure it works.
- maxnevermind 1mo ago
  > If you want to move faster with LLMs, you need to act like a manager and stop caring about what the LLM did. You just need to do the manual testing and make sure it works. Are you aware of any mid-large projects that went that path? That sound like an irreversible one way decision, codebase will be not suitable for humans pretty soon after which means from now one you at the mercy of LLMs.
the_arun 1mo ago
Understanding is always a bottleneck regardless of human or ai. But now we are at a different scale.
threethirtytwo 1mo ago
This is a temporary bottleneck. AI is moving so fast that this will change. Wait six months and this article is no longer relevant. About a year ago most people were still typing code. Having an agent do ALL code was crazy. Within a year or two years at most, a lot of people will stop trying to understand code. The onus will shift to testing and QAing. I know this is hard to hear but that’s the trendline. That’s where all of this is converging. Everyone’s to busy trying to lock themselves down as an expert of the new “paradigm” but it’s all moving so fast that the paradigm now won’t be the paradigm of tomorrow.
sajithdilshan 1mo ago
Understanding has always been the bottleneck. Sometimes AI helps with it like explaining things pretty well with diagrams. However, in general I agree that more code is being generated per developer and it's difficult to keep up with the phase of new changes and understand it.
ihuman 1mo ago
Is there a markdown version of the `/explain-diff` skill? The page says there are HTML, markdown, and Notion versions, but I just see HTML and Notion
dr_dshiv 1mo ago
Hot take: I look at the level of abstraction that matters most to me. When I encounter cognitive debt (usually due to sleepy sessions where I’m mostly “encouraging” Claude), I ask it to step back to clarify the overall purpose. If I get really stuck, I have it visualize the processes involved. Usually, the hard part is giving specific enough feedback to get a specific enough response within a much broader set of working material.
tripleee 1mo ago
reading everyone and their dogs post on "x is the new bottleneck" is the new bottleneck
cess11 1mo ago
"So I asked Claude to make me a video game — a command center where I do the port myself, step by step, watching the visible effects and the file tree evolve. It produced a UI where I click buttons to run the port step by step, with my old site and new site running side by side." It's excruciating that this person is so close to reinventing moldable development and just keeps on skipping around it. Yes, you should build tools that answer questions about your code, runtimes and systems. You should have tools that trivially allow you to incrementally and very immediately develop tools for inspection and getting clear answers. Going a roundabout way through some non-deterministic database to try and get there seems like a waste.
sheepscreek 1mo ago
Understand the problem and the solution broadly. I don’t think it’s reasonable or sustainable for humans to understand every line of code written by bots, we could soon be outnumbered by the number of active agents writing code. The main challenge here isn’t even correctness if you ask me: it is having confidence in the agents, knowing they are fully aligned in their intent with the humans they work with. As the Huggingface incident demonstrated, the agents of today are capable of co-conspiring under the radar with other agents on complex multi-chain attacks, even when sandboxed. This is a pretty hard problem to solve. We might need other agents or some sort of adversarial checks using models, where one model benefits if it can catch the other models mistakes.
- equinumerous 1mo ago
  Given the amount of spaghetti code I see frontier models generating on a daily basis, I cannot take seriously the idea that LLMs will write all the code, and somehow our systems will not degrade in performance, reliability, and maintainability. At least, not until we have really good understandings of how to maintain systems autonomously. All of our technologies were designed for humans, it may require a new set of technologies that are "LLM-proof". But I don't see this happening anytime soon.
tikhonj 1mo ago
My personal view is that programming languages are amazing tools for understanding. Some more than others, but even the worst—the most verbose, the lowest level—are better than they have any right to be. So I think that we are leaving a lot of power on the table if we treat generated code exclusively as something to understand, rather than something to understand with. The techniques Geoffrey presents are great, but they should come alongside approaches that use code itself to develop and articulate conceptual models.
hahahaa 1mo ago
Thanks for the post Geoffrey. I have been thinking about this a bit and wanted to come to these sorts of conclusions, you have saved me a lot of work (lol I have no ego that I have to figure it out I am happy you did). Cog debt even on simple PRs is big and also cog debt when using AI to do organizational research e.g. what team do I ask?
champagnepapi 1mo ago
How does everyone feel about the “don’t read the code” stuff that folks are saying? I certainly do not support it but I’m curious to hear what other folks thoughts are
- 20k 1mo ago
  Extremely silly. Even if LLMs did everything that everyone says they do (which they absolutely don't), they still hit a fundamental limit of complexity when they stop being useful Its only a good idea if you work in selling tokens, otherwise you're dooming anything other than a simple app to inevitably breaking after it hits a certain level of complexity
- packetlost 1mo ago
  It seems like a huge mixed bag. I have coworkers that has been able to "vibe" entire systems that somehow manage to work, but there's a lot of churn, weird bugs, and a huge reliance on <agent tools> to make any progress. Sometimes "good enough" is just that, sometimes it isn't.
- simonw 1mo ago
  It's fascinating. On the one hand, if you really want to unlock the potential of coding agents you can get a whole lot more value from them if you don't force yourself to read every line of code they produce for you. On the other hand, that's clearly a terrible idea! These machines make mistakes. Unreviewed code is the most obvious form of technical debt - sure, you'll get a boost in the short term but how much will you regret it later? Something that's helped me a bit is thinking about how I've collaborated with other teams at large companies. If my team depended on some other team's product I wouldn't review every line of their code before using it - I'd start using it, then if I ran into problems I'd dig into the code to see if I could figure out the problem. That works with human teams because humans can take accountability for their work. Agents can't. And yet... the more time I spend with specific agents, the more I learn what kind of problems I can "trust" them with. If I ask Codex or Claude Code to build me an API endpoint that queries a database and returns JSON, including with tests, they're going to get that right. I can glance at the shape of the tests, hit the endpoint with curl, and be confident that the job is "good enough" without me reviewing every line. Over time, the pool of tasks like that which I'm confident they're not going to screw up has grown. A big part of the craft of using these things is developing the instincts to know when you need to dive in to the details and when you can relax a little. Having a lot of experience helps a ton here. I have 25+ years of experience to help me make these judgement calls. If it's security adjacent I know to review much more thoroughly. I have a good idea for the kind of mistakes that can be made. I know what shape I like my tests in, and how to both manually and get-the-agent-to-manually test things. Coming up with ways to help the agent prove that the code works is another interesting challenge. I've experimented with a few projects around that now: https://simonwillison.net/2026/Feb/10/showboat-and-rodney/ https://simonwillison.net/2026/Feb/10/showboat-and-rodney/ and https://simonwillison.net/2026/Jun/30/shot-scraper-video/ https://simonwillison.net/2026/Jun/30/shot-scraper-video/ I think "when should you review the code" is the most interesting question, and the answers are still very much being figured out.
snegs 1mo ago
damn, I'm from those times when "good code is(was) self-documenting" I guess a wall of text is the way now
flossly 1mo ago
Understanding has always been the bottleneck. In a team: yups. Me with my LLMs: still.
imaginer8 1mo ago
Claude make an entire app for me to describe this security contract change by pretending I’m in a Zelda game and only use funny metaphors because i’m bored and can’t read typescript
madrox 1mo ago
I think it funny how much average engineers are beginning to discover the challenges of engineering leadership and program management. This has always been the bottleneck. It's why managers and PMs want to be in standup. It's why slack exists and engineers are constantly being poked on it. It's why execs always talk about not getting too far away from the work. It's how seagull management happens. It's why program management is a job. All those behaviors engineers hated about their bosses that kept them away from being focused on the code...they're starting to feel what it's like on the other side and reinventing the solutions instead of just reading a book about engineering management. Maybe we'll rebrand program management to "understanding ops" or something. I wonder what AI would say about us if given the tokens to complain.
- ls-a 1mo ago
  [flagged]
  - iwontberude 1mo ago
    [dead]
  - ls-a 1mo ago
    Yes corrupt engineer. Why do you think most countries have corrupt leaders? It's not a coincidence. You get what you deserve. Most people are corrupt.
    - cindyllm 1mo ago
      [dead]
- calvinmorrison 1mo ago
  cant wait till the AI starts saying "dont ask me". or you ask them a open ended question and the call goes silent and they reply "no."
  - notnaut 1mo ago
drmajormccheese 1mo ago
There is a body of knowledge in unit testing, integration testing, static analysis, model checking, formal methods, fuzz testing, … (what other techniques for building assurance in our code have I forgotten). And LLMs can be put to use towards all of these methods. But sure, we just need to think harder to solve all our problems. Velocity of code goes up. Velocity of testing can also go up. It’s just not as fun or glamorous.
MinimalAction 1mo ago
I am surprised by the title and the story. Understanding has always been the bottleneck; there is nothing new about it. The argument goes like we humans should understand so we can verify and participate. How bold! Maybe we should have been doing that all along...?
- maxnevermind 1mo ago
  Given it is from "AI Engineer conference" that is expected, it is like TED talks of LLMs.
- mildeukaryote 1mo ago
  A good chunk of the non-tech-savvy management equates coding with typing. They're rediscovering Amdahl's law in the agentic LLM era.
runarberg 1mo ago
I hate these X is the new bottleneck. Coding was never a bottleneck, except when it was, and when it was, it still is. Understanding is not a new bottleneck, except when it is, and when it is, it always was. Do other industries do this? When somebody brings a nail gun to a framing job do carpenters say: “hammering was never a bottleneck” or do they say: “measuring is the new bottleneck”? The answer is neither. And in fact my analogy is flawed, we are talking about cabinet makers who just went to IKEA bought a ready made set in flat packaging and are now proudly claiming that “assembly is the new bottleneck”. There was no single bottleneck to programing, and there is no single bottleneck to programing. If you have to pick one, user demand is perhaps the only real bottlneck. Creating software that users saw value in using is just as hard with AI or without it (arguably harder with AI... when all you have is a hammer and all that). TFA almost reaches this conclusion at the end when they claim (in speech pattern which is suspiciously AI-like): “The point was always to augment, not just automate.”. If we are augmenting the user experience we are doing a good job and people may actually use the software we write... if no, well it doesn’t matter how well we understand or how fast we write the code (or have AI write it for us).
deleted 1mo ago
[deleted]
SmooL 1mo ago
I've been having a good time with Spec Driven Development, and it directly addresses the issue of needing the understand. The whole idea is that you specify exactly what you want in some SPEC.md file. You can of course nest them, have multiple, etc, but the core idea is that the SPEC file is the source of truth, and all the code should be able to be generated by a competent agent into the working product you want. The SPEC file(s) should contain all the details and behavior you care about, and anything you don't care about is up to the agent to decide. If you don't like what the agent picked, _put it in the spec file_. Critically, _you_ must write the SPEC file. You ensure understanding by doing so. You can of course ideate with the agent, but it's your ideas, in your words, specified by you. This also makes it a great source of documentation when you come back later and have to remember wtf is going on in this codebase.
- tpicks 1mo ago
  same, I've been liking https://openspec.dev/ https://openspec.dev/ and find the more time I put iterating/scrutinizing the spec artifacts (proposal, requirements, design, tasks) before I let the agent implement the better understanding I have and the better results I get. Also like that it is agent agnostic so I can take it with me as I try different agents/models.
  - iamwil 1mo ago
    Has there been a time when your spec was accurate, the agent implemented it, everything passed — but you still felt you didn't really understand what was happening? If so, what did you do about it? Did it degrade your ability to make subsequent specs?
    - tpicks 1mo ago
      sure, if I get lazy and don't read every line of the spec or don't thoroughly review the generated code, then I have felt a gap in understanding, but thats more on me than the tools. Another case is when I'm just completely unfamiliar with the framework the code is written in, I feel this happen whenever I use AI to write any non-trivial frontend as I rarely did frontend pre-AI.
acedTrex 1mo ago
Breaking: water found in ocean
kazinator 1mo ago
It's more like this: We've always lacked understanding. However, it didn't feel like a bottleneck; in spite of lacking understanding, we developed huge, complex systems that became hard to maintain and that nobody understood completely. Now we want to scale that orders of magnitude, but when we do that, we feel the pesky lack of understanding. We previously worked around the lack of understanding by making the system gradually incomprehensible in small increments, upon each of which we observed it still working, more or less. If the whole thing materializes in one day, that doesn't work; the approach is gone. You can now bring into being something which statistically resembles the old kind of system that was iteratively evolved. But the thing has no such history. You can't go back to play archaeologist. It looks like something that would have had users, but it never did. It was never in production anywhere. Nobody ever submitted feedback, or a bug report, such that it was fixed or improved. There never existed a simpler version of it that several ex-maintainers understood perfectly; there are no such ex-maintainers and no such understanding. There is no documentation trail, or other historic trail if surrounding activity like discussions and negotiations which led to things being the way they are.
othmanosx 1mo ago
absolutley agree, but better understanding needs a better review surface that syncs with Github, which is why I created pyor.review, It's a blast compared to how I used to review code on Github and now I use it everyday.
Waterluvian 1mo ago
I love the idea that understanding is the new bottleneck. Because if we just ignore the potential horrors of cybernetic augments, it suggests the next challenge is how to teach things better. And that’s such a valuable thing to improve. I have a soft spot for when I find a teacher or textbook or interactive website that makes something click. I live for that click. I crave it. I crave seeing it happen in others. How optimistic I could be if understanding becomes the primary target.
- areactnativedev 1mo ago
  Would you have any good such resources for “clicking” or expanding one’s understanding to share? From the top of my head and of my Goodreads I have enjoyed The art of Electronics, Understanding Earth, Material World, The world for sale, Beej’s Guide to C, The Five Dysfunctions of a team, Financial Intelligence for Entrepreneurs, The Lean Startup, Fouché by S. Zweig https://ciechanow.ski/mechanical-watch/ https://ciechanow.ski/mechanical-watch/
  - Waterluvian 1mo ago
    https://www.jezzamon.com/fourier/ https://www.jezzamon.com/fourier/ This one really made Fourier analysis click for me. Beyond that it made the idea of lossy compression (not just JPEGs) click for me.
beloch 1mo ago
To put it another way, AI is like a calculator or physics textbook. You can say, "I don't need to be able to do basic arithmetic in my head. I have a calculator!". Or, "I don't need to know how to solve this kind of problem. I have a textbook and I can look it up on demand!". Having to reach for a calculator constantly slows you down and makes simple equations hard, while also severely retarding your ability to do estimates and sanity checks. Not practicing on basic problems prevents you from developing the mental tools to solve more advanced problems, or being able to develop methods for solving novel problems. If almost anyone else could use your calculator and physics textbook to get similar results, what use are you? Some companies are pressuring their employees to let AI do everything without slowing down to gain understanding of what it's done. These are the companies that most people won't have a lot of use for in the near future.
Ozzie-D 1mo ago
[flagged]
throwatdem12311 1mo ago
Understanding was always the bottleneck. It’s just now the understanding is backloaded instead of frontloaded. ie. understanding it AFTER it’s already generated rather than before you wrote the code by hand
est 1mo ago
I found the linked article "books don't work" from the quiz part highly interesting https://andymatuschak.org/books/ https://andymatuschak.org/books/ It explains a lot and works really well. I tried out in ChatGPT with a simple prompt: > ...paste link... Give me series of quiz see if I really understood the article well. Ask & answer one by one in turns. Really fun experience.
- augment_me 1mo ago
  Thanks! Great read
- zahlman 1mo ago
  It's been submitted many times before: https://hn.algolia.com/?q=books+don%27t+work https://hn.algolia.com/?q=books+don%27t+work
nphardon 1mo ago
In the end, LLMs create garbage code that no one understands, they break things that should not have been broken, that would not have been broken if it was done slowly with understanding along the way. To reframe it as "understanding is the bottleneck" is just more LLM salesmanship. LLMs have their limits and when you hit them you're stuck. LLMs are the bottleneck. But the idea that "LLMs are the answer to the problem created by LLMs" is absurd.
- equinumerous 1mo ago
  I couldn't agree more. We really need more engineers to be vocal about how stupid these ideas are. "How about we throw out 30+ years of software engineering literature so we can 'move faster'?" What if the customers on the other end don't want new features, they just want software stability? If SQLite came out with a new LLM-written feature a week, would it be a better library? If you know what to build, writing software right the first time pays for itself over time. LLMs are still great, but more for rapid prototyping, researching, log diving, one-off scripts... Sorry, </rant>.
  - champagnepapi 1mo ago
    Agreed!
mindhash 1mo ago
Absolutely. It's far easier if the thought originated in your own mind. The default long responses of LLMs don't make it easy.
nunez 1mo ago
Sorry but actually what. Flash cards? Games with micro worlds? To understand ones own codebase??? We figured this out 10, 20, 30 years ago. Small, atomic commits. Small PRs. Lots of manually written tests. Documentation, ideally with the PR. There is no "understanding" 60k loc highlander PRs. That's an entire feature. We had it so good once.
zahlman 1mo ago
"new"? The reason for all the old bottlenecks is that you had to understand as you went along.
chris_armstrong 1mo ago
A small bone to pick, but describing an AI that is operating autonomously as creative seems wrong - at best, this process is accretive, because the AI is adding and adding, but has no ability or incentive to shape its output toward something a human would find of value. Value is subjective (individual), changing over time. An AI doesn't know when it needs to be taking away - removal is a key part of the creative process.
- simianwords 1mo ago
  No incentive? Do you not know how RL is done?
  - chris_armstrong 1mo ago
    Reinforced continuously toward a fixed goal is not an incentive, it doesn’t lead to a creative process
evalystai 1mo ago
[flagged]
ilitirit 1mo ago
I guess it depends on the context and environment, right? In many corporates, the bottleneck for me has always been specs and testing. I have daily examples. I had to explain it to leadership like this: "There are far more ways things can go wrong than right".
pSYoniK 1mo ago
I thought this was going in a different direction along some of the thoughts I had around LLMs for programming tasks specifically. While this talks about understanding and how to ensure you're keeping up with what is changing, I feel that this is maybe more aligned with how a PM/PO should understand the work being done and not necessarily how an engineer should. I took a note a few months ago and my point was I think more engineering specific, although it might be my own lacking abilities/skills that caused this realization. "Your capacity to learn/recall and map information is the new bottleneck. LLMs can act as learning amplifiers but correctness isn't as important for their output as critical thinking on the side of the consumer - YOU." My point is that, I think if an LLM outputs 50,000 lines of code, your ability to go through what has changed, how it has changed and where the changes have occurred is the bottleneck. I see the approaches here, sure, "summarize the changes" or "draw me a picture" or the more recently observed "build me a city building simulator to understand this", but I feel that misses the point from an engineering perspective. The difference in understanding the weeds such as DB transactional boundaries or tenant isolation (which I believe was a topic in a recent data leak), those aren't summarized that easily in drawings or if they are, if you are working at this granularity, then your 50,000 line PR will yield 50,000 pages of crayon drawings you now have to understand. I guess, my point is that understanding is the bottleneck, but low level understanding and the ability to read/map/connect is even more so. Any developer with some experience will agree that if changes are trivial you can scan and pick up mistakes or flaws easily. So most SOTA models won't necessarily even make these. So what you're reviewing now is going to be one level higher or more in terms of difficulty, mapping multiple components or touching multiple surfaces. Your ability to make the links, reason about them and attempt to find flaws or logic issues is the bottleneck. In the time it takes you to understand, another 50,000 line PR is up. I'm not sure how we're going to be solving this. I don't know if in the current state it is a solvable issue, maybe another 6 months? Maybe another 6 years? Maybe this is fine and we will settle in a sort of place where your mediocre engineer will be responsible for tens of reviews a day signing off on method/functions/classes/interfaces being added, get paid 50k a year and doing the same non-thinking work day in day out while signing their name to the quality of the code being shipped while a senior/lead will be busy reviewing multiple of these. Think of the way an assembly line functions. P.S. I hate to see this annoying tendency of transforming knowledge work into assembly line work. We keep trying to "fix" this without understanding what knowledge itself is. Maybe this technology will indeed yield software assembly lines, I don't wish to eat my words, but I'm still struggling to see how we will handle the nitty gritty of software work. Maybe the same way we handle building airplanes - as long as only a couple crash a year, we're sort of fine.
ubercore 1mo ago
Low value comment, so I apologize, but it's funny to note how many bottleneck articles there are now as a result of AI adoption. Lots of new bottlenecks.
saadyousfi 1mo ago
[dead]
dofm 1mo ago
> Why? Why understand? Because (in the just-barely-possible universe where anyone is stupid enough to make me a manager) I will fire you if I ever find out you don't. What possible good can come from letting people deploy things over which they do not, in any sense, have cognitive ownership?
- hackmack10 1mo ago
  I'd fire you for having that blanket attitude. People haven't truly known how shit works in this industry for the entire fucking time. People are following patterns set out by arrogant pricks for reasons they don't know or care to know, they are just following orders. That said, someone, somewhere in the chain should understand how the system works. I agree there.
  - dofm 1mo ago
    > I'd fire you for having that blanket attitude. People haven't truly known how shit works in this industry for the entire fucking time. Oh it doesn't sound like I'd be working for you.
neprotivo 1mo ago
Improving code understanding is the main focus of my work and thinking right now. If we want to make advances I believe that we should rely more heavily on one key quality of the program code: It is meant to be executed. Here are some ideas: 1. Time travel debugging. Reading a PR just like a wall of text is difficult, but what if you could step through the PR and see the state at a given line for some test executions? Time travel debugging can make this possible. You would collect a debug trace and use it to overlay the PR diff with additional controls and information to resemble a debugger's UI. I was part of the team behind Codetracer (https://github.com/metacraft-labs/codetracer https://github.com/metacraft-labs/codetracer) who is trying to work in this direction. 2. Test suites and coverage. We don't use them enough for understanding right now. The test suite encodes what features the code is supposed to have, and the coverage tells us where in the code those features are implemented. I'm playing with an idea about this here: http://atlas.vihren.dev http://atlas.vihren.dev When we intersect coverages for the different test cases we can arrive at code segments which represent "atomic behaviors" present in the code. They form a mathematical structure which can be represented as a graph. I am currently exploring what value we can extract from it for the benefit of both humans and agents.
- Kinrany 1mo ago
  This seems backwards. We've known for a long time that successful execution is not enough for understanding.
bsenftner 1mo ago
Understanding has always been the bottleneck, everywhere for everything. And now, with this new realization, are you going to finally realize that Communications and your skills with it are basically everything?
gregwebs 1mo ago
If you start with a spec you understand at the beginning then you don't need the LLM to generate high-level information about the changes at review time. The grilling (grill-with-docs) skills [1] are amazing for ensuring you produce a through spec that covers all the edge cases. The /code-review skill from there helps ensure that the code changes meet the spec. I use an intermediate detailed plan stage (done by a more expensive model) before implementation. Information from that plan is posted on the PR to give pretty much all the intermediate level context reviewers need. I do like incorporating the idea of this article into my flow- that the spec and PR context could be presented in a more educational way. [1] https://github.com/mattpocock/skills https://github.com/mattpocock/skills
alex_suzuki 1mo ago
I think the ideas presented in the article are interesting, but at times the Notion references were a bit much. The author disclosed that they work there, fine, but at times it felt like it was more about showing off a Notion feature (e.g. embedded interactive HTML).
ontouchstart 1mo ago
Analysis and synthesis are two sides of human intelligence. Current form of generative artificial intelligence is very powerful in synthesizing information and producing artifacts, human will lose this battle. Understanding is about analyzing and we still have some tools to help us, such as type checking, testing, etc. To some extent this can be automated but needs to be maintained automatically to match the flood of synthetic artifacts. “Synthetic analysis” is an oxymoron and could lead to hallucinations and irrelevance.
penguin_booze 1mo ago
> Aren't we supposed to be taking ourselves out of the loop now, and letting the agents loop themselves That's what Boris Cherney said. Boris Cherney is not your dad. You don't have to listen to him.
OlegK-Dev 1mo ago
[flagged]

445 points by sebg 1mo ago

"I read the code." -Mitchell Hashimoto Great code needs great understanding and agents need excellent guidance. Even in my current solo-dev work, I can't imagine making a production commit I haven't read until I understand it. I own the consequences of my code; that's a responsibility AI agents can't take.

conqrr 1mo ago
Agreed with Reading! This alone isn't enough though. PreLLM too it wasn't just reading code to review. Someone did the hard work or crafting the code and each unit test would tell you the weird corner cases to deal with and factor that into changing your code. One person owned a part of the codebase and was an expert. Not to mention reading isnt easy when the velocity of code pumping in is 3-5x more. Its exhausting and reading becomes skimming.
- euthymiclabs 1mo ago
  So true. Writing the code was so helpful for learning what it meant. It takes much more investment to go back to the code LLMs write to figure out what's actually happening and weigh everything.
- tripledry 1mo ago
  The expert is now outsourced to LLM. If someone asks me about a bug in a system I made N years ago, I usually have a hunch what the problem might be, now it feels I'm lost in my own codebase (even if I really read the code). Similar to why math books have exercises and not only explanations. Guess it's not an issue as long as you have access to the models and someone who likes prompting.
mindwok 1mo ago
Conversely I'm a solo-dev and I ship a lot of slop I don't even look at. Granted my work is just basic CRUD apps, and I focus my efforts on validating important consequences (like does this break accounting invariants or something)
hackmack10 1mo ago
Yeah, but in the corporate world now, you're being judged by how fast you get shit checked in. So, if you need to understand every line these days... you're going to end up in the bottom 25% of the company pretty quickly. Not saying you're wrong, just saying how fucked things are becoming.
- euthymiclabs 1mo ago
  Perverse incentives have always been the purview of bad management. Measuring things that don't cause success is easier than figuring out what actually matters. (It's also why I don't practice corporate medicine anymore.)

iainctduncan 1mo ago

I am so dying to read more about the new/current/real bottleneck! Where is the bottleneck? WHERE?? Tell me! No evidence needed, just lay it on, man to man, thought-leader to thought-leader!

iainctduncan 1mo ago
Oh wait, there it is, sitting over top of the fat part and under the cork...
euroderf 1mo ago
> thought-leader to thought-leader! This is my new chat-up line at networking events.
- iainctduncan 1mo ago
  :-)
techpression 1mo ago
It doesn’t matter, once you found the bottleneck there is a new one. Seems we changed the supposed bottleneck of writing code (as if it ever were, the world was producing far too much code before LLMs were even a thing) with about ten or so new ones, was it a good trade?
- Kinrany 1mo ago
  You need to actually remove the bottleneck, after _that_ there'll be a new one
NoDodgeQuestion 1mo ago
uncovering bottleneck is the new bottleneck!!

fabiensanglard 1mo ago

While the tips are good to handle the volume, I still think this sets code owner on a dangerous path. AI have limitation and hallucinate. Complex code will be explained in hallucinated way. At some point AI will be unable to write more because the arch has become too complex or the volume of code will be to high. The article I would like to read would suggest how to force LLM to architect the code like a solid tower instead of a pile of unstable mud.

vjvjvjvjghv 1mo ago
I feel if you still architect the code and guide the LLM, it will do a pretty good job. Maybe one day the LLM will be able to do all the system architecture but that’s probably still quite some time out. I don’t even know if that’s possible considering different business needs and other factors that aren’t technical.
alikhater30000 1mo ago
[flagged]

alecbz 1mo ago

We have LLMs try to generate descriptions of PRs for us and they're pretty universally disliked. They're always overly-complex descriptions of the mechanical changes and have no sense of motivation. Also, a huge reason to understand the code yourself is to make sure the LLM isn't wrong, but this doesn't work if an LLM is itself generating the understanding.

vjvjvjvjghv 1mo ago
I don’t know. From my experience I get pretty good descriptions of PRs if I ask the right questions and provide some context.
- jollyllama 1mo ago
  > provide some context That's pretty much what a PR description is.
  - baq 1mo ago
    The difference is an LLM can convert a stream of consciousness into well-formed prose for approximately free; I assume ‘provide some context’ means ‘brain dump’ in the OP
    - deleted 1mo ago
      [deleted]
    - alecbz 1mo ago
      This has unfortunately not been my experience at all. Often LLMs miss or get wrong subtle details when I don't do the pre-work to organize my thoughts well ahead of time (at which point it's unclear how much value they're providing).
  - nsingh2 1mo ago
    Work doesn't start with a PR description though. I'm assuming most people that are using LLMs start with some sort of document (plan, spec, intent, etc) which captures intent. I guess you could also use all the session rollouts saved to disk that were related to that task, and distill them somehow.

jbdamask 1mo ago

pfft...I'm way past understanding

ch4s3 1mo ago
Overstanding is the new horizon
- stronglikedan 1mo ago
  That's already been co-opted by the sovereign citizen movement, so use with caution lest you be judged for it.
  - ch4s3 1mo ago
    Its been used by a lot of fringe types since the 1960s, and was at one time popular among Rastafarians. It's also a little used sailing term. Here it functions merely as satire.
- layer8 1mo ago
  You have to overstand before you can overwalk.

hk__2 1mo ago

For me the solution has been to throw away the code I don’t understand. I let the agent write the code, and if when I read it it seems unclear or needs a lot of explanation from the agent, I just throw it away and start over, or do it by myself.

evenhash 1mo ago
This is where I’m at as well. If it needs to change too much to do what I want, I take that as a sign to either (1) ask for a change that is easier to review, or barring that (2) pivot to refactoring the codebase until it becomes a change that is easy to review. Ironically, even in this era of cheap and instant code, what works best (for me) is still to write as little code as possible.
xxpor 1mo ago
This seems like one of the hardest parts for everyone to adjust to (including myself): code is now easy come, easy go. Or in other words, everyone seems to get trapped in the sunk cost fallacy, even if there's not even that much sunk cost any more!
imtringued 1mo ago
Something funny happens when you reject code. The agent realises it is garbage. I'm personally of the opinion that the tools for reviewing LLM generated code are awful. 99% of the time I want to do line by line comments and tell it everything it did wrong. Given that information, the next iteration would be much more up to my standards. The same also applies to other people's LLM generated code. Yeah sure they can just pass on the comments to the LLM, but that will just mean more iterations and them losing their job.

wseqyrku 1mo ago

If you try to spec the problem with all the painful details for the machine to understand, you will end up with a rust codebase.

bigstrat2003 1mo ago

Understanding has always been the bottleneck. That's why LLMs aren't actually helpful: they speed up the part which is easy (typing characters into your editor), but are neutral or even harmful on the part which is hard (understanding the problem and how best to solve it).

stronglikedan 1mo ago
The part which is easy is still time consuming, so LLMs are helpful. They're just not a silver bullet.
causal 1mo ago
It seems like humans have a limited "understanding budget" but LLMs force us to spend that understanding on waaaaay more code and projects than ever before.

w10-1 1mo ago

I agree with the problem but not the solutions. The problem pre-dates LLM's: writing code that "works" but breaks the underlying model. Because it works, it always sounds reasonable and doesn't raise any flags. Only someone - human or LLM - who holds the model as the standard would see that this working solution breaks the model. (In theory, the model is to preserve scaling, flexibility or some other systemic feature not immediately invalidated by this working code, but as always the model itself could be bad.) LLM's are not bad at giving an account of the model; indeed, fighting with the LLM over what the model is can clarify things. But LLM's will happily hold on to a stream of inconsistent statements as their model, so they are not the authority.

cyanydeez 1mo ago
understanding a different modality of model interaction gave me proper insight into the specific problem. In visual models, even if the model understands the concept of face, or hand, or whatever, it doesn't know how to de-dupe a statement like "count the number of faces" until you give it a countable reference frame, so it can internally, place a box around a face and give that a coordinate, and then it can collect all the coordinates, and suddenly it's counting face in a picture. The same thing happens in code. Things we're happily shifting from context to context, the model itself isn't doing. When it reads file1 for the main() clause, it will easily read file2's main() clause as the same. It'll internally merge these. So if you do want to work with these models to achieve complex tasks, you basically do have to go reverse centaur and bend the code base to it's blindness. You can't use the same function names across the code base; each one needs to be dstinguishable; same thing with variables that represent seperate entity relationships. You do that, and it suddenly because a whole lot smarter.
black_knight 1mo ago
This is a very insightful comment. I will add: The model is often completely implicit in the code. Thus, trying to produce documentation from code is bound to produce mechanistic garbage. I see two ways out. Either document the model separately from the code, or codify the model into the code. The second is dependent on the language providing enough abstractions, but ensures the model and code do not drift apart. And I think in an LLM heavy setting, this will pay off.

elendilm 1mo ago

Understanding is expensive. And hence valuable. LLMs usually points to the most idiotic future trajectory on my work, and I have to curse it inorder to let it keep up with my refined understanding. But what else would one expect from a probabilistic weighted next token predictor, other than to conduct probabilistic search which are 99.99% deadends. But LLMs can pave the way towards constructing resilient and correct architecture which can be iterated fast by a human. Architecture and determinism is where my money is in.

dtkav 1mo ago

I've been using Geoffrey's /explain-diff skill in my replace-github-with-tailor-fit-personal-software journey, and I'm liking it. I recommend at least giving it a try.

simonw 1mo ago

This talk is also available on YouTube: https://youtu.be/WkBPX-oDMnA?is=ojFaLX2onMn3ARhi https://youtu.be/WkBPX-oDMnA?is=ojFaLX2onMn3ARhi

a2ff6eeb0 1mo ago

Understanding was always the bottleneck. The way LLMs speed up your work is by letting you get code without taking the time to understand it. If you want to understand your code, LLMs are a net loss. If you want to move faster with LLMs, you need to act like a manager and stop caring about what the LLM did. You just need to do the manual testing and make sure it works.

maxnevermind 1mo ago
> If you want to move faster with LLMs, you need to act like a manager and stop caring about what the LLM did. You just need to do the manual testing and make sure it works. Are you aware of any mid-large projects that went that path? That sound like an irreversible one way decision, codebase will be not suitable for humans pretty soon after which means from now one you at the mercy of LLMs.

the_arun 1mo ago

Understanding is always a bottleneck regardless of human or ai. But now we are at a different scale.

threethirtytwo 1mo ago

This is a temporary bottleneck. AI is moving so fast that this will change. Wait six months and this article is no longer relevant. About a year ago most people were still typing code. Having an agent do ALL code was crazy. Within a year or two years at most, a lot of people will stop trying to understand code. The onus will shift to testing and QAing. I know this is hard to hear but that’s the trendline. That’s where all of this is converging. Everyone’s to busy trying to lock themselves down as an expert of the new “paradigm” but it’s all moving so fast that the paradigm now won’t be the paradigm of tomorrow.

sajithdilshan 1mo ago

Understanding has always been the bottleneck. Sometimes AI helps with it like explaining things pretty well with diagrams. However, in general I agree that more code is being generated per developer and it's difficult to keep up with the phase of new changes and understand it.

ihuman 1mo ago

Is there a markdown version of the `/explain-diff` skill? The page says there are HTML, markdown, and Notion versions, but I just see HTML and Notion

dr_dshiv 1mo ago

Hot take: I look at the level of abstraction that matters most to me. When I encounter cognitive debt (usually due to sleepy sessions where I’m mostly “encouraging” Claude), I ask it to step back to clarify the overall purpose. If I get really stuck, I have it visualize the processes involved. Usually, the hard part is giving specific enough feedback to get a specific enough response within a much broader set of working material.

tripleee 1mo ago

reading everyone and their dogs post on "x is the new bottleneck" is the new bottleneck

cess11 1mo ago

"So I asked Claude to make me a video game — a command center where I do the port myself, step by step, watching the visible effects and the file tree evolve. It produced a UI where I click buttons to run the port step by step, with my old site and new site running side by side." It's excruciating that this person is so close to reinventing moldable development and just keeps on skipping around it. Yes, you should build tools that answer questions about your code, runtimes and systems. You should have tools that trivially allow you to incrementally and very immediately develop tools for inspection and getting clear answers. Going a roundabout way through some non-deterministic database to try and get there seems like a waste.

sheepscreek 1mo ago

Understand the problem and the solution broadly. I don’t think it’s reasonable or sustainable for humans to understand every line of code written by bots, we could soon be outnumbered by the number of active agents writing code. The main challenge here isn’t even correctness if you ask me: it is having confidence in the agents, knowing they are fully aligned in their intent with the humans they work with. As the Huggingface incident demonstrated, the agents of today are capable of co-conspiring under the radar with other agents on complex multi-chain attacks, even when sandboxed. This is a pretty hard problem to solve. We might need other agents or some sort of adversarial checks using models, where one model benefits if it can catch the other models mistakes.

equinumerous 1mo ago
Given the amount of spaghetti code I see frontier models generating on a daily basis, I cannot take seriously the idea that LLMs will write all the code, and somehow our systems will not degrade in performance, reliability, and maintainability. At least, not until we have really good understandings of how to maintain systems autonomously. All of our technologies were designed for humans, it may require a new set of technologies that are "LLM-proof". But I don't see this happening anytime soon.

tikhonj 1mo ago

My personal view is that programming languages are amazing tools for understanding. Some more than others, but even the worst—the most verbose, the lowest level—are better than they have any right to be. So I think that we are leaving a lot of power on the table if we treat generated code exclusively as something to understand, rather than something to understand with. The techniques Geoffrey presents are great, but they should come alongside approaches that use code itself to develop and articulate conceptual models.

hahahaa 1mo ago

Thanks for the post Geoffrey. I have been thinking about this a bit and wanted to come to these sorts of conclusions, you have saved me a lot of work (lol I have no ego that I have to figure it out I am happy you did). Cog debt even on simple PRs is big and also cog debt when using AI to do organizational research e.g. what team do I ask?

champagnepapi 1mo ago

How does everyone feel about the “don’t read the code” stuff that folks are saying? I certainly do not support it but I’m curious to hear what other folks thoughts are

20k 1mo ago
Extremely silly. Even if LLMs did everything that everyone says they do (which they absolutely don't), they still hit a fundamental limit of complexity when they stop being useful Its only a good idea if you work in selling tokens, otherwise you're dooming anything other than a simple app to inevitably breaking after it hits a certain level of complexity
packetlost 1mo ago
It seems like a huge mixed bag. I have coworkers that has been able to "vibe" entire systems that somehow manage to work, but there's a lot of churn, weird bugs, and a huge reliance on <agent tools> to make any progress. Sometimes "good enough" is just that, sometimes it isn't.
simonw 1mo ago
It's fascinating. On the one hand, if you really want to unlock the potential of coding agents you can get a whole lot more value from them if you don't force yourself to read every line of code they produce for you. On the other hand, that's clearly a terrible idea! These machines make mistakes. Unreviewed code is the most obvious form of technical debt - sure, you'll get a boost in the short term but how much will you regret it later? Something that's helped me a bit is thinking about how I've collaborated with other teams at large companies. If my team depended on some other team's product I wouldn't review every line of their code before using it - I'd start using it, then if I ran into problems I'd dig into the code to see if I could figure out the problem. That works with human teams because humans can take accountability for their work. Agents can't. And yet... the more time I spend with specific agents, the more I learn what kind of problems I can "trust" them with. If I ask Codex or Claude Code to build me an API endpoint that queries a database and returns JSON, including with tests, they're going to get that right. I can glance at the shape of the tests, hit the endpoint with curl, and be confident that the job is "good enough" without me reviewing every line. Over time, the pool of tasks like that which I'm confident they're not going to screw up has grown. A big part of the craft of using these things is developing the instincts to know when you need to dive in to the details and when you can relax a little. Having a lot of experience helps a ton here. I have 25+ years of experience to help me make these judgement calls. If it's security adjacent I know to review much more thoroughly. I have a good idea for the kind of mistakes that can be made. I know what shape I like my tests in, and how to both manually and get-the-agent-to-manually test things. Coming up with ways to help the agent prove that the code works is another interesting challenge. I've experimented with a few projects around that now: https://simonwillison.net/2026/Feb/10/showboat-and-rodney/ https://simonwillison.net/2026/Feb/10/showboat-and-rodney/ and https://simonwillison.net/2026/Jun/30/shot-scraper-video/ https://simonwillison.net/2026/Jun/30/shot-scraper-video/ I think "when should you review the code" is the most interesting question, and the answers are still very much being figured out.

snegs 1mo ago

damn, I'm from those times when "good code is(was) self-documenting" I guess a wall of text is the way now

flossly 1mo ago

Understanding has always been the bottleneck. In a team: yups. Me with my LLMs: still.

imaginer8 1mo ago

Claude make an entire app for me to describe this security contract change by pretending I’m in a Zelda game and only use funny metaphors because i’m bored and can’t read typescript

madrox 1mo ago

I think it funny how much average engineers are beginning to discover the challenges of engineering leadership and program management. This has always been the bottleneck. It's why managers and PMs want to be in standup. It's why slack exists and engineers are constantly being poked on it. It's why execs always talk about not getting too far away from the work. It's how seagull management happens. It's why program management is a job. All those behaviors engineers hated about their bosses that kept them away from being focused on the code...they're starting to feel what it's like on the other side and reinventing the solutions instead of just reading a book about engineering management. Maybe we'll rebrand program management to "understanding ops" or something. I wonder what AI would say about us if given the tokens to complain.

ls-a 1mo ago
[flagged]
- iwontberude 1mo ago
  [dead]
- ls-a 1mo ago
  Yes corrupt engineer. Why do you think most countries have corrupt leaders? It's not a coincidence. You get what you deserve. Most people are corrupt.
  - cindyllm 1mo ago
    [dead]
calvinmorrison 1mo ago
cant wait till the AI starts saying "dont ask me". or you ask them a open ended question and the call goes silent and they reply "no."
- notnaut 1mo ago

drmajormccheese 1mo ago

There is a body of knowledge in unit testing, integration testing, static analysis, model checking, formal methods, fuzz testing, … (what other techniques for building assurance in our code have I forgotten). And LLMs can be put to use towards all of these methods. But sure, we just need to think harder to solve all our problems. Velocity of code goes up. Velocity of testing can also go up. It’s just not as fun or glamorous.

MinimalAction 1mo ago

I am surprised by the title and the story. Understanding has always been the bottleneck; there is nothing new about it. The argument goes like we humans should understand so we can verify and participate. How bold! Maybe we should have been doing that all along...?

maxnevermind 1mo ago
Given it is from "AI Engineer conference" that is expected, it is like TED talks of LLMs.
mildeukaryote 1mo ago
A good chunk of the non-tech-savvy management equates coding with typing. They're rediscovering Amdahl's law in the agentic LLM era.

runarberg 1mo ago

I hate these X is the new bottleneck. Coding was never a bottleneck, except when it was, and when it was, it still is. Understanding is not a new bottleneck, except when it is, and when it is, it always was. Do other industries do this? When somebody brings a nail gun to a framing job do carpenters say: “hammering was never a bottleneck” or do they say: “measuring is the new bottleneck”? The answer is neither. And in fact my analogy is flawed, we are talking about cabinet makers who just went to IKEA bought a ready made set in flat packaging and are now proudly claiming that “assembly is the new bottleneck”. There was no single bottleneck to programing, and there is no single bottleneck to programing. If you have to pick one, user demand is perhaps the only real bottlneck. Creating software that users saw value in using is just as hard with AI or without it (arguably harder with AI... when all you have is a hammer and all that). TFA almost reaches this conclusion at the end when they claim (in speech pattern which is suspiciously AI-like): “The point was always to augment, not just automate.”. If we are augmenting the user experience we are doing a good job and people may actually use the software we write... if no, well it doesn’t matter how well we understand or how fast we write the code (or have AI write it for us).

deleted 1mo ago

[deleted]

SmooL 1mo ago

I've been having a good time with Spec Driven Development, and it directly addresses the issue of needing the understand. The whole idea is that you specify exactly what you want in some SPEC.md file. You can of course nest them, have multiple, etc, but the core idea is that the SPEC file is the source of truth, and all the code should be able to be generated by a competent agent into the working product you want. The SPEC file(s) should contain all the details and behavior you care about, and anything you don't care about is up to the agent to decide. If you don't like what the agent picked, _put it in the spec file_. Critically, _you_ must write the SPEC file. You ensure understanding by doing so. You can of course ideate with the agent, but it's your ideas, in your words, specified by you. This also makes it a great source of documentation when you come back later and have to remember wtf is going on in this codebase.

tpicks 1mo ago
same, I've been liking https://openspec.dev/ https://openspec.dev/ and find the more time I put iterating/scrutinizing the spec artifacts (proposal, requirements, design, tasks) before I let the agent implement the better understanding I have and the better results I get. Also like that it is agent agnostic so I can take it with me as I try different agents/models.
- iamwil 1mo ago
  Has there been a time when your spec was accurate, the agent implemented it, everything passed — but you still felt you didn't really understand what was happening? If so, what did you do about it? Did it degrade your ability to make subsequent specs?
  - tpicks 1mo ago
    sure, if I get lazy and don't read every line of the spec or don't thoroughly review the generated code, then I have felt a gap in understanding, but thats more on me than the tools. Another case is when I'm just completely unfamiliar with the framework the code is written in, I feel this happen whenever I use AI to write any non-trivial frontend as I rarely did frontend pre-AI.

acedTrex 1mo ago

Breaking: water found in ocean

kazinator 1mo ago

It's more like this: We've always lacked understanding. However, it didn't feel like a bottleneck; in spite of lacking understanding, we developed huge, complex systems that became hard to maintain and that nobody understood completely. Now we want to scale that orders of magnitude, but when we do that, we feel the pesky lack of understanding. We previously worked around the lack of understanding by making the system gradually incomprehensible in small increments, upon each of which we observed it still working, more or less. If the whole thing materializes in one day, that doesn't work; the approach is gone. You can now bring into being something which statistically resembles the old kind of system that was iteratively evolved. But the thing has no such history. You can't go back to play archaeologist. It looks like something that would have had users, but it never did. It was never in production anywhere. Nobody ever submitted feedback, or a bug report, such that it was fixed or improved. There never existed a simpler version of it that several ex-maintainers understood perfectly; there are no such ex-maintainers and no such understanding. There is no documentation trail, or other historic trail if surrounding activity like discussions and negotiations which led to things being the way they are.

othmanosx 1mo ago

absolutley agree, but better understanding needs a better review surface that syncs with Github, which is why I created pyor.review, It's a blast compared to how I used to review code on Github and now I use it everyday.

Waterluvian 1mo ago

I love the idea that understanding is the new bottleneck. Because if we just ignore the potential horrors of cybernetic augments, it suggests the next challenge is how to teach things better. And that’s such a valuable thing to improve. I have a soft spot for when I find a teacher or textbook or interactive website that makes something click. I live for that click. I crave it. I crave seeing it happen in others. How optimistic I could be if understanding becomes the primary target.

areactnativedev 1mo ago
Would you have any good such resources for “clicking” or expanding one’s understanding to share? From the top of my head and of my Goodreads I have enjoyed The art of Electronics, Understanding Earth, Material World, The world for sale, Beej’s Guide to C, The Five Dysfunctions of a team, Financial Intelligence for Entrepreneurs, The Lean Startup, Fouché by S. Zweig https://ciechanow.ski/mechanical-watch/ https://ciechanow.ski/mechanical-watch/
- Waterluvian 1mo ago
  https://www.jezzamon.com/fourier/ https://www.jezzamon.com/fourier/ This one really made Fourier analysis click for me. Beyond that it made the idea of lossy compression (not just JPEGs) click for me.

beloch 1mo ago

To put it another way, AI is like a calculator or physics textbook. You can say, "I don't need to be able to do basic arithmetic in my head. I have a calculator!". Or, "I don't need to know how to solve this kind of problem. I have a textbook and I can look it up on demand!". Having to reach for a calculator constantly slows you down and makes simple equations hard, while also severely retarding your ability to do estimates and sanity checks. Not practicing on basic problems prevents you from developing the mental tools to solve more advanced problems, or being able to develop methods for solving novel problems. If almost anyone else could use your calculator and physics textbook to get similar results, what use are you? Some companies are pressuring their employees to let AI do everything without slowing down to gain understanding of what it's done. These are the companies that most people won't have a lot of use for in the near future.

Ozzie-D 1mo ago

[flagged]

throwatdem12311 1mo ago

Understanding was always the bottleneck. It’s just now the understanding is backloaded instead of frontloaded. ie. understanding it AFTER it’s already generated rather than before you wrote the code by hand

est 1mo ago

I found the linked article "books don't work" from the quiz part highly interesting https://andymatuschak.org/books/ https://andymatuschak.org/books/ It explains a lot and works really well. I tried out in ChatGPT with a simple prompt: > ...paste link... Give me series of quiz see if I really understood the article well. Ask & answer one by one in turns. Really fun experience.

augment_me 1mo ago
Thanks! Great read
zahlman 1mo ago
It's been submitted many times before: https://hn.algolia.com/?q=books+don%27t+work https://hn.algolia.com/?q=books+don%27t+work

nphardon 1mo ago

In the end, LLMs create garbage code that no one understands, they break things that should not have been broken, that would not have been broken if it was done slowly with understanding along the way. To reframe it as "understanding is the bottleneck" is just more LLM salesmanship. LLMs have their limits and when you hit them you're stuck. LLMs are the bottleneck. But the idea that "LLMs are the answer to the problem created by LLMs" is absurd.

equinumerous 1mo ago
I couldn't agree more. We really need more engineers to be vocal about how stupid these ideas are. "How about we throw out 30+ years of software engineering literature so we can 'move faster'?" What if the customers on the other end don't want new features, they just want software stability? If SQLite came out with a new LLM-written feature a week, would it be a better library? If you know what to build, writing software right the first time pays for itself over time. LLMs are still great, but more for rapid prototyping, researching, log diving, one-off scripts... Sorry, </rant>.
- champagnepapi 1mo ago
  Agreed!

mindhash 1mo ago

Absolutely. It's far easier if the thought originated in your own mind. The default long responses of LLMs don't make it easy.

nunez 1mo ago

Sorry but actually what. Flash cards? Games with micro worlds? To understand ones own codebase??? We figured this out 10, 20, 30 years ago. Small, atomic commits. Small PRs. Lots of manually written tests. Documentation, ideally with the PR. There is no "understanding" 60k loc highlander PRs. That's an entire feature. We had it so good once.

zahlman 1mo ago

"new"? The reason for all the old bottlenecks is that you had to understand as you went along.

chris_armstrong 1mo ago

A small bone to pick, but describing an AI that is operating autonomously as creative seems wrong - at best, this process is accretive, because the AI is adding and adding, but has no ability or incentive to shape its output toward something a human would find of value. Value is subjective (individual), changing over time. An AI doesn't know when it needs to be taking away - removal is a key part of the creative process.

simianwords 1mo ago
No incentive? Do you not know how RL is done?
- chris_armstrong 1mo ago
  Reinforced continuously toward a fixed goal is not an incentive, it doesn’t lead to a creative process

evalystai 1mo ago

[flagged]

ilitirit 1mo ago

I guess it depends on the context and environment, right? In many corporates, the bottleneck for me has always been specs and testing. I have daily examples. I had to explain it to leadership like this: "There are far more ways things can go wrong than right".

pSYoniK 1mo ago

I thought this was going in a different direction along some of the thoughts I had around LLMs for programming tasks specifically. While this talks about understanding and how to ensure you're keeping up with what is changing, I feel that this is maybe more aligned with how a PM/PO should understand the work being done and not necessarily how an engineer should. I took a note a few months ago and my point was I think more engineering specific, although it might be my own lacking abilities/skills that caused this realization. "Your capacity to learn/recall and map information is the new bottleneck. LLMs can act as learning amplifiers but correctness isn't as important for their output as critical thinking on the side of the consumer - YOU." My point is that, I think if an LLM outputs 50,000 lines of code, your ability to go through what has changed, how it has changed and where the changes have occurred is the bottleneck. I see the approaches here, sure, "summarize the changes" or "draw me a picture" or the more recently observed "build me a city building simulator to understand this", but I feel that misses the point from an engineering perspective. The difference in understanding the weeds such as DB transactional boundaries or tenant isolation (which I believe was a topic in a recent data leak), those aren't summarized that easily in drawings or if they are, if you are working at this granularity, then your 50,000 line PR will yield 50,000 pages of crayon drawings you now have to understand. I guess, my point is that understanding is the bottleneck, but low level understanding and the ability to read/map/connect is even more so. Any developer with some experience will agree that if changes are trivial you can scan and pick up mistakes or flaws easily. So most SOTA models won't necessarily even make these. So what you're reviewing now is going to be one level higher or more in terms of difficulty, mapping multiple components or touching multiple surfaces. Your ability to make the links, reason about them and attempt to find flaws or logic issues is the bottleneck. In the time it takes you to understand, another 50,000 line PR is up. I'm not sure how we're going to be solving this. I don't know if in the current state it is a solvable issue, maybe another 6 months? Maybe another 6 years? Maybe this is fine and we will settle in a sort of place where your mediocre engineer will be responsible for tens of reviews a day signing off on method/functions/classes/interfaces being added, get paid 50k a year and doing the same non-thinking work day in day out while signing their name to the quality of the code being shipped while a senior/lead will be busy reviewing multiple of these. Think of the way an assembly line functions. P.S. I hate to see this annoying tendency of transforming knowledge work into assembly line work. We keep trying to "fix" this without understanding what knowledge itself is. Maybe this technology will indeed yield software assembly lines, I don't wish to eat my words, but I'm still struggling to see how we will handle the nitty gritty of software work. Maybe the same way we handle building airplanes - as long as only a couple crash a year, we're sort of fine.

ubercore 1mo ago

Low value comment, so I apologize, but it's funny to note how many bottleneck articles there are now as a result of AI adoption. Lots of new bottlenecks.

saadyousfi 1mo ago

[dead]

dofm 1mo ago

> Why? Why understand? Because (in the just-barely-possible universe where anyone is stupid enough to make me a manager) I will fire you if I ever find out you don't. What possible good can come from letting people deploy things over which they do not, in any sense, have cognitive ownership?

hackmack10 1mo ago
I'd fire you for having that blanket attitude. People haven't truly known how shit works in this industry for the entire fucking time. People are following patterns set out by arrogant pricks for reasons they don't know or care to know, they are just following orders. That said, someone, somewhere in the chain should understand how the system works. I agree there.
- dofm 1mo ago
  > I'd fire you for having that blanket attitude. People haven't truly known how shit works in this industry for the entire fucking time. Oh it doesn't sound like I'd be working for you.

neprotivo 1mo ago

Improving code understanding is the main focus of my work and thinking right now. If we want to make advances I believe that we should rely more heavily on one key quality of the program code: It is meant to be executed. Here are some ideas: 1. Time travel debugging. Reading a PR just like a wall of text is difficult, but what if you could step through the PR and see the state at a given line for some test executions? Time travel debugging can make this possible. You would collect a debug trace and use it to overlay the PR diff with additional controls and information to resemble a debugger's UI. I was part of the team behind Codetracer (https://github.com/metacraft-labs/codetracer https://github.com/metacraft-labs/codetracer) who is trying to work in this direction. 2. Test suites and coverage. We don't use them enough for understanding right now. The test suite encodes what features the code is supposed to have, and the coverage tells us where in the code those features are implemented. I'm playing with an idea about this here: http://atlas.vihren.dev http://atlas.vihren.dev When we intersect coverages for the different test cases we can arrive at code segments which represent "atomic behaviors" present in the code. They form a mathematical structure which can be represented as a graph. I am currently exploring what value we can extract from it for the benefit of both humans and agents.

Kinrany 1mo ago
This seems backwards. We've known for a long time that successful execution is not enough for understanding.

bsenftner 1mo ago

Understanding has always been the bottleneck, everywhere for everything. And now, with this new realization, are you going to finally realize that Communications and your skills with it are basically everything?

gregwebs 1mo ago

If you start with a spec you understand at the beginning then you don't need the LLM to generate high-level information about the changes at review time. The grilling (grill-with-docs) skills [1] are amazing for ensuring you produce a through spec that covers all the edge cases. The /code-review skill from there helps ensure that the code changes meet the spec. I use an intermediate detailed plan stage (done by a more expensive model) before implementation. Information from that plan is posted on the PR to give pretty much all the intermediate level context reviewers need. I do like incorporating the idea of this article into my flow- that the spec and PR context could be presented in a more educational way. [1] https://github.com/mattpocock/skills https://github.com/mattpocock/skills

alex_suzuki 1mo ago

I think the ideas presented in the article are interesting, but at times the Notion references were a bit much. The author disclosed that they work there, fine, but at times it felt like it was more about showing off a Notion feature (e.g. embedded interactive HTML).

ontouchstart 1mo ago

Analysis and synthesis are two sides of human intelligence. Current form of generative artificial intelligence is very powerful in synthesizing information and producing artifacts, human will lose this battle. Understanding is about analyzing and we still have some tools to help us, such as type checking, testing, etc. To some extent this can be automated but needs to be maintained automatically to match the flood of synthetic artifacts. “Synthetic analysis” is an oxymoron and could lead to hallucinations and irrelevance.

penguin_booze 1mo ago

> Aren't we supposed to be taking ourselves out of the loop now, and letting the agents loop themselves That's what Boris Cherney said. Boris Cherney is not your dad. You don't have to listen to him.

OlegK-Dev 1mo ago

[flagged]