6 ms·

I've yet to be convinced by any article, including this one, that attempts to draw boxes around what coding agents are and aren't good at in a way that is robus

I've yet to be convinced by any article, including this one, that attempts to draw boxes around what coding agents are and aren't good at in a way that is robust on a 6 to 12 month horizon. I agree that the examples listed here are relatable, and I've seen similar in my uses of various coding harnesses, including, to some degree, ones driven by opus 4.5. But my general experience with using LLMs for development over the last few years has been that: 1. Initially models could at best assemble a simple procedural or compositional sequences of commands or functions to accomplish a basic goal, perhaps meeting tests or type checking, but with no overall coherence, 2. To being able to structure small functions reasonably, 3. To being able to structure large functions reasonably, 4. To being able to structure medium-sized files reasonably, 5. To being able to structure large files, and small multi-file subsystems, somewhat reasonably. So the idea that they are now falling down on the multi-module or multi-file or multi-microservice level is both not particularly surprising to me and also both not particularly indicative of future performance. There is a hierarchy of scales at which abstraction can be applied, and it seems plausible to me that the march of capability improvement is a continuous push upwards in the scale at which agents can reasonably abstract code. Alternatively, there could be that there is a legitimate discontinuity here, at which anything resembling current approaches will max out, but I don't see strong evidence for it here.

groby_b 8mo ago
LLMs are bad at creating abstraction boundaries since inception. People have been calling it out since inception. (Heck, even I got a twitter post somewhere >12 months old calling that out, and I'm not exactly a leading light of the effort) It is in no way size-related. The technology cannot create new concepts/abstractions, and so fails at abstraction. Reliably.
- w0m 8mo ago
  I believe his argument is that now that you've defined the limitation, it's a ceiling that will likely be cracked in the relatively near future.
  - emp17344 8mo ago
    Well, hallucinations have been identified as an issue since the inception of LLMs, so this doesn’t appear true.
    - johnfn 8mo ago
      Hallucinations are more or less a solved problem for me ever since I made a simple harness to have Codex/Claude check its work by using static typechecking.
      - emp17344 8mo ago
        But there aren’t very many domains where this type of verification is even possible.
        nextaccountic 8mo ago
        Then you apply LLMs in domains where things can be checked Indeed I expect to see a huge push into formally verified software just because sound mathematical proofs provide an excellent verifier to put into a LLM hardness. Just see how Aristotle has been successful at math, and it could be applied to coding too Maybe Lean will become the new Python https://harmonic.fun/news#blog-post-verina-bench-sota https://harmonic.fun/news#blog-post-verina-bench-sota
        filoeleven 8mo ago
        "LLMs reliably fail at abstraction." "This limitation will go away soon." "Hallucinations haven't." "I found a workaround for that." "That doesn't work for most things." "Then don't use LLMs for most things."
        baq 8mo ago
        Um, yes? Except ‘most things’ are not much at all by volume.
- TeMPOraL 8mo ago
  > The technology cannot create new concepts/abstractions, and so fails at abstraction. Reliably. That statement is way too strong, as it implies either that humans cannot create new concepts/abstractions, or that magic exists.
  - atty 8mo ago
    I think both your statement and their statement are too strong. There is no reason to think LLMs can do everything a human can do, which seems to be your implication. On the other hand, the technology is still improving, so maybe it’ll get there.
    - TeMPOraL 8mo ago
      My take is that: 1) LLMs cannot do everything humans can, but 2) There's no fundamental reason preventing some future technology to do everything humans can, and 3) LLMs are explicitly designed and trained to mimic human capabilities in fully general sense. Point 2) is the "or else magic exists" bit; point 3) says you need a more specific reason to justify assertion that LLMs can't create new concepts/abstractions, given that they're trained in order to achieve just that. Note: I read OP as saying they fundamentally can't and thus never will. If they meant just that the current breed can't, I'm not going to dispute it.
      - Jensson 8mo ago
        > 3) LLMs are explicitly designed and trained to mimic human capabilities in fully general sense. This is wrong, LLM are trained to mimic human writing not to mimic human capabilities. Writing is just the end result not the inner workings of a human, most of what we do happens before we write it down. You could argue you think that writing captures everything about humans, but that is another belief you have to add to your takes. So first that LLM are explicitly designed to mimic human writing, and then that human writing captures human capabilities in a fully general sense.
        TeMPOraL 8mo ago
- 131hn 8mo ago
  There’s only one way to implement a mission, an algorithm, a task. But there’s an infinity of path, inconsistants, fuzzy and always subjective way to live. Thàt’s our lives, that’s the code LLM are trained on. I do not think, and hope, it will ever change much
Uehreka 8mo ago
It feels like a lot of people keep falling into the trap of thinking we’ve hit a plateau, and that they can shift from “aggressively explore and learn the thing” mode to “teach people solid facts” mode. A week ago Scott Hanselman went on the Stack Overflow podcast to talk about AI-assisted coding. I generally respect that guy a lot, so I tuned in and… well it was kind of jarring. The dude kept saying things in this really confident and didactic (teacherly) tone that were months out of date. In particular I recall him making the “You’re absolutely right!” joke and asserting that LLMs are generally very sycophantic, and I was like “Ah, I guess he’s still on Claude Code and hasn’t tried Codex with GPT 5”. I haven’t heard an LLM say anything like that since October, and in general I find GPT 5.x to actually be a huge breakthrough in terms of asserting itself when I’m wrong and not flattering my every decision. But that news (which would probably be really valuable to many people listening) wasn’t mentioned on the podcast I guess because neither of the guys had tried Codex recently. And I can’t say I blame them: It’s really tough to keep up with all the changes but also spend enough time in one place to learn anything deeply. But I think a lot of people who are used to “playing the teacher role” may need to eat a slice of humble pie and get used to speaking in uncertain terms until such a time as this all starts to slow down.
- MoltenMan 8mo ago
  I agree with a lot of what you've said, but I completely disagree that LLM's are no longer sycophantic. GPT-5 is definitely still very sycophantic, 'You're absolutely right!' still happens, etc. It's true it happens far less in a pure coding context (Claude Code / Codex) but I suspect only because of the system prompts, and those tools are by far in the minority of LLM usage. I think it's enlightening to open up ChatGPT on the web with no custom instructions and just send a regular request and see the way it responds.
- orbital-decay 8mo ago
  > in general I find GPT 5.x to actually be a huge breakthrough in terms of asserting itself when I’m wrong That's just a different bias purposefully baked into GPT-5's engineered personality on post-training. It always tries to contradict the user, including the cases where it's confidently wrong, and keeps justifying the wrong result in a funny manner if pressed or argued with (as in, it would have never made that obvious mistake if it wasn't bickering with the user). GPT-5.0 in particular was extremely strongly finetuned to do this. And in longer replies or multiturn convos, it falls into a loop on contradictory behavior far too easily. This is no better than sycophancy. LLMs need an order of magnitude better nuance/calibration/training, this requires human involvement and scales poorly. Fundamental LLM phenomena (ICL, repetition, serial position biases, consequences of RL-based reasoning etc) haven't really changed, and they're worth studying for a layman to get some intuition. However, they vary a lot model to model due to subtle architectural and training differences, and impossible to keep up because there are so many models and so few benchmarks that measure these phenomena.
danpalmer 8mo ago
I used to get made up APIs in functions, now I get them in modules. I used to get confidently incorrect assertions in files now I get them across codebases. Hell, I get poorly defined APIs across files and still get them between functions. LLMs aren't good at writing well defined APIs at any level of the stack. They can attempt it at levels of the stack they couldn't a year ago, but they're still terrible at it unless the problem is so well known enough that they can regurgitate well reviewed code.
- refactor_master 8mo ago
  I still get made-up Python types all the time with Gemini. Really quite distracting when your codebase is massive and triggers a type error, and Gemini says "To solve it you just need to use WrongType[ThisCannotBeUsedHere[Object]]" and then I spend 15 minutes running in circles, because everything from there on is just a downward spiral, until I shut off the AI noise and just read the docs.
  - baq 8mo ago
    Gemini unfortunately sucks at calling tools, including ‘read the docs’ tool… it’s a great model otherwise. I’m sure Hassabis’ team is on it since it’s how the model can ground itself in non-coding contexts, too.
- measurablefunc 8mo ago
  This is the right answer. Unless there is some equivalent of it on the open internet which their search engine can find you should not expect a good outcome.
  - danpalmer 8mo ago
    "good outcome" is pretty subjective, I do get useful productivity gains from some LLM work, but the issues are the same as they always have been.
    - measurablefunc 8mo ago
skybrian 8mo ago
The article is mostly reporting on the present. (Note the "yet" in the title.) There's only one sentence where it handwaves about the future. I do think that line should have been cut.
pankajdoharey 8mo ago
Claude is overrated premium piece of developer tech, i have produced equally good results from Gemini and Way better with GPT - medium. And GPT Medium is a really good model at assembling and debugging stuff than Claude. Claude hallucinates when asked why something is correct or should be done. All Models fail equally in some or the other aspect, which point to the fact that these models have strength's and weaknesses, and GPT just happens to be a good overall model. But dev community is so stuck up on Claude for no good reason other than shiny tooling : "Claude Code", besides that the models can be equally worse as the competition. The Benchmarks do not explain the full story. In general though the Thumb rule is if the Model says you are Brilliant, Thats genius or Now thats a deep and insightful question you asked... Its time to start a new session.
wouldbecouldbe 8mo ago
I feel like the main challenge is where to be "loose" and where to be "strict", Claude takes too much liberty often. Assuming things, adding some mock data to make it work, using local storage because there is no db. This makes it work well out of the box, and means I can prompt half ass and have great results. But it also long term causes issues. It can be prompted away, but it needs constant reminder. This seems like a hard problem to solve. I feel like it can already almost do everything if you have the correct vision / structure in mind and have the patience to prompt properly. It's worst feature is debugging hard errors, it will just keep trying everything and can get pretty wild instead of entering plan mode and really discuss & think things true.

groby_b 8mo ago

LLMs are bad at creating abstraction boundaries since inception. People have been calling it out since inception. (Heck, even I got a twitter post somewhere >12 months old calling that out, and I'm not exactly a leading light of the effort) It is in no way size-related. The technology cannot create new concepts/abstractions, and so fails at abstraction. Reliably.

w0m 8mo ago
I believe his argument is that now that you've defined the limitation, it's a ceiling that will likely be cracked in the relatively near future.
- emp17344 8mo ago
  Well, hallucinations have been identified as an issue since the inception of LLMs, so this doesn’t appear true.
  - johnfn 8mo ago
    Hallucinations are more or less a solved problem for me ever since I made a simple harness to have Codex/Claude check its work by using static typechecking.
    - emp17344 8mo ago
      But there aren’t very many domains where this type of verification is even possible.
      - nextaccountic 8mo ago
        Then you apply LLMs in domains where things can be checked Indeed I expect to see a huge push into formally verified software just because sound mathematical proofs provide an excellent verifier to put into a LLM hardness. Just see how Aristotle has been successful at math, and it could be applied to coding too Maybe Lean will become the new Python https://harmonic.fun/news#blog-post-verina-bench-sota https://harmonic.fun/news#blog-post-verina-bench-sota
        filoeleven 8mo ago
        "LLMs reliably fail at abstraction." "This limitation will go away soon." "Hallucinations haven't." "I found a workaround for that." "That doesn't work for most things." "Then don't use LLMs for most things."
        baq 8mo ago
        Um, yes? Except ‘most things’ are not much at all by volume.
TeMPOraL 8mo ago
> The technology cannot create new concepts/abstractions, and so fails at abstraction. Reliably. That statement is way too strong, as it implies either that humans cannot create new concepts/abstractions, or that magic exists.
- atty 8mo ago
  I think both your statement and their statement are too strong. There is no reason to think LLMs can do everything a human can do, which seems to be your implication. On the other hand, the technology is still improving, so maybe it’ll get there.
  - TeMPOraL 8mo ago
    My take is that: 1) LLMs cannot do everything humans can, but 2) There's no fundamental reason preventing some future technology to do everything humans can, and 3) LLMs are explicitly designed and trained to mimic human capabilities in fully general sense. Point 2) is the "or else magic exists" bit; point 3) says you need a more specific reason to justify assertion that LLMs can't create new concepts/abstractions, given that they're trained in order to achieve just that. Note: I read OP as saying they fundamentally can't and thus never will. If they meant just that the current breed can't, I'm not going to dispute it.
    - Jensson 8mo ago
      > 3) LLMs are explicitly designed and trained to mimic human capabilities in fully general sense. This is wrong, LLM are trained to mimic human writing not to mimic human capabilities. Writing is just the end result not the inner workings of a human, most of what we do happens before we write it down. You could argue you think that writing captures everything about humans, but that is another belief you have to add to your takes. So first that LLM are explicitly designed to mimic human writing, and then that human writing captures human capabilities in a fully general sense.
      - TeMPOraL 8mo ago
131hn 8mo ago
There’s only one way to implement a mission, an algorithm, a task. But there’s an infinity of path, inconsistants, fuzzy and always subjective way to live. Thàt’s our lives, that’s the code LLM are trained on. I do not think, and hope, it will ever change much

Uehreka 8mo ago

It feels like a lot of people keep falling into the trap of thinking we’ve hit a plateau, and that they can shift from “aggressively explore and learn the thing” mode to “teach people solid facts” mode. A week ago Scott Hanselman went on the Stack Overflow podcast to talk about AI-assisted coding. I generally respect that guy a lot, so I tuned in and… well it was kind of jarring. The dude kept saying things in this really confident and didactic (teacherly) tone that were months out of date. In particular I recall him making the “You’re absolutely right!” joke and asserting that LLMs are generally very sycophantic, and I was like “Ah, I guess he’s still on Claude Code and hasn’t tried Codex with GPT 5”. I haven’t heard an LLM say anything like that since October, and in general I find GPT 5.x to actually be a huge breakthrough in terms of asserting itself when I’m wrong and not flattering my every decision. But that news (which would probably be really valuable to many people listening) wasn’t mentioned on the podcast I guess because neither of the guys had tried Codex recently. And I can’t say I blame them: It’s really tough to keep up with all the changes but also spend enough time in one place to learn anything deeply. But I think a lot of people who are used to “playing the teacher role” may need to eat a slice of humble pie and get used to speaking in uncertain terms until such a time as this all starts to slow down.

MoltenMan 8mo ago
I agree with a lot of what you've said, but I completely disagree that LLM's are no longer sycophantic. GPT-5 is definitely still very sycophantic, 'You're absolutely right!' still happens, etc. It's true it happens far less in a pure coding context (Claude Code / Codex) but I suspect only because of the system prompts, and those tools are by far in the minority of LLM usage. I think it's enlightening to open up ChatGPT on the web with no custom instructions and just send a regular request and see the way it responds.
orbital-decay 8mo ago
> in general I find GPT 5.x to actually be a huge breakthrough in terms of asserting itself when I’m wrong That's just a different bias purposefully baked into GPT-5's engineered personality on post-training. It always tries to contradict the user, including the cases where it's confidently wrong, and keeps justifying the wrong result in a funny manner if pressed or argued with (as in, it would have never made that obvious mistake if it wasn't bickering with the user). GPT-5.0 in particular was extremely strongly finetuned to do this. And in longer replies or multiturn convos, it falls into a loop on contradictory behavior far too easily. This is no better than sycophancy. LLMs need an order of magnitude better nuance/calibration/training, this requires human involvement and scales poorly. Fundamental LLM phenomena (ICL, repetition, serial position biases, consequences of RL-based reasoning etc) haven't really changed, and they're worth studying for a layman to get some intuition. However, they vary a lot model to model due to subtle architectural and training differences, and impossible to keep up because there are so many models and so few benchmarks that measure these phenomena.

danpalmer 8mo ago

I used to get made up APIs in functions, now I get them in modules. I used to get confidently incorrect assertions in files now I get them across codebases. Hell, I get poorly defined APIs across files and still get them between functions. LLMs aren't good at writing well defined APIs at any level of the stack. They can attempt it at levels of the stack they couldn't a year ago, but they're still terrible at it unless the problem is so well known enough that they can regurgitate well reviewed code.

refactor_master 8mo ago
I still get made-up Python types all the time with Gemini. Really quite distracting when your codebase is massive and triggers a type error, and Gemini says "To solve it you just need to use WrongType[ThisCannotBeUsedHere[Object]]" and then I spend 15 minutes running in circles, because everything from there on is just a downward spiral, until I shut off the AI noise and just read the docs.
- baq 8mo ago
  Gemini unfortunately sucks at calling tools, including ‘read the docs’ tool… it’s a great model otherwise. I’m sure Hassabis’ team is on it since it’s how the model can ground itself in non-coding contexts, too.
measurablefunc 8mo ago
This is the right answer. Unless there is some equivalent of it on the open internet which their search engine can find you should not expect a good outcome.
- danpalmer 8mo ago
  "good outcome" is pretty subjective, I do get useful productivity gains from some LLM work, but the issues are the same as they always have been.
  - measurablefunc 8mo ago

skybrian 8mo ago

The article is mostly reporting on the present. (Note the "yet" in the title.) There's only one sentence where it handwaves about the future. I do think that line should have been cut.

pankajdoharey 8mo ago

Claude is overrated premium piece of developer tech, i have produced equally good results from Gemini and Way better with GPT - medium. And GPT Medium is a really good model at assembling and debugging stuff than Claude. Claude hallucinates when asked why something is correct or should be done. All Models fail equally in some or the other aspect, which point to the fact that these models have strength's and weaknesses, and GPT just happens to be a good overall model. But dev community is so stuck up on Claude for no good reason other than shiny tooling : "Claude Code", besides that the models can be equally worse as the competition. The Benchmarks do not explain the full story. In general though the Thumb rule is if the Model says you are Brilliant, Thats genius or Now thats a deep and insightful question you asked... Its time to start a new session.

wouldbecouldbe 8mo ago

I feel like the main challenge is where to be "loose" and where to be "strict", Claude takes too much liberty often. Assuming things, adding some mock data to make it work, using local storage because there is no db. This makes it work well out of the box, and means I can prompt half ass and have great results. But it also long term causes issues. It can be prompted away, but it needs constant reminder. This seems like a hard problem to solve. I feel like it can already almost do everything if you have the correct vision / structure in mind and have the patience to prompt properly. It's worst feature is debugging hard errors, it will just keep trying everything and can get pretty wild instead of entering plan mode and really discuss & think things true.