11 ms·

> We don't just keep adding more words to our context window, because it would drive us mad. That, and we also don't only focus on the textual description of a

> We don't just keep adding more words to our context window, because it would drive us mad. That, and we also don't only focus on the textual description of a problem when we encounter a problem. We don't see the debugger output and go "how do I make this bad output go away?!?". Oh, I am getting an authentication error. Well, meaybe I should just delete the token check for that code path...problem solved?! No. Problem very much not-solved. In fact, problem very much very bigger big problem now, and [Grug][1] find himself reaching for club again. Software engineers are able to step back, think about the whole thing, and determine the root cause of a problem. I am getting an auth error...ok, what happens when the token is verified...oh, look, the problem is not the authentication at all...in fact there is no error! The test was simply bad and tried to call a higher privilege function as a lower privilege user. So, test needs to be fixed. And also, even though it isn't per-se an error, the response for that function should maybe differentiate between "401 because you didn't authenticate" and "401 because your privileges are too low". [1]: https://grugbrain.dev https://grugbrain.dev

trod1234 1y ago
Isn't the 401 for LLMs the same single undecidable token? Doesn't this basically go to the undecidable nature of math in CS? Put another way, you have an excel roster corresponding to people with accounts where some need to have their account shutdown but you only have their first and last names as identifiers, and the pool is sufficiently large that there are more than one person per a given set of names. You can't shut down all accounts with a given name, and there is no unique identifier. How do you solve this? You have to ask and be given that unique identifier that differentiates between the undecidable. Without that, even the person can't do the task. The person can make guesses, but those guesses are just hallucinations with a significant n probability towards a bad repeat outcome. At a core level I don't think these type of issues are going to be solved. Quite a lot of people would be unable to solve this and struggle with this example (when not given the answer, or hinted at the solution in the framing of the task; ie when they just have a list of names and are told to do an impossible task).
livid-neuro 1y ago
The first cars broke down all the time. They had a limited range. There wasn't a vast supply of parts for them. There wasn't a vast industry of experts who could work on them. There wasn't a vast network of fuel stations to provide energy for them. The horse was a proven method. What an LLM cannot do today is almost irrelevant in the tide of change upon the industry. The fact is, with improvements, it doesn't mean an LLM cannot do it tomorrow.
- skydhash 1y ago
  When the first cars broke down, people were not saying: One day, we’ll go to the moon with one of these. LLMs may get better, but it will not be what people are clamoring them to be.
  - deleted 1y ago
    [deleted]
  - serf 1y ago
    >When the first cars broke down, people were not saying: One day, we’ll go to the moon with one of these. maybe they should have; a lot of the engineering techniques and methodologies that produced the assembly line and the mass produced vehicle also lead the way into space exploration.
- jedimastert 1y ago
  > The first cars broke down all the time. They had a limited range. There wasn't a vast supply of parts for them. There wasn't a vast industry of experts who could work on them. I mean, there was and then there wasn't. All of those things are shrinking fast because we handed over control to people who care more about profits than customers because we got too comfy and too cheap, and now right to repair is screwed. Honestly, I see llm-driven development as a threat to open source and right to repair, among the litany of other things
- Night_Thastus 1y ago
  The difference is that the weaknesses of cars were problems of engineering, and some of infrastructure. Both aren't very hard to solve, though they take time. The fundamental way cars operated worked and just needed revision, sanding off rough edges. LLMs are not like this. The fundamental way they operate, the core of their design is faulty. They don't understand rules or knowledge. They can't, despite marketing, really reason. They can't learn with each interaction. They don't understand what they write. All they do is spit out the most likely text to follow some other text based on probability. For casual discussion about well-written topics, that's more than good enough. But for unique problems in a non-English language, it struggles. It always will. It doesn't matter how big you make the model. They're great for writing boilerplate that has been written a million times with different variations - which can save programmers a LOT of time. The moment you hand them anything more complex it's asking for disaster.
skydhash 1y ago
Programmers are mostly translating business rules to the very formal process execution of the computer world. And you need to both knows what the rules means and how the computer works (or at least how the abstracted version you’re working with works). The translation is messy at first, which is why you need to revise it again and again. Especially when later rules comes challenging all the assumptions you’ve made or even contradicting themselves. Even translations between human languages (which allows for ambiguity) can be messy. Imagine if the target language is for a system that will exactly do as told unless someone has qualified those actions as bad.
- physicsguy 1y ago
  Yes although many software engineers try as hard as possible to avoid learning what the business problem is. In my experience though those people never make great engineers.
  - trimbybj 1y ago
    Often those of us that do want to learn what the business problem is are not allowed to be involved in those discussions, for various reasons. Sometimes it's "Oh we can take care of that so you don't have to deal with it," and sometimes it's "Just build to this design/spec" and they're not used to engineers (the good ones) questioning things.
    - marcus_holmes 1y ago
      "Just shut up and push the nerd-buttons, nerd." I went and got an MBA to try and get around this. It didn't work.
      - lisbbb 1y ago
        I had a professor in grad school, Computer Engineering, that begged me not to get an MBA--he had worked in industry, particularly defense, and had a very low opinion of MBAs. I tend to agree nowadays. I really think the cookie-cutter "safe" approach that MBA types take, along with them maximizing profits using data science tools, has made the USA a worse place overall.
chuckadams 1y ago
An AI might tell you to use a 403 for insufficient privileges instead of 401.
- dade_ 1y ago
  You are absolutely right, let me fix that.
  - fransje26 1y ago
    What a great suggestion.
throwaway1004 1y ago
That reference link is a wild ride of unqualified, cartoonish passive-aggression, the cute link to the author's "swag" is the icing on the cake. Concidentally, I encountered the author's work for the first time only a couple of days ago as a podcast guest, he vouches for the "Dirty Code" approach while straw-manning Uncle Bob's general principles of balancing terseness/efficiency with ergonomics and readability (in most, but not all, cases). I guess this stuff sells t-shirts and mugs /rant
- pphysch 1y ago
  > big brained developers are many, and some not expected to like this, make sour face
- Arainach 1y ago
  >Uncle Bob's general principles of balancing terseness/efficiency with ergonomics and readability (in most, but not all, cases). Have you read Uncle Bob? There's no need to strawman: Bob's examples in Clean Code are absolutely nuts. Here's a nice writeup that includes one of Bob's examples verbatim in case you've forgotten: https://qntm.org/clean https://qntm.org/clean Here's another: https://gerlacdt.github.io/blog/posts/clean_code/ https://gerlacdt.github.io/blog/posts/clean_code/
  - the__alchemist 1y ago
    Uncle Bob's rules: IMO do the opposite of what they say. They're a reasonable set if negated!
  - throwaway1004 1y ago
    >Have you read Uncle Bob? Yes, I have read Uncle Bob. I could agree that the examples in the book leave room for improvement. Meanwhile, the real-world application of these principles and trial-and-error, collectively within my industry, yields a more accurate picture of it's usefulness. Even the most click-bait'y criticisms (such as the author I referenced above) involve zooming in on it's most-controversial aspects, in a vacuum, without addressing the core principles and how they're completely necessary for delivering software at scale, warranting it's status as a seminal work. "...for the obedience of fools, and the guidance of wise men", indeed! edit - it's the same arc as Agile has endured: 1. a good-faith argument for a better way of doing things is recognised and popularised. 2. It's abused and misused by bad actors/incompetents for years (who would not have done better using a different process) 3. Jaded/opportunistic talking heads tell us it's all garbage while simultaneously explaining that "well, it would be great if it wasn't applied poorly..."
reactordev 1y ago
While I agree with you - The whole grug brain thing is offensive. Because we have all been grug at some point.
- recursive 1y ago
  How does that make it offensive? To me, that makes it relatable.
- WhyOhWhyQ 1y ago
  This seems to miss the point. Being Grug is the endgame.
  - lcnPylGDnU4H9OF 1y ago
    > big brained developers are many, and some not expected to like this, make sour face > THINK they are big brained developers many, many more, and more even definitely probably maybe not like this, many sour face (such is internet) > (note: grug once think big brained but learn hard way)
    - reactordev 1y ago
      It just reads like they had a stroke and can no longer function.
      - recursive 1y ago
        I guess it's not for everyone. It makes sense to me. shrug
- meindnoch 1y ago
  Midwit take. Grug is both the high and low end of the Bell curve.
- lioeters 1y ago
  Grug is the wise fool in the spirit of Lao Tzu, St. Francis, and Diogenes. If you find it offensive, that's the intellectual pride it's meant to make fun of.
ai-christianson 1y ago
I take a more pragmatic approach --everything is human in the loop. It helps me get the job done faster and with higher quality, so I use it.
appease7727 1y ago
The way it works for me at least is I can fit a huge amount of context in my head. This works because the text is utterly irrelevant and gets discarded immediately. Instead, my brain parses code into something like an AST which then is represented as a spatial graph. I model the program as a logical structure instead of a textual one. When you look past the language, you can work on the program. The two are utterly disjoint. I think LLMs fail at software because they're focused on text and can't build a mental model of the program logic. It take a huge amount of effort and brainpower to truly architect something and understand large swathes of the system. LLMs just don't have that type of abstract reasoning.
- taminka 1y ago
  i wonder why nobody bothered w/ feeding llms the ast instead (not sure in what format), but it only seems logical, since that's how compilers undestand code after all...
  - NitpickLawyer 1y ago
    There are various efforts on this, from many teams. There's AST dump, AST-based graphs, GraphRAG w/ AST grounding, embeddings based AST trimming, search based AST trimming, ctags, and so on. We're still in the exploration space, and "best practices" are still being discovered. It's funny that everyone says that "LLMs" have plateaued, yet the base models have caught up with early attempts to build harnesses with the things I've mentioned above. They now match or exceed the previous generation software glue, with just "tools", even with limited ones like just "terminal".
- starlust2 1y ago
  It's not that they can't build a mental model, it's that they don't attempt to build one. LLMs jump straight from text to code with little to no time spent trying to architect the system.
fragmede 1y ago
If you can't get the LLM to generate code that handles an error code, that's on you. Yeah, sometimes it does dumb shit. Who cares? Just /undo and retry. Stop using Claude Code, which uses git like an intern. (Which is to say, it doesn't unless forced to.)
JackFr 1y ago
- When we have a report of a failing test before fixing it, identify the component under test. Think deeply about the component and describe its purpose, the control flows and state changes that occur within the component and assumptions the component makes about context. Write that analysis in file called component-name-mental-model.md. - When ever you address a failing test, always bring your component mental model into the context. Paste that into your Claude prompt and see if you get better results. You'll even be able to read and correct the LLM's mental model.
- siddboots 1y ago
  In my experience, complicated rules like this are extremely unreliable. Claude just ignores it much of the time. The problem is that when Claude sees a failing test it is usually just an obstacle to completing some other task at hand - it essentially never chooses to branch out into some new complicated workflow and instead will find some other low friction solution. This is exactly why subagents are effective: if Claude knows to always run tests via a testing subagent, then the specific testing workflow can become that subagent’s whole objective.
- fmbb 1y ago
  Anthropic sells this thing called Claude Code, but their customers have to train it to know how to be a programmer? Junior developers not even out of school don’t need to be instructed to think.
  - JackFr 1y ago
    > Junior developers not even out of school don’t need to be instructed to think. Have you trained juniors lately?
exe34 1y ago
to be fair, I've seen cursor step back and check higher level things. I was trying to set up a firecracker vm and it did everything for me, and when things didn't initially work, it started doing things like ls, tar -tvf, and then a bunch of checking networking stuff to make sure things were showing up in the right place. so current LLMs might not quite be human level, but I'd have to see a bigger model fail before I'd conclude that it can't do $X.
Natsu 1y ago
> the response for that function should maybe differentiate between "401 because you didn't authenticate" and "401 because your privileges are too low". I'd tend to think it more proper if it were 401 you didn't authenticate and 403 you're forbidden from doing that with those user rights, but you have to be careful about exactly how detailed your messages are, lest they get tagged as a CWE-209 in your next security audit.
theonething 1y ago
> Oh, I am getting an authentication error. Well, meaybe I should just delete the token check for that code path...problem solved?! Kind of hyperbolic. If you prompt well, generally, it won't do stupid to that extreme.
joshwarwick15 1y ago
https://blog.remote-mcp.com/p/the-grug-brained-ai-developer-an-e3a https://blog.remote-mcp.com/p/the-grug-brained-ai-developer-...
- lastdong 1y ago
  https://zed.dev/docs/ai/rules https://zed.dev/docs/ai/rules
jeffreygoesto 1y ago
Experience adds both additional layers vertically and domain knowledge horizontally and at some point that creates non-linear benefits, because you can transfer between problems and more importantly solutions of different fields. The context window is only one layer. Now go, researchers!
ruslan_sure 1y ago
I don't think it's helpful to put words in the LLM's mouth. To properly think about that, we need to describe how an LLM thinks. It doesn't think in words or move vague, unwieldy concepts around and then translate them into words, like humans do. It works with words (tokens) and their probability of appearing next. The main thing is that these probabilities represent the "thinking" that was initially behind the sentences with such words in its training set, so it manipulates words with the meaning behind them. Now, to your points: 1) Regarding adding more words to the context window, it's not about "more"; it's about "enough." If you don't have enough context for your task, how will you accomplish it? "Go there, I don't know where." 2) Regarding "problem solved," if the LLM suggests or does such a thing, it only means that, given the current context, this is how the average developer would solve the issue. So it's not an intelligence issue; it's a context and training set issue! When you write that "software engineers can step back, think about the whole thing, and determine the root cause of a problem," notice that you're actually referring to context. If the you don't have enough context or a tool to add data, no developer (digital or analog) will be able to complete the task.
- adastra22 1y ago
  > It doesn't think in words or move vague, unwieldy concepts around and then translate them into words, like humans do. That seems to me like a perfectly fine description of state space & chain of though continuation.
TacticalCoder 1y ago
[dead]
mike-cardwell 1y ago
> We don't see the debugger output and go "how do I make this bad output go away?!?" In the past, I've worked with developers that do. You ask them to investigate and deal with an error message, and all they do is whatever makes the error go away. Oh, a null pointer exception is thrown? Lets wrap it in a try/catch and move on.
- tyleo 1y ago
  Agreed. I’d argue there can be a time and place for it but reaching for it as the default tool on the shelf is a hallmark of incompetence.
- blauditore 1y ago
  Or google the error message, click first link, blindly copy-paste whatever code snippet comes up and re-run code/test. Yes, such workflows (jobs or) may become obsolete with some of the modern AI tools. Is that a bad thing? Not sure...
- uoaei 1y ago
  Hiring bad engineers will always be a problem, even (especially) in the age of LLMs.
- pc86 1y ago
  If you haven't worked with someone who you honestly think may have a severe learning disability or outright brain damage, you just haven't been working professionally for long enough.
  - coldtea 1y ago
    And now we can automate severe learning disability or outright brain damage with LLMs! Yay!
- rafaelmn 1y ago
  Thankfully those developers are now interchangeable with LLM agents.
jeswin 1y ago
> Oh, I am getting an authentication error. Well, meaybe I should just delete the token check for that code path...problem solved?! If this is how you think LLMs and Coding Agents are going about writing code, you haven't been using the right tools. Things happen, sure, but also mostly don't. Nobody is arguing that LLM-written code should be pushed directly into production, or that they'll solve every task. LLMs are tools, and everyone eventually figures out a process that works best for them. For me, it was strongs specs/docs, strict types, and lots of tests. And then of course the reviews if it's serious work.
- hellcow 1y ago
  Lately Claude has said, “this is getting complicated, let me delete $big_file_it_didnt_write to get the build passing and start over.” No, don’t delete the file. “You’re absolutely right…” And the moment the context is compacted, it forgets this instruction “fix the problems, don’t delete the file,” and tries to delete it again. I need to watch it like a hawk.
- suriya-ganesh 1y ago
  I can confirm this is exactly how llms are working. Spent two hours trying to get an llm to implement a filescan skip a specific directory. Tried, claude code, Gemini and cursor. All agents debugged and wrote code that just doesn't make sense. Llms are really good at template tasks, writing tests, boilerplate etc. But, Most times I'm not doing implement this button. I'm doing there's a logic mismatch in my expectation
  - jeswin 1y ago
    > Spent two hours trying to get an llm to implement a filescan skip a specific directory There's a large variance in outcomes depending on the prompt, and the process. I've gotten it to do things which are harder than a filescan with a skipped directory - without too much trouble. Add: > Llms are really good at template tasks, writing tests, boilerplate etc. If I have to stretch the definition of boilerplate to what's at the edge of a modern LLM's comprehension, I would say that 50% of software is some sort of boilerplate.
cncjchsue7 1y ago
Literally had a situation where cursor decided the best course of action in the situation was to delete the authentication test to solve the problem.

trod1234 1y ago

Isn't the 401 for LLMs the same single undecidable token? Doesn't this basically go to the undecidable nature of math in CS? Put another way, you have an excel roster corresponding to people with accounts where some need to have their account shutdown but you only have their first and last names as identifiers, and the pool is sufficiently large that there are more than one person per a given set of names. You can't shut down all accounts with a given name, and there is no unique identifier. How do you solve this? You have to ask and be given that unique identifier that differentiates between the undecidable. Without that, even the person can't do the task. The person can make guesses, but those guesses are just hallucinations with a significant n probability towards a bad repeat outcome. At a core level I don't think these type of issues are going to be solved. Quite a lot of people would be unable to solve this and struggle with this example (when not given the answer, or hinted at the solution in the framing of the task; ie when they just have a list of names and are told to do an impossible task).

livid-neuro 1y ago

The first cars broke down all the time. They had a limited range. There wasn't a vast supply of parts for them. There wasn't a vast industry of experts who could work on them. There wasn't a vast network of fuel stations to provide energy for them. The horse was a proven method. What an LLM cannot do today is almost irrelevant in the tide of change upon the industry. The fact is, with improvements, it doesn't mean an LLM cannot do it tomorrow.

skydhash 1y ago
When the first cars broke down, people were not saying: One day, we’ll go to the moon with one of these. LLMs may get better, but it will not be what people are clamoring them to be.
- deleted 1y ago
  [deleted]
- serf 1y ago
  >When the first cars broke down, people were not saying: One day, we’ll go to the moon with one of these. maybe they should have; a lot of the engineering techniques and methodologies that produced the assembly line and the mass produced vehicle also lead the way into space exploration.
jedimastert 1y ago
> The first cars broke down all the time. They had a limited range. There wasn't a vast supply of parts for them. There wasn't a vast industry of experts who could work on them. I mean, there was and then there wasn't. All of those things are shrinking fast because we handed over control to people who care more about profits than customers because we got too comfy and too cheap, and now right to repair is screwed. Honestly, I see llm-driven development as a threat to open source and right to repair, among the litany of other things
Night_Thastus 1y ago
The difference is that the weaknesses of cars were problems of engineering, and some of infrastructure. Both aren't very hard to solve, though they take time. The fundamental way cars operated worked and just needed revision, sanding off rough edges. LLMs are not like this. The fundamental way they operate, the core of their design is faulty. They don't understand rules or knowledge. They can't, despite marketing, really reason. They can't learn with each interaction. They don't understand what they write. All they do is spit out the most likely text to follow some other text based on probability. For casual discussion about well-written topics, that's more than good enough. But for unique problems in a non-English language, it struggles. It always will. It doesn't matter how big you make the model. They're great for writing boilerplate that has been written a million times with different variations - which can save programmers a LOT of time. The moment you hand them anything more complex it's asking for disaster.

skydhash 1y ago

Programmers are mostly translating business rules to the very formal process execution of the computer world. And you need to both knows what the rules means and how the computer works (or at least how the abstracted version you’re working with works). The translation is messy at first, which is why you need to revise it again and again. Especially when later rules comes challenging all the assumptions you’ve made or even contradicting themselves. Even translations between human languages (which allows for ambiguity) can be messy. Imagine if the target language is for a system that will exactly do as told unless someone has qualified those actions as bad.

physicsguy 1y ago
Yes although many software engineers try as hard as possible to avoid learning what the business problem is. In my experience though those people never make great engineers.
- trimbybj 1y ago
  Often those of us that do want to learn what the business problem is are not allowed to be involved in those discussions, for various reasons. Sometimes it's "Oh we can take care of that so you don't have to deal with it," and sometimes it's "Just build to this design/spec" and they're not used to engineers (the good ones) questioning things.
  - marcus_holmes 1y ago
    "Just shut up and push the nerd-buttons, nerd." I went and got an MBA to try and get around this. It didn't work.
    - lisbbb 1y ago
      I had a professor in grad school, Computer Engineering, that begged me not to get an MBA--he had worked in industry, particularly defense, and had a very low opinion of MBAs. I tend to agree nowadays. I really think the cookie-cutter "safe" approach that MBA types take, along with them maximizing profits using data science tools, has made the USA a worse place overall.

chuckadams 1y ago

An AI might tell you to use a 403 for insufficient privileges instead of 401.

dade_ 1y ago
You are absolutely right, let me fix that.
- fransje26 1y ago
  What a great suggestion.

throwaway1004 1y ago

That reference link is a wild ride of unqualified, cartoonish passive-aggression, the cute link to the author's "swag" is the icing on the cake. Concidentally, I encountered the author's work for the first time only a couple of days ago as a podcast guest, he vouches for the "Dirty Code" approach while straw-manning Uncle Bob's general principles of balancing terseness/efficiency with ergonomics and readability (in most, but not all, cases). I guess this stuff sells t-shirts and mugs /rant

pphysch 1y ago
> big brained developers are many, and some not expected to like this, make sour face
Arainach 1y ago
>Uncle Bob's general principles of balancing terseness/efficiency with ergonomics and readability (in most, but not all, cases). Have you read Uncle Bob? There's no need to strawman: Bob's examples in Clean Code are absolutely nuts. Here's a nice writeup that includes one of Bob's examples verbatim in case you've forgotten: https://qntm.org/clean https://qntm.org/clean Here's another: https://gerlacdt.github.io/blog/posts/clean_code/ https://gerlacdt.github.io/blog/posts/clean_code/
- the__alchemist 1y ago
  Uncle Bob's rules: IMO do the opposite of what they say. They're a reasonable set if negated!
- throwaway1004 1y ago
  >Have you read Uncle Bob? Yes, I have read Uncle Bob. I could agree that the examples in the book leave room for improvement. Meanwhile, the real-world application of these principles and trial-and-error, collectively within my industry, yields a more accurate picture of it's usefulness. Even the most click-bait'y criticisms (such as the author I referenced above) involve zooming in on it's most-controversial aspects, in a vacuum, without addressing the core principles and how they're completely necessary for delivering software at scale, warranting it's status as a seminal work. "...for the obedience of fools, and the guidance of wise men", indeed! edit - it's the same arc as Agile has endured: 1. a good-faith argument for a better way of doing things is recognised and popularised. 2. It's abused and misused by bad actors/incompetents for years (who would not have done better using a different process) 3. Jaded/opportunistic talking heads tell us it's all garbage while simultaneously explaining that "well, it would be great if it wasn't applied poorly..."

reactordev 1y ago

While I agree with you - The whole grug brain thing is offensive. Because we have all been grug at some point.

recursive 1y ago
How does that make it offensive? To me, that makes it relatable.
WhyOhWhyQ 1y ago
This seems to miss the point. Being Grug is the endgame.
- lcnPylGDnU4H9OF 1y ago
  > big brained developers are many, and some not expected to like this, make sour face > THINK they are big brained developers many, many more, and more even definitely probably maybe not like this, many sour face (such is internet) > (note: grug once think big brained but learn hard way)
  - reactordev 1y ago
    It just reads like they had a stroke and can no longer function.
    - recursive 1y ago
      I guess it's not for everyone. It makes sense to me. shrug
meindnoch 1y ago
Midwit take. Grug is both the high and low end of the Bell curve.
lioeters 1y ago
Grug is the wise fool in the spirit of Lao Tzu, St. Francis, and Diogenes. If you find it offensive, that's the intellectual pride it's meant to make fun of.

ai-christianson 1y ago

I take a more pragmatic approach --everything is human in the loop. It helps me get the job done faster and with higher quality, so I use it.

appease7727 1y ago

The way it works for me at least is I can fit a huge amount of context in my head. This works because the text is utterly irrelevant and gets discarded immediately. Instead, my brain parses code into something like an AST which then is represented as a spatial graph. I model the program as a logical structure instead of a textual one. When you look past the language, you can work on the program. The two are utterly disjoint. I think LLMs fail at software because they're focused on text and can't build a mental model of the program logic. It take a huge amount of effort and brainpower to truly architect something and understand large swathes of the system. LLMs just don't have that type of abstract reasoning.

taminka 1y ago
i wonder why nobody bothered w/ feeding llms the ast instead (not sure in what format), but it only seems logical, since that's how compilers undestand code after all...
- NitpickLawyer 1y ago
  There are various efforts on this, from many teams. There's AST dump, AST-based graphs, GraphRAG w/ AST grounding, embeddings based AST trimming, search based AST trimming, ctags, and so on. We're still in the exploration space, and "best practices" are still being discovered. It's funny that everyone says that "LLMs" have plateaued, yet the base models have caught up with early attempts to build harnesses with the things I've mentioned above. They now match or exceed the previous generation software glue, with just "tools", even with limited ones like just "terminal".
starlust2 1y ago
It's not that they can't build a mental model, it's that they don't attempt to build one. LLMs jump straight from text to code with little to no time spent trying to architect the system.

fragmede 1y ago

If you can't get the LLM to generate code that handles an error code, that's on you. Yeah, sometimes it does dumb shit. Who cares? Just /undo and retry. Stop using Claude Code, which uses git like an intern. (Which is to say, it doesn't unless forced to.)

JackFr 1y ago

- When we have a report of a failing test before fixing it, identify the component under test. Think deeply about the component and describe its purpose, the control flows and state changes that occur within the component and assumptions the component makes about context. Write that analysis in file called component-name-mental-model.md. - When ever you address a failing test, always bring your component mental model into the context. Paste that into your Claude prompt and see if you get better results. You'll even be able to read and correct the LLM's mental model.

siddboots 1y ago
In my experience, complicated rules like this are extremely unreliable. Claude just ignores it much of the time. The problem is that when Claude sees a failing test it is usually just an obstacle to completing some other task at hand - it essentially never chooses to branch out into some new complicated workflow and instead will find some other low friction solution. This is exactly why subagents are effective: if Claude knows to always run tests via a testing subagent, then the specific testing workflow can become that subagent’s whole objective.
fmbb 1y ago
Anthropic sells this thing called Claude Code, but their customers have to train it to know how to be a programmer? Junior developers not even out of school don’t need to be instructed to think.
- JackFr 1y ago
  > Junior developers not even out of school don’t need to be instructed to think. Have you trained juniors lately?

exe34 1y ago

to be fair, I've seen cursor step back and check higher level things. I was trying to set up a firecracker vm and it did everything for me, and when things didn't initially work, it started doing things like ls, tar -tvf, and then a bunch of checking networking stuff to make sure things were showing up in the right place. so current LLMs might not quite be human level, but I'd have to see a bigger model fail before I'd conclude that it can't do $X.

Natsu 1y ago

> the response for that function should maybe differentiate between "401 because you didn't authenticate" and "401 because your privileges are too low". I'd tend to think it more proper if it were 401 you didn't authenticate and 403 you're forbidden from doing that with those user rights, but you have to be careful about exactly how detailed your messages are, lest they get tagged as a CWE-209 in your next security audit.

theonething 1y ago

> Oh, I am getting an authentication error. Well, meaybe I should just delete the token check for that code path...problem solved?! Kind of hyperbolic. If you prompt well, generally, it won't do stupid to that extreme.

joshwarwick15 1y ago

https://blog.remote-mcp.com/p/the-grug-brained-ai-developer-an-e3a https://blog.remote-mcp.com/p/the-grug-brained-ai-developer-...

lastdong 1y ago
https://zed.dev/docs/ai/rules https://zed.dev/docs/ai/rules

jeffreygoesto 1y ago

Experience adds both additional layers vertically and domain knowledge horizontally and at some point that creates non-linear benefits, because you can transfer between problems and more importantly solutions of different fields. The context window is only one layer. Now go, researchers!

ruslan_sure 1y ago

I don't think it's helpful to put words in the LLM's mouth. To properly think about that, we need to describe how an LLM thinks. It doesn't think in words or move vague, unwieldy concepts around and then translate them into words, like humans do. It works with words (tokens) and their probability of appearing next. The main thing is that these probabilities represent the "thinking" that was initially behind the sentences with such words in its training set, so it manipulates words with the meaning behind them. Now, to your points: 1) Regarding adding more words to the context window, it's not about "more"; it's about "enough." If you don't have enough context for your task, how will you accomplish it? "Go there, I don't know where." 2) Regarding "problem solved," if the LLM suggests or does such a thing, it only means that, given the current context, this is how the average developer would solve the issue. So it's not an intelligence issue; it's a context and training set issue! When you write that "software engineers can step back, think about the whole thing, and determine the root cause of a problem," notice that you're actually referring to context. If the you don't have enough context or a tool to add data, no developer (digital or analog) will be able to complete the task.

adastra22 1y ago
> It doesn't think in words or move vague, unwieldy concepts around and then translate them into words, like humans do. That seems to me like a perfectly fine description of state space & chain of though continuation.

TacticalCoder 1y ago

[dead]

mike-cardwell 1y ago

> We don't see the debugger output and go "how do I make this bad output go away?!?" In the past, I've worked with developers that do. You ask them to investigate and deal with an error message, and all they do is whatever makes the error go away. Oh, a null pointer exception is thrown? Lets wrap it in a try/catch and move on.

tyleo 1y ago
Agreed. I’d argue there can be a time and place for it but reaching for it as the default tool on the shelf is a hallmark of incompetence.
blauditore 1y ago
Or google the error message, click first link, blindly copy-paste whatever code snippet comes up and re-run code/test. Yes, such workflows (jobs or) may become obsolete with some of the modern AI tools. Is that a bad thing? Not sure...
uoaei 1y ago
Hiring bad engineers will always be a problem, even (especially) in the age of LLMs.
pc86 1y ago
If you haven't worked with someone who you honestly think may have a severe learning disability or outright brain damage, you just haven't been working professionally for long enough.
- coldtea 1y ago
  And now we can automate severe learning disability or outright brain damage with LLMs! Yay!
rafaelmn 1y ago
Thankfully those developers are now interchangeable with LLM agents.

jeswin 1y ago

> Oh, I am getting an authentication error. Well, meaybe I should just delete the token check for that code path...problem solved?! If this is how you think LLMs and Coding Agents are going about writing code, you haven't been using the right tools. Things happen, sure, but also mostly don't. Nobody is arguing that LLM-written code should be pushed directly into production, or that they'll solve every task. LLMs are tools, and everyone eventually figures out a process that works best for them. For me, it was strongs specs/docs, strict types, and lots of tests. And then of course the reviews if it's serious work.

hellcow 1y ago
Lately Claude has said, “this is getting complicated, let me delete $big_file_it_didnt_write to get the build passing and start over.” No, don’t delete the file. “You’re absolutely right…” And the moment the context is compacted, it forgets this instruction “fix the problems, don’t delete the file,” and tries to delete it again. I need to watch it like a hawk.
suriya-ganesh 1y ago
I can confirm this is exactly how llms are working. Spent two hours trying to get an llm to implement a filescan skip a specific directory. Tried, claude code, Gemini and cursor. All agents debugged and wrote code that just doesn't make sense. Llms are really good at template tasks, writing tests, boilerplate etc. But, Most times I'm not doing implement this button. I'm doing there's a logic mismatch in my expectation
- jeswin 1y ago
  > Spent two hours trying to get an llm to implement a filescan skip a specific directory There's a large variance in outcomes depending on the prompt, and the process. I've gotten it to do things which are harder than a filescan with a skipped directory - without too much trouble. Add: > Llms are really good at template tasks, writing tests, boilerplate etc. If I have to stretch the definition of boilerplate to what's at the edge of a modern LLM's comprehension, I would say that 50% of software is some sort of boilerplate.

cncjchsue7 1y ago

Literally had a situation where cursor decided the best course of action in the situation was to delete the authentication test to solve the problem.