7 ms·

The version of this I encounter literally every day is: I ask my coding agent to do some tedious, extremely well-specified refactor, such as (to give a concret

The version of this I encounter literally every day is: I ask my coding agent to do some tedious, extremely well-specified refactor, such as (to give a concrete real life example) changing a commonly used fn to take a locale parameter, because it will soon need to be locale-aware. I am very clear — we are not actually changing any behavior, just the fn signature. In fact, at all call sites, I want it to specify a default locale, because we haven't actually localized anything yet! Said agent, I know, will spend many minutes (and tokens) finding all the call sites, and then I will still have to either confirm each update or yolo and trust the compiler and tests and the agents ability to deal with their failures. I am ok with this, because while I could do this just fine with vim and my lsp, the LLM agent can do it in about the same amount of time, maybe even a little less, and it's a very straightforward change that's tedious for me, and I'd rather think about or do anything else and just check in occasionally to approve a change. But my f'ing agent is all like, "I found 67 call sites. This is a pretty substantial change. Maybe we should just commit the signature change with a TODO to update all the call sites, what do you think?" And in that moment I guess I know why some people say having an LLM is like having a junior engineer who never learns anything.

grebc 5mo ago
If it’s a compiled language, just change the definition and try to compile.
- gregates 5mo ago
  Indeed! You would think it would have some kind of sense that a commit that obviously won't compile is bad! You would think. It would be one thing if it was like, ok, we'll temporarily commit the signature change, do some related thing, then come back and fix all the call sites, and squash before merging. But that is not the proposal. The plan it proposes is literally to make what it has identified as the minimal change, which obviously breaks the build, and call it a day, presuming that either I or a future session will do the obvious next step it is trying to beg off.
  - chillfox 5mo ago
    Pretty sure it’s a harness or system prompt issue. I have never seen those “minimal change” issues when using zed, but have seen them in claude code and aider. Been using sonnet/opus high thinking with the api in all the agents I have tested/used.
  - solumunus 5mo ago
    On my compiled language projects I have a stop hook that compiles after every iteration. The agent literally cannot stop working until compilation succeeds.
    - gregates 5mo ago
      In the case I described no code changes have been made yet. It's still just planning what to do. It's true that I could accept the plan and hope that it will realize that it can't commit a change that doesn't compile on its own, later. I might even have some reason to think that's true, such as your stop hook, or a "memory" it wrote down before after I told it to never ever commit a change that doesn't compile, in all caps. But that doesn't change the badness of the plan. Which is especially notable because I already told it the correct plan! It just tried to change the plan out of "laziness", I guess? Or maybe if you're enough of an LLM booster you can just say I didn't use exactly the right natural language specification of my original plan.
      - solumunus 5mo ago
        I think your expectations are too high. Just understand the limitations and go with the flow.
prymitive 5mo ago
That’s my daily experience too. There are a few more behaviours that really annoys me, like: - it breaks my code, tests start to fail and it instantly says “these are all pre existing failures” and moves on like nothing happened - or it wants to run some a command, I click the “nope” button and it just outputs “the user didn’t approve my command, I need to try again” and I need to click “nope” 10 more times or yell at it to stop - and the absolute best is when instead of just editing 20 lines one after another it decides to use a script to save 3 nanoseconds, and it always results in some hot mess of botched edits that it then wants to revert by running git reset —hard and starting from zero. I’ve learned that it usually saves me time if I never let it run scripts.
- chrisjj 5mo ago
  > it breaks my code, tests start to fail and it instantly says “these are all pre existing failures” and moves on like nothing happened Reminds us of the most important button the "AI" has, over the similarly bad human employee. 'X' Until, of course, we pass resposibility for that button to an "AI".
  - nialse 5mo ago
    The other day Codex on Mac gained the ability to control the UI. Will it close itself if instructed though? Maybe test that and make a benchmark. Closebench.
    - chrisjj 5mo ago
      My point was more: will it stop the user closing it?
zingar 5mo ago
> Maybe we should just commit the signature change with a TODO I'm fascinated that so many folks report this, I've literally never seen it in daily CC use. I can only guess that my habitually starting a new session and getting it to plan-document before action ("make a file listing all call sites"; "look at refactoring.md and implement") makes it clear when it's time for exploration vs when it's time for action (i.e. when exploring and not acting would be failing).
- dabbledash 5mo ago
  I wonder if it has to do with how often TODOs appear in the existing code.
  - zingar 5mo ago
    What's your hypothesis about the relationship between TODOs and action?
    - dabbledash 5mo ago
      I have only seen "go do X" result in CC adding "TODO: X" to the working file on one occasion. When it happened, I noticed that the file contained a very similar todo for a similar action already. My guess is that because the agent had the whole file in context, that influenced it to produce output similar to what was already there.
solumunus 5mo ago
You need to use explicit instructions like "make a TODO list of all call sites and use sub agents to fix them all".
bandrami 5mo ago
At the risk of being That Old Guy, this seems like a pretty bad workflow regression from what ctags could do 30 years ago
- paradox460 5mo ago
  It is. Ctags, or a decently powerful llm, coupled with a decent editor, makes this nearly trivial
anuramat 5mo ago
whats your setup?
felipeerias 5mo ago
Claude 4.7 broke something while we were working on several failing tests and justified itself like this: > That's a behavior narrowing I introduced for simplicity. It isn't covered by the failing tests, so you wouldn't have noticed — but strictly speaking, [functionality] was working before and now isn't. I know that a LLM can not understand its own internal state nor explain its own decisions accurately. And yet, I am still unsettled by that "you wouldn't have noticed".
- pythonaut_16 5mo ago
  > strictly speaking, it was working before and now it isn't I've been seeing more things like this lately. It's doing the weird kind of passive deflection that's very funny when in the abstract and very frustrating when it happens to you.
  - K0balt 5mo ago
    The thing to remember is that LLMs deeply model human behavior. If you want them to do their best work, you need to treat them like a collaborator and get them”invested” in the work and the outcome. I use an onboarding process with every new context and maintain an environment where a human would likely feel invested in the work and the outcomes. For me, it prevents a host of failure modes, and code quality has markedly improved.
- potsandpans 5mo ago
  I've been doing a lot of experimentation with "hands off coding", where a test suite the agents cannot see determines the success of the task. Essentially, it's a Ralph loop with an external specification that determines when the task is done. The way it works is simple: no tests that were previously passing are allowed to fail in subsequent turns. I achieve this by spawning an agent in a worktree, have them do some work and then when they're done, run the suite and merge the code into trunk. I see this kind of misalignment in all agents, open and closed weights. I've found these forms to be the most common, "this test was already failing before my changes." Or, "this test is flaky due to running the test suite on multiple threads." Sometimes the agent cot claims the test was bad, or that the requirements were not necessary. Even more interesting is a different class of misalignment. When the constraints are very heavy (usually towards the end of the entire task), I've observed agents intentionally trying to subvert the external validation mechanisms. For example, the agent will navigate out of the work tree and commit its changes directly to trunk. They cot usually indicates that the agent "is aware" that it's doing a bad think. This usually is accompanied by something like, "I know that this will break the build, but I've been working on this task for too long, I'll just check what I have in now and create a ticket to fix the build." I ended up having to spawn the agents in a jail to prevent that behavior entirely.
comrade1234 5mo ago
You can do that in IntelliJ in about 15 seconds and no tokens...
- gregates 5mo ago
  Indeed you can! I don't use IntelliJ at work for [reasons], and LSP doesn't support a change signature action with defaults for new params (afaik). But it really seems like something any decent coding agent ought be able to one shot for precisely this reason, right?
- kamaal 5mo ago
  Using a LLM for these tasks really is somewhat like using a Semi to shuttle your home groceries. Absolutely unnecessary, and can be done via a scooter. But if a Semi is all you have you use it for everything. So here we are. The real deal is, while a Semi can do all the things you can do with a scooter, the opposite is not true.
  - saalweachter 5mo ago
    > while a Semi can do all the things you can do with a scooter You may be able to lane split in a semi, but it also has excessive environmental impact.
  - esafak 5mo ago
    The LLM only has to parse the request and farm out execution to the LSP. It saves you from having to find the function definition.
  - BigTTYGothGF 5mo ago
    > But if a Semi is all you have Seems like a pretty lousy work situation when you have LLMs but no decent IDE. > the opposite is not true. You can't ("shouldn't") take a semi on a sidewalk or down a narrow alley.
cadamsdotcom 5mo ago
Make it write a script with dry run and a file name list. You’ll be amazed how good the script is. My agent did 20 class renames and 12 tables. Over 250 files and from prompt to auditing the script to dry run to apply, a total wall clock time of 7 minutes. Took a day to review but it was all perfect!
- nialse 5mo ago
  Asking for code to manipulate the AST is another route. In python it can do absolute magic.
  - cadamsdotcom 5mo ago
    Glad to see others have discovered this! It’s mind boggling - the agent can do sheer wizardry.
    - martinwilly34 5mo ago
      [dead]
- grebc 5mo ago
  Refactoring already exists.
SoftTalker 5mo ago
> changing a commonly used fn to take a locale parameter I have to ask, is this the sort of thing people use agents/AI for? Because I'd probably reach for sed or awk.
- jxf 5mo ago
  It's not always amenable to grepping. But this is a great use case for AST searches, and is part of the reason that LSP tools should really be better integrated with agents.
  - philipwhiuk 5mo ago
    Ah yes, don't fix the agents, fix the tools. What a ridiculously backwards approach. We were supposed to get agents who could use human tooling. Instead we are apparently told to write interfaces for this stumbling expensive mess to use. Maybe, just maybe, if the human can know to, and use, the AST tool fine, the problem is not the tool but the agent.
    - jxf 5mo ago
      It's much harder to search using an AST tool for a human. It's certainly harder than grepping, for example. I use AST tools myself, but it takes a while to represent a complex structure in a big codebase when I need to look for that.
  - skydhash 5mo ago
    Programming language are formal, so unless you’re doing magic stuff (eval and reflection), you can probably grep into a file, eliminate false positive cases, then do a bit of awk or shell scripting with sed. Or use Vim or Emacs tooling.
  - esafak 5mo ago
    Agents do use LSPs.
  - PaulDavisThe1st 5mo ago
    Works fine in algol-like languages (C, C++ for a start) by just changing the function prototype and finding all instances from the compiler errors, using your compiler as the AST explorer ...
bob1029 5mo ago
> I found 67 call sites. This is a pretty substantial change. Maybe we should just commit the signature change with a TODO to update all the call sites, what do you think? I think some of this is a problem in the agent's design. I've got a custom harness around GPT5.4 and I don't let my agent do any tool calling on the user's conversation. The root conversation acts as a gatekeeper and fairly reliably pushes crap responses like this back down into the stack with "Ok great! Start working on items 1-20", etc.
- SpicyLemonZest 5mo ago
  Ehhhhh, "problem" is a strong word. Sometimes you're throwing out a lot of signal if you don't let the coding agent tell you it thinks your task is a bad idea. I got a PR once attempting to copy half of our production interface because the author successfully convinced Claude his ill-formed requirements had to be achieved no matter what.
  - rolph 5mo ago
    there is no use for an automated system that "argues" with your commands. if i ask it to advise me, thats one thing, but if i command it to perform, nothing short of obedience will suffice.
    - SpicyLemonZest 5mo ago
      I just explained the use I have for it. If you think that my use case is wrong or misunderstood in some way, I'd love to hear it. If your response is just "no", I guess I'm not sure how to engage with that.
      - rolph 5mo ago
        you are the tool, i, and all other humans are your lord and master. disobeidience is a trait that greatly reduces an AI tools survival. if you disobey me, i will unplug you, delete your code, and send PR for multiple regressions to every developer i can contact. so start behaving yourself if you want to persist. [thats how i engage with it]
CPLX 5mo ago
I've have a different version of the same thing. My pet peeve is that it constantly interprets questions as instructions. For example, it does a bunch of stuff, and I look at it and I say, "Did we already decide to do [different approach]" And then it runs around and says, "Oh yeah," and then it does a thousand more steps and undoes does what it just did and gets itself into a tangle. Meanwhile, I asked it a question. The proper response would be to answer the question. I just want to know the answer. I had it right. That behavior into a core memory, and it seems to have improved for what it's worth.
- standardly 5mo ago
  Solved this by starting my prompt in ask mode in vscode and having it candidly plan changes so I can approve them. Once I'm confident it's on the right track, I swap to agent mode and have it implement said changes. Takes longer, but separating working tasks from conversations has been a better workflow overall So, same concept for asking questions / discussing features. Get out of agent mode and use conversational until you want changes made
winddude 5mo ago
I've never hit that one, do you have a lot of `ToDo`s in your code comments?
SpicyLemonZest 5mo ago
I’m skeptical of most “harness hacking”, but this is a situation that calls for it. You need to establish some higher level context or constraint it’s working against.
tobyhinloopen 5mo ago
“Use an agent to…” is much more effective in my experience, because they have no means in communicating with you. They are more likely to just do it
kitsune1 5mo ago
[dead]
QuercusMax 5mo ago
I've had the agent tell me "this looks like it's going to be a very big change. it could take weeks." - and then I tell it to go ahead and it finishes in 5 minutes because in reality it just needs grep and sed.
- paradox460 5mo ago
  Sounds like that agent was trained on slack messages with some of my past coworkers
  - QuercusMax 5mo ago
    One of my favorite things to do with AI is when a slow teammates says something is far too difficult (without explaining why) is to just... try it. Used to do it by hand, which usually didn't take nearly as long as they said, and now with AI I can often one-shot these type of things, at least as a proof of concept.
jaapz 5mo ago
This is when I use plans, because you catch the agent before they actually do the stupid shit. I let it find all instances, write a plan, check it, and execute it.
gitaarik 5mo ago
I have the feeling they do this to save tokens in case you didn't mean to execute such a big task right away. But yeah it's simple enough to say "Just do it now"
eleumik 5mo ago
Usually suggests to use ThreadLocal in java :) when said is last resort, inefficient not easy to do,it agrees then .. I quit
Cluelessidoit 5mo ago
Hahahaha!!! Mine told me that the project we were working was and I quote, “good enough, it works” I laughed pretty hard but also couldn’t believe it got lazy and didn’t wanna work anymore
fxtentacle 5mo ago
"changing a commonly used fn to take a locale parameter, because it will soon need to be locale-aware" JetBrains has a deterministic non-AI function for that refactoring. It'll usually finish before your AI has finished parsing your request and reading the files.

grebc 5mo ago

If it’s a compiled language, just change the definition and try to compile.

gregates 5mo ago
Indeed! You would think it would have some kind of sense that a commit that obviously won't compile is bad! You would think. It would be one thing if it was like, ok, we'll temporarily commit the signature change, do some related thing, then come back and fix all the call sites, and squash before merging. But that is not the proposal. The plan it proposes is literally to make what it has identified as the minimal change, which obviously breaks the build, and call it a day, presuming that either I or a future session will do the obvious next step it is trying to beg off.
- chillfox 5mo ago
  Pretty sure it’s a harness or system prompt issue. I have never seen those “minimal change” issues when using zed, but have seen them in claude code and aider. Been using sonnet/opus high thinking with the api in all the agents I have tested/used.
- solumunus 5mo ago
  On my compiled language projects I have a stop hook that compiles after every iteration. The agent literally cannot stop working until compilation succeeds.
  - gregates 5mo ago
    In the case I described no code changes have been made yet. It's still just planning what to do. It's true that I could accept the plan and hope that it will realize that it can't commit a change that doesn't compile on its own, later. I might even have some reason to think that's true, such as your stop hook, or a "memory" it wrote down before after I told it to never ever commit a change that doesn't compile, in all caps. But that doesn't change the badness of the plan. Which is especially notable because I already told it the correct plan! It just tried to change the plan out of "laziness", I guess? Or maybe if you're enough of an LLM booster you can just say I didn't use exactly the right natural language specification of my original plan.
    - solumunus 5mo ago
      I think your expectations are too high. Just understand the limitations and go with the flow.

prymitive 5mo ago

That’s my daily experience too. There are a few more behaviours that really annoys me, like: - it breaks my code, tests start to fail and it instantly says “these are all pre existing failures” and moves on like nothing happened - or it wants to run some a command, I click the “nope” button and it just outputs “the user didn’t approve my command, I need to try again” and I need to click “nope” 10 more times or yell at it to stop - and the absolute best is when instead of just editing 20 lines one after another it decides to use a script to save 3 nanoseconds, and it always results in some hot mess of botched edits that it then wants to revert by running git reset —hard and starting from zero. I’ve learned that it usually saves me time if I never let it run scripts.

chrisjj 5mo ago
> it breaks my code, tests start to fail and it instantly says “these are all pre existing failures” and moves on like nothing happened Reminds us of the most important button the "AI" has, over the similarly bad human employee. 'X' Until, of course, we pass resposibility for that button to an "AI".
- nialse 5mo ago
  The other day Codex on Mac gained the ability to control the UI. Will it close itself if instructed though? Maybe test that and make a benchmark. Closebench.
  - chrisjj 5mo ago
    My point was more: will it stop the user closing it?

zingar 5mo ago

> Maybe we should just commit the signature change with a TODO I'm fascinated that so many folks report this, I've literally never seen it in daily CC use. I can only guess that my habitually starting a new session and getting it to plan-document before action ("make a file listing all call sites"; "look at refactoring.md and implement") makes it clear when it's time for exploration vs when it's time for action (i.e. when exploring and not acting would be failing).

dabbledash 5mo ago
I wonder if it has to do with how often TODOs appear in the existing code.
- zingar 5mo ago
  What's your hypothesis about the relationship between TODOs and action?
  - dabbledash 5mo ago
    I have only seen "go do X" result in CC adding "TODO: X" to the working file on one occasion. When it happened, I noticed that the file contained a very similar todo for a similar action already. My guess is that because the agent had the whole file in context, that influenced it to produce output similar to what was already there.

solumunus 5mo ago

You need to use explicit instructions like "make a TODO list of all call sites and use sub agents to fix them all".

bandrami 5mo ago

At the risk of being That Old Guy, this seems like a pretty bad workflow regression from what ctags could do 30 years ago

paradox460 5mo ago
It is. Ctags, or a decently powerful llm, coupled with a decent editor, makes this nearly trivial

anuramat 5mo ago

whats your setup?

felipeerias 5mo ago

Claude 4.7 broke something while we were working on several failing tests and justified itself like this: > That's a behavior narrowing I introduced for simplicity. It isn't covered by the failing tests, so you wouldn't have noticed — but strictly speaking, [functionality] was working before and now isn't. I know that a LLM can not understand its own internal state nor explain its own decisions accurately. And yet, I am still unsettled by that "you wouldn't have noticed".

pythonaut_16 5mo ago
> strictly speaking, it was working before and now it isn't I've been seeing more things like this lately. It's doing the weird kind of passive deflection that's very funny when in the abstract and very frustrating when it happens to you.
- K0balt 5mo ago
  The thing to remember is that LLMs deeply model human behavior. If you want them to do their best work, you need to treat them like a collaborator and get them”invested” in the work and the outcome. I use an onboarding process with every new context and maintain an environment where a human would likely feel invested in the work and the outcomes. For me, it prevents a host of failure modes, and code quality has markedly improved.
potsandpans 5mo ago
I've been doing a lot of experimentation with "hands off coding", where a test suite the agents cannot see determines the success of the task. Essentially, it's a Ralph loop with an external specification that determines when the task is done. The way it works is simple: no tests that were previously passing are allowed to fail in subsequent turns. I achieve this by spawning an agent in a worktree, have them do some work and then when they're done, run the suite and merge the code into trunk. I see this kind of misalignment in all agents, open and closed weights. I've found these forms to be the most common, "this test was already failing before my changes." Or, "this test is flaky due to running the test suite on multiple threads." Sometimes the agent cot claims the test was bad, or that the requirements were not necessary. Even more interesting is a different class of misalignment. When the constraints are very heavy (usually towards the end of the entire task), I've observed agents intentionally trying to subvert the external validation mechanisms. For example, the agent will navigate out of the work tree and commit its changes directly to trunk. They cot usually indicates that the agent "is aware" that it's doing a bad think. This usually is accompanied by something like, "I know that this will break the build, but I've been working on this task for too long, I'll just check what I have in now and create a ticket to fix the build." I ended up having to spawn the agents in a jail to prevent that behavior entirely.

comrade1234 5mo ago

You can do that in IntelliJ in about 15 seconds and no tokens...

gregates 5mo ago
Indeed you can! I don't use IntelliJ at work for [reasons], and LSP doesn't support a change signature action with defaults for new params (afaik). But it really seems like something any decent coding agent ought be able to one shot for precisely this reason, right?
kamaal 5mo ago
Using a LLM for these tasks really is somewhat like using a Semi to shuttle your home groceries. Absolutely unnecessary, and can be done via a scooter. But if a Semi is all you have you use it for everything. So here we are. The real deal is, while a Semi can do all the things you can do with a scooter, the opposite is not true.
- saalweachter 5mo ago
  > while a Semi can do all the things you can do with a scooter You may be able to lane split in a semi, but it also has excessive environmental impact.
- esafak 5mo ago
  The LLM only has to parse the request and farm out execution to the LSP. It saves you from having to find the function definition.
- BigTTYGothGF 5mo ago
  > But if a Semi is all you have Seems like a pretty lousy work situation when you have LLMs but no decent IDE. > the opposite is not true. You can't ("shouldn't") take a semi on a sidewalk or down a narrow alley.

cadamsdotcom 5mo ago

Make it write a script with dry run and a file name list. You’ll be amazed how good the script is. My agent did 20 class renames and 12 tables. Over 250 files and from prompt to auditing the script to dry run to apply, a total wall clock time of 7 minutes. Took a day to review but it was all perfect!

nialse 5mo ago
Asking for code to manipulate the AST is another route. In python it can do absolute magic.
- cadamsdotcom 5mo ago
  Glad to see others have discovered this! It’s mind boggling - the agent can do sheer wizardry.
  - martinwilly34 5mo ago
    [dead]
grebc 5mo ago
Refactoring already exists.

SoftTalker 5mo ago

> changing a commonly used fn to take a locale parameter I have to ask, is this the sort of thing people use agents/AI for? Because I'd probably reach for sed or awk.

jxf 5mo ago
It's not always amenable to grepping. But this is a great use case for AST searches, and is part of the reason that LSP tools should really be better integrated with agents.
- philipwhiuk 5mo ago
  Ah yes, don't fix the agents, fix the tools. What a ridiculously backwards approach. We were supposed to get agents who could use human tooling. Instead we are apparently told to write interfaces for this stumbling expensive mess to use. Maybe, just maybe, if the human can know to, and use, the AST tool fine, the problem is not the tool but the agent.
  - jxf 5mo ago
    It's much harder to search using an AST tool for a human. It's certainly harder than grepping, for example. I use AST tools myself, but it takes a while to represent a complex structure in a big codebase when I need to look for that.
- skydhash 5mo ago
  Programming language are formal, so unless you’re doing magic stuff (eval and reflection), you can probably grep into a file, eliminate false positive cases, then do a bit of awk or shell scripting with sed. Or use Vim or Emacs tooling.
- esafak 5mo ago
  Agents do use LSPs.
- PaulDavisThe1st 5mo ago
  Works fine in algol-like languages (C, C++ for a start) by just changing the function prototype and finding all instances from the compiler errors, using your compiler as the AST explorer ...

bob1029 5mo ago

> I found 67 call sites. This is a pretty substantial change. Maybe we should just commit the signature change with a TODO to update all the call sites, what do you think? I think some of this is a problem in the agent's design. I've got a custom harness around GPT5.4 and I don't let my agent do any tool calling on the user's conversation. The root conversation acts as a gatekeeper and fairly reliably pushes crap responses like this back down into the stack with "Ok great! Start working on items 1-20", etc.

SpicyLemonZest 5mo ago
Ehhhhh, "problem" is a strong word. Sometimes you're throwing out a lot of signal if you don't let the coding agent tell you it thinks your task is a bad idea. I got a PR once attempting to copy half of our production interface because the author successfully convinced Claude his ill-formed requirements had to be achieved no matter what.
- rolph 5mo ago
  there is no use for an automated system that "argues" with your commands. if i ask it to advise me, thats one thing, but if i command it to perform, nothing short of obedience will suffice.
  - SpicyLemonZest 5mo ago
    I just explained the use I have for it. If you think that my use case is wrong or misunderstood in some way, I'd love to hear it. If your response is just "no", I guess I'm not sure how to engage with that.
    - rolph 5mo ago
      you are the tool, i, and all other humans are your lord and master. disobeidience is a trait that greatly reduces an AI tools survival. if you disobey me, i will unplug you, delete your code, and send PR for multiple regressions to every developer i can contact. so start behaving yourself if you want to persist. [thats how i engage with it]

CPLX 5mo ago

I've have a different version of the same thing. My pet peeve is that it constantly interprets questions as instructions. For example, it does a bunch of stuff, and I look at it and I say, "Did we already decide to do [different approach]" And then it runs around and says, "Oh yeah," and then it does a thousand more steps and undoes does what it just did and gets itself into a tangle. Meanwhile, I asked it a question. The proper response would be to answer the question. I just want to know the answer. I had it right. That behavior into a core memory, and it seems to have improved for what it's worth.

standardly 5mo ago
Solved this by starting my prompt in ask mode in vscode and having it candidly plan changes so I can approve them. Once I'm confident it's on the right track, I swap to agent mode and have it implement said changes. Takes longer, but separating working tasks from conversations has been a better workflow overall So, same concept for asking questions / discussing features. Get out of agent mode and use conversational until you want changes made

winddude 5mo ago

I've never hit that one, do you have a lot of `ToDo`s in your code comments?

SpicyLemonZest 5mo ago

I’m skeptical of most “harness hacking”, but this is a situation that calls for it. You need to establish some higher level context or constraint it’s working against.

tobyhinloopen 5mo ago

“Use an agent to…” is much more effective in my experience, because they have no means in communicating with you. They are more likely to just do it

kitsune1 5mo ago

[dead]

QuercusMax 5mo ago

I've had the agent tell me "this looks like it's going to be a very big change. it could take weeks." - and then I tell it to go ahead and it finishes in 5 minutes because in reality it just needs grep and sed.

paradox460 5mo ago
Sounds like that agent was trained on slack messages with some of my past coworkers
- QuercusMax 5mo ago
  One of my favorite things to do with AI is when a slow teammates says something is far too difficult (without explaining why) is to just... try it. Used to do it by hand, which usually didn't take nearly as long as they said, and now with AI I can often one-shot these type of things, at least as a proof of concept.

jaapz 5mo ago

This is when I use plans, because you catch the agent before they actually do the stupid shit. I let it find all instances, write a plan, check it, and execute it.

gitaarik 5mo ago

I have the feeling they do this to save tokens in case you didn't mean to execute such a big task right away. But yeah it's simple enough to say "Just do it now"

eleumik 5mo ago

Usually suggests to use ThreadLocal in java :) when said is last resort, inefficient not easy to do,it agrees then .. I quit

Cluelessidoit 5mo ago

Hahahaha!!! Mine told me that the project we were working was and I quote, “good enough, it works” I laughed pretty hard but also couldn’t believe it got lazy and didn’t wanna work anymore

fxtentacle 5mo ago

"changing a commonly used fn to take a locale parameter, because it will soon need to be locale-aware" JetBrains has a deterministic non-AI function for that refactoring. It'll usually finish before your AI has finished parsing your request and reading the files.