6 ms·

Preventing garbage just requires that you take into account the cognitive limits of the agent. For example ... 1) Don't ask for large / complex change. Ask for

Preventing garbage just requires that you take into account the cognitive limits of the agent. For example ... 1) Don't ask for large / complex change. Ask for a plan but ask it to implement the plan in small steps and ask the model to test each step before starting the next. 2) For really complex steps, ask the model to write code to visualize the problem and solution. 3) If the model fails on a given step, ask it to add logging to the code, save the logs, run the tests and the review the logs to determine what went wrong. Do this repeatedly until the step works well. 4) Ask the model to look at your existing code and determine how it was designed to implement a task. Some times the model will put all of the changes in one file but your code has a cleaner design the model doesn't take into account. I've seen other people blog about their tricks and tips. I do still see garbage results but not as high as 95%.

jason_zig 1y ago
I've seen people post this same advice and I agree with you that it works but you would think they would absorb this common strategy and integrate it as part of the underlying product at this point...
- tombot 1y ago
  Claude Code at least now lets you use its best model for planning mode and its cheapest model for coding mode.
  - candiddevmike 1y ago
    The consulting world parallels here are funny
    - baq 1y ago
      Humans are agents after all
- noosphr 1y ago
  The people who build the models don't understand how to use the models. It's like asking people who design CPUs to build data-centers. I've interviewed with three tier one AI labs and _no-one_ I talked to had any idea where the business value of their models came in. Meanwhile Chinese labs are releasing open source models that do what you need. At this point I've build local agentic tools that are better than anything Claude and OAI have as paid offerings, including the $2,000 tier. Of course they cost between a few dollars to a few hundred dollars per query so until hardware gets better they will stay happily behind corporate moats and be used by the people blessed to burn money like paper.
  - Barbing 1y ago
    Very interesting. And plausible. Doesn't specifically seem to jive with the claim Anthropic made where they were worried about Claude Code being their secret sauce, leaving them unsure whether to publicly release it. (I know some skeptical about that claim.)
  - criemen 1y ago
    > The people who build the models don't understand how to use the models. It's like asking people who design CPUs to build data-centers. This doesn't match the sentiment on hackernews and elsewhere that claude code is the superior agentic coding tool, as it's developed by one of the AI labs, instead of a developer tool company.
    - noosphr 1y ago
      Claude code is babies first agentic tool. You don't see better ones from code tooling companies because the economics don't work out. No one is going to pay $1,000 for a two line change on a 500,000k line code base after waiting four hours. LLMs today the equivalent of a 4bit ALU without memory being sold as a fully functional personal computer. And like ALUs today, you will need _thousands_ of LLMs to get anything useful done, also like ALUs in 1950 we're a long way off from a personal computer being possible.
      - 1y ago
- nostrademons 1y ago
  A lot of it is integrated into the product at this point. If you have a particularly tricky bug, you can just tell Claude "I have this bug. I expected output 'foo' and got output 'bar'. What went wrong?" It will inspect the code and sometimes suggest a fix. If you run it and it still doesn't work, you can say "Nope, still not working", and Claude will add debug output to the whole program, tell you to run it again, and paste the debug output back into the console. Then it will use your example to write tests, and run against them.
deleted 1y ago
[deleted]
paulcole 1y ago
> Ask for a plan but ask it to implement the plan in small steps and ask the model to test each step before starting the next. Tried this on a developer I worked with once and he just scoffed at me and pushed to prod on a Friday.
- NitpickLawyer 1y ago
  > scoffed at me and pushed to prod on a Friday. that's the --yolo flag in cc :D
rvnx 1y ago
Your tips are perfect. Most users will just give a vague tasks like: "write a clone of Steam" or "create a rocket" and then they blame Claude Code. If you want AI to code for you, you have to decompose your problem like a product owner would do. You can get helped by AI as well, but you should have a plan and specifications. Once your plan is ready, you have to decompose the problem into different modules, then make sure each modules are tested. The issue is often with the user, not the tool, as they have to learn how to use the tool first.
- wordofx 1y ago
  > Most users will just give a vague tasks like: "write a clone of Steam" or "create a rocket" and then they blame Claude Code. This seems like half of HN with how much HN hates AI. Those who hate it or say it’s not useful to them seem to be fighting against it and not wanting to learn how to use it. I still haven’t seen good examples of it not working even with obscure languages or proprietary stuff.
  - LtWorf 1y ago
    If you have to iterate 10 times, that is "not working", since it already wasted way more time than doing it manually to begin with.
  - drzaiusx11 1y ago
    Anyone who has mentored as part of a junior engineer internship program AND has attempted to use current gen ai tooling will notice the parallels immediately. There are key differences though that are worth highlighting. The main difference is that with the current batch of genai tools, the AI's context resets after use, whereas a (good) intern truly learns from prior behavior. Additionally, as you point out, the language and frameworks need to be part of the training set since AI isn't really "learning" it's just prepolulating a context window for its pre-existing knowledge (token prediction), so ymmv depending on hidden variables from the secret (to you, the consumers) training data and weights. I use Ruby primarily these days, which is solidly in the "boring tech" camp and most AIs fail to produce useful output that isn't rails boilerplate. If I did all my IC contributions via directed intern commits I'd leave the industry out of frustration. Using only AI outputs for producing code changes would be akin to torture (personally.) Edit: To clarify I'm not against AI use, I'm just stating that with the current generation of tools it is a pretty lackluster experience when it comes to net new code generation. It excells at one off throwaway scripts and making large tedious redactors less drudgerly. I wouldn't pivot to it being my primary method of code generation until some of the more blatant productiviy losses are addressed.
MikeTheGreat 1y ago
Genuine question: What do you mean by " ask it to implement the plan in small steps"? One option is to write "Please implement this change in small steps?" more-or-less exactly Another option is to figure out the steps and then ask it "Please figure this out in small steps. The first step is to add code to the parser so that it handles the first new XML element I'm interested in, please do this by making the change X, we'll get to Y and Z later" I'm sure there's other options, too.
- Benjammer 1y ago
  My method is that I work together with the LLM to figure out the step-by-step plan. I give an outline of what I want to do, and give some breadcrumbs for any relevant existing files that are related in some way, ask it to figure out context for my change and to write up a summary of the full scope of the change we're making, including an index of file paths to all relevant files with a very concise blurb about what each file does/contains, and then also to produce a step-by-step plan at the end. I generally always have to tell it to NOT think about this like a traditional engineering team plan, this is a senior engineer and LLM code agent working together, think only about technical architecture, otherwise you get "phase 1 (1-2 weeks), phase 2 (2-4 weeks), step a (4-8 hours)" sort of nonsense timelines in your plan. Then I review the steps myself to make sure they are coherent and make sense, and I poke and prod the LLM to fix anything that seems weird, either fixing context or directions or whatever. Then I feed the entire document to another clean context window (or two or three) and ask it to "evaluate this plan for cohesiveness and coherency, tell me if it's ready for engineering or if there's anything underspecified or unclear" and iterate on that like 1-3 times until I run a fresh context window and it says "This plan looks great, it's well crafted, organized, etc...." and doesn't give feedback. Then I go to a fresh context window and tell it "Review the document @MY_PLAN.md thoroughly and begin implementation of step 1, stop after step 1 before doing step 2" and I start working through the steps with it.
  - lkjdsklf 1y ago
    The problem is, by the time you’ve gone through the process of making a granular plan and all that, you’ve lost all productivity gains of using the agent. As an engineer, especially as you get more experience, you can kind of visualize the plan for a change very quickly and flesh out the next step while implementing the current step All you have really accomplished with the kind of process described is make the worlds least precise, most verbose programming language
adastra22 1y ago
This is why the jobs market for new grads and early career folks has dried up. A seasoned developer knows that this is how you manage work in general, and just treats the AI like they would a junior developer—and gets good results.
- CuriouslyC 1y ago
  Why bother handing stuff to a junior when an agent will do it faster while asking fewer questions, and even if the first draft code isn't amazing, you can just quality gate with an LLM reviewer that has been instructed to be brutal and do a manual pass when the code gets by the LLM reviewer.
  - LtWorf 1y ago
    Because juniors learn while LLMs don't and you must explain the same thing over and over forever.
    - adastra22 1y ago
      If you are explaining things more than once, you are doing it wrong. Which is not on you as the tools currently suck big time. But it is quite possible to have LLM agents “learn” by intelligently matching context (including historical lessons learned) to conversation.
MangoCoffee 1y ago
I've been vibe coding a couple of personal projects. I've found that test-driven development fits very well with vibe coding, and it's just as you said break up the problem into small, testable chunks, get the AI to write unit tests first, and then implement the actual code
- yodsanklai 1y ago
  Actually, all good engineering principles which reduce cognitive load for humans work for AI as well.
  - colordrops 1y ago
    This is the big secret. Keep code modular, small, single purpose, encapsulated, and it works great with vibe coding. I want to write a protocol/meta language similar to the markdown docs that Claude et al create that is per module, and defines behavior, so you actually program and compose modules with well defined interfaces in natural language. I'm surprised someone hasn't done it already.
    - drzaiusx11 1y ago
      Isn't what you're describing exactly what Kiro aims to solve?
      - colordrops 1y ago
        Possibly, I've never heard of Kiro, will look into it.
    - adastra22 1y ago
      My set of Claude agent files have an explicit set of interface definitions. Is that what you’re talking about?
      - colordrops 1y ago
        Are Claude agent files per module? If so, then I guess so.
rmonvfer 1y ago
I’d like to add: keep some kind of development documentation where you describe in detail the patterns and architecture of your application and it’s components. I’ve seen incredible improvements just by doing this and using precise prompting to get Claude to implement full services by itself, tests included. Of course it requires manual correction later but just telling Claude to check the development documentation before starting work on a feature prevents most hallucinations (that and telling it to use the Context7 MCP for external documentation), at least in my experience. The downside to this is that 30% of your context window will be filled with documentation but hey, at least it won’t hallucinate API methods or completely forget that it shouldn’t reimplement something. Just my 2 cents.
deleted 1y ago
[deleted]
nostrademons 1y ago
I've found that an effective tactic for larger, more complex tasks is to tell it "Don't write any code now. I'm going to describe each of the steps of the problem in more detail. The rough outline is going to be 1) Read this input 2) Generate these candidates 3) apply heuristics to score candidates 4) prioritize and rank candidates 5) come up with this data structure reflecting the output 6) write the output back to the DB in this schema". Claude will then go and write a TODO list in the code (and possibly claude.md if you've run /init), and prompt you for the details of each stage. I've even done this for an hour, told Claude "I have to stop now. Generate code for the finished stages and write out comments so you can pick up where you left off next time" and then been able to pick up next time with minimal fuss.
- hex4def6 1y ago
  FYI: You can force "Plan mode" by pressing shift-tab. That will prevent it from eagerly implementing stuff.
  - jaggederest 1y ago
    > That will prevent it from eagerly implementing stuff. In theory. In practice, it's not a very secure sandbox and Claude will happily go around updating files if you insist / the prompt is bad / it goes off on a tangent. I really should just set up a completely sandboxed VM for it so that I don't care if it goes rm-rf happy.
    - adastra22 1y ago
      Plan mode disabled the tools, so I don’t see how it would do that. A sandboxed devcontainer is worth setting up though. Lets me run it with —dangerously-skip-permissions
      - jaggederest 1y ago
        I don't know either but I've seen it write to files in plan mode. Very confusing.
dontlaugh 1y ago
At that point, why not just write the code yourself?
- lucasyvas 1y ago
  I reached this conclusion pretty quickly. With all the hand holding I can write it faster - and it’s not bragging, almost anyone experienced here could do the same. Writing the code is the fast and easy part once you know what you want to do. I use AI as a rubber duck to shorten that cycle, then write it myself.
  - 2muchcoffeeman 1y ago
    I’ve been trapped in a hole of “can I get the agent to do this?” And the change would have taken me 1/10th the time. Choosing the battles to pick is part of the skill at the moment. I use AI for a lot of boiler plate, tedious tasks I can’t quite do a vim recording for, small targeted scripts.
    - skydhash 1y ago
      How many of these boilerplate do you actually have to do? Any script or complicated command that I had to write was worthy to be recorded in some bash alias or preserved somewhere. But they mostly live in my bash history or right next to the project. The boilerplate argument is becoming quite old.
      - indiosmo 1y ago
        One recent example of boilerplate for me is I’ve been writing dbt models and I get it to write the schema.yml file for me based on the sql. It’s basically just a translation, but with dozens of tables, each with dozens of columns it gets tedious pretty fast. If given other files from the project as context it’s also pretty good at generating the table and column descriptions for documentation, which I would probably just not write at all if doing it by hand.
      - 2muchcoffeeman 1y ago
rco8786 1y ago
I feel like I do all of this stuff and still end up with unusable code in most cases, and the cases where I don't I still usually have to hand massage it into something usable. Sometimes it gets it right and it's really cool when it does, but anecdotally for me it doesn't seem to be making me any more efficient.
- jaggederest 1y ago
  The key is prompting. Prompt to within an inch of your life. Treat prompts as source code - edit them in files, use @ notation to bring them into the console. Use Claude to generate its own prompts - https://github.com/wshobson/commands/ https://github.com/wshobson/commands/ and https://github.com/wshobson/agents/ https://github.com/wshobson/agents/ are very handy, they include a prompt-engineer persona. I'm at the point now where I have to yell at the AI once in a while, but I touch essentially zero code manually, and it's acceptable quality. Once I stopped and tried to fully refactor a commit that CC had created, but I was only able to make marginal improvements in return for an enormous time commitment. If I had spent that time improving my prompts and running refactoring/cleanup passes in CC, I suspect I would have come out ahead. So I'm deliberately trying not to do that. I expect at some point on a Friday (last Friday was close) I will get frustrated and go build things manually. But for now it's a cognitive and effort reduction for similar quality. It helps to use the most standard libraries and languages possible, and great tests are a must. Edit: Also, use the "thinking" commands. think / think hard / think harder / ultrathink are your best friend when attempting complicated changes (of course, if you're attempting complicated changes, don't.)
  - shaunxcode 1y ago
    I am convinced that this comment once read aloud in the cadence of Ginsberg is a work of art!
    - jaggederest 1y ago
      Now I'm trying to find a text-to-Ginsberg translator. Maybe he's who I sound like in my head.
plaguuuuuu 1y ago
I've been using a few LLMs/agents for a while and I still struggle with getting useful output from it. In order for it not to do useless stuff I need to expend more energy on prompting than writing stuff myself. I find myself getting paranoid about minutia in the prompt, turns of phrase, unintended associations in case it gives shit-tier code because my prompt looked too much like something off experts-exchange or whatever. What I really want is something like a front-end framework but for LLM prompting, that takes away a lot of the fucking about with generalised stuff like prompt structure, default to best practices for finding something in code, or designing a new feature, or writing tests..
- Mars008 1y ago
  > What I really want is something like a front-end framework but for LLM prompting It's not simple to even imagine ideal solution. The more you think about it the more complicated your solution becomes. Simple solution will be restricted to your use cases. Generic is either visual or a programming language. I's like to have visual constructor, graph of actions, but it's complicated. The language is more powerful.
salty_frog 1y ago
This is my algorithm for wetware llms.
ants_everywhere 1y ago
IMO by far the best improvement would be to make it easier for the agent to force the agent to use a success criterion. Right now it's not easy prompting claude code (for example) to keep fixing until a test suite passes. It always does some fixed amount of work until it feels it's most of the way there and stops. So I have to babysit to keep telling it that yes I really mean for it to make the tests pass.
com2kid 1y ago
> 1) Don't ask for large / complex change. Ask for a plan but ask it to implement the plan in small steps and ask the model to test each step before starting the next. I asked Claude Code to read a variable from a .env file. It proceeded to write a .env parser from scratch. I then asked it to just use Node's built in .env file parsing.... This was the 2nd time in the same session that it wrote a .env file parser from scratch. :/ Claude Code is amazing, but it'll goes off and does stupid even for simple requests.
- NitpickLawyer 1y ago
  Check your settings, they might be unable to read .env files as a guardrail.
  - com2kid 1y ago
    I just reminded it to use the built in .env support and it did the right thing. If you ignore that I had to pay for its initial failure...
- theshrike79 1y ago
  It doesn't say no. For me it built a full-ass YAML parser when it couldn't use Viper to parse the configuration correctly :) It was a fully vibe-coded project (I like playing stupid and seeing what the LLM does), but it got caught when the config got a bit more complex and its shitty regex-yaml-parser didn't work anymore. :)
biggc 1y ago
Thin sounds a lot like making a change yourself.
- therein 1y ago
  It appeals to some people because they'd rather manage a bot and get it to do something they told it to do rather than do it themselves.
whateveracct 1y ago
that sounds like just coding it yourself with extra steps
- baq 1y ago
  Exactly, then you launch ten copies of yourself and write code to manage that yourself, maybe.
ccorcos 1y ago
Seems like this logic could all be represented in Claude.md and some agents. Has anyone done this? I’d love to just import that into my project because I’m using some of these tactics but it’s fairly manual and tedious.
renegat0x0 1y ago
Huh, I thought that AI was made to be magic. Click and it generates code. Turns out it is like magic, but you are an apprentice, and still have to learn how to wield it.
- dotancohen 1y ago
  All sufficiently advanced technology...

jason_zig 1y ago

I've seen people post this same advice and I agree with you that it works but you would think they would absorb this common strategy and integrate it as part of the underlying product at this point...

tombot 1y ago
Claude Code at least now lets you use its best model for planning mode and its cheapest model for coding mode.
- candiddevmike 1y ago
  The consulting world parallels here are funny
  - baq 1y ago
    Humans are agents after all
noosphr 1y ago
The people who build the models don't understand how to use the models. It's like asking people who design CPUs to build data-centers. I've interviewed with three tier one AI labs and _no-one_ I talked to had any idea where the business value of their models came in. Meanwhile Chinese labs are releasing open source models that do what you need. At this point I've build local agentic tools that are better than anything Claude and OAI have as paid offerings, including the $2,000 tier. Of course they cost between a few dollars to a few hundred dollars per query so until hardware gets better they will stay happily behind corporate moats and be used by the people blessed to burn money like paper.
- Barbing 1y ago
  Very interesting. And plausible. Doesn't specifically seem to jive with the claim Anthropic made where they were worried about Claude Code being their secret sauce, leaving them unsure whether to publicly release it. (I know some skeptical about that claim.)
- criemen 1y ago
  > The people who build the models don't understand how to use the models. It's like asking people who design CPUs to build data-centers. This doesn't match the sentiment on hackernews and elsewhere that claude code is the superior agentic coding tool, as it's developed by one of the AI labs, instead of a developer tool company.
  - noosphr 1y ago
    Claude code is babies first agentic tool. You don't see better ones from code tooling companies because the economics don't work out. No one is going to pay $1,000 for a two line change on a 500,000k line code base after waiting four hours. LLMs today the equivalent of a 4bit ALU without memory being sold as a fully functional personal computer. And like ALUs today, you will need _thousands_ of LLMs to get anything useful done, also like ALUs in 1950 we're a long way off from a personal computer being possible.
    - 1y ago
nostrademons 1y ago
A lot of it is integrated into the product at this point. If you have a particularly tricky bug, you can just tell Claude "I have this bug. I expected output 'foo' and got output 'bar'. What went wrong?" It will inspect the code and sometimes suggest a fix. If you run it and it still doesn't work, you can say "Nope, still not working", and Claude will add debug output to the whole program, tell you to run it again, and paste the debug output back into the console. Then it will use your example to write tests, and run against them.

deleted 1y ago

[deleted]

paulcole 1y ago

> Ask for a plan but ask it to implement the plan in small steps and ask the model to test each step before starting the next. Tried this on a developer I worked with once and he just scoffed at me and pushed to prod on a Friday.

NitpickLawyer 1y ago
> scoffed at me and pushed to prod on a Friday. that's the --yolo flag in cc :D

rvnx 1y ago

Your tips are perfect. Most users will just give a vague tasks like: "write a clone of Steam" or "create a rocket" and then they blame Claude Code. If you want AI to code for you, you have to decompose your problem like a product owner would do. You can get helped by AI as well, but you should have a plan and specifications. Once your plan is ready, you have to decompose the problem into different modules, then make sure each modules are tested. The issue is often with the user, not the tool, as they have to learn how to use the tool first.

wordofx 1y ago
> Most users will just give a vague tasks like: "write a clone of Steam" or "create a rocket" and then they blame Claude Code. This seems like half of HN with how much HN hates AI. Those who hate it or say it’s not useful to them seem to be fighting against it and not wanting to learn how to use it. I still haven’t seen good examples of it not working even with obscure languages or proprietary stuff.
- LtWorf 1y ago
  If you have to iterate 10 times, that is "not working", since it already wasted way more time than doing it manually to begin with.
- drzaiusx11 1y ago
  Anyone who has mentored as part of a junior engineer internship program AND has attempted to use current gen ai tooling will notice the parallels immediately. There are key differences though that are worth highlighting. The main difference is that with the current batch of genai tools, the AI's context resets after use, whereas a (good) intern truly learns from prior behavior. Additionally, as you point out, the language and frameworks need to be part of the training set since AI isn't really "learning" it's just prepolulating a context window for its pre-existing knowledge (token prediction), so ymmv depending on hidden variables from the secret (to you, the consumers) training data and weights. I use Ruby primarily these days, which is solidly in the "boring tech" camp and most AIs fail to produce useful output that isn't rails boilerplate. If I did all my IC contributions via directed intern commits I'd leave the industry out of frustration. Using only AI outputs for producing code changes would be akin to torture (personally.) Edit: To clarify I'm not against AI use, I'm just stating that with the current generation of tools it is a pretty lackluster experience when it comes to net new code generation. It excells at one off throwaway scripts and making large tedious redactors less drudgerly. I wouldn't pivot to it being my primary method of code generation until some of the more blatant productiviy losses are addressed.

MikeTheGreat 1y ago

Genuine question: What do you mean by " ask it to implement the plan in small steps"? One option is to write "Please implement this change in small steps?" more-or-less exactly Another option is to figure out the steps and then ask it "Please figure this out in small steps. The first step is to add code to the parser so that it handles the first new XML element I'm interested in, please do this by making the change X, we'll get to Y and Z later" I'm sure there's other options, too.

Benjammer 1y ago
My method is that I work together with the LLM to figure out the step-by-step plan. I give an outline of what I want to do, and give some breadcrumbs for any relevant existing files that are related in some way, ask it to figure out context for my change and to write up a summary of the full scope of the change we're making, including an index of file paths to all relevant files with a very concise blurb about what each file does/contains, and then also to produce a step-by-step plan at the end. I generally always have to tell it to NOT think about this like a traditional engineering team plan, this is a senior engineer and LLM code agent working together, think only about technical architecture, otherwise you get "phase 1 (1-2 weeks), phase 2 (2-4 weeks), step a (4-8 hours)" sort of nonsense timelines in your plan. Then I review the steps myself to make sure they are coherent and make sense, and I poke and prod the LLM to fix anything that seems weird, either fixing context or directions or whatever. Then I feed the entire document to another clean context window (or two or three) and ask it to "evaluate this plan for cohesiveness and coherency, tell me if it's ready for engineering or if there's anything underspecified or unclear" and iterate on that like 1-3 times until I run a fresh context window and it says "This plan looks great, it's well crafted, organized, etc...." and doesn't give feedback. Then I go to a fresh context window and tell it "Review the document @MY_PLAN.md thoroughly and begin implementation of step 1, stop after step 1 before doing step 2" and I start working through the steps with it.
- lkjdsklf 1y ago
  The problem is, by the time you’ve gone through the process of making a granular plan and all that, you’ve lost all productivity gains of using the agent. As an engineer, especially as you get more experience, you can kind of visualize the plan for a change very quickly and flesh out the next step while implementing the current step All you have really accomplished with the kind of process described is make the worlds least precise, most verbose programming language

adastra22 1y ago

This is why the jobs market for new grads and early career folks has dried up. A seasoned developer knows that this is how you manage work in general, and just treats the AI like they would a junior developer—and gets good results.

CuriouslyC 1y ago
Why bother handing stuff to a junior when an agent will do it faster while asking fewer questions, and even if the first draft code isn't amazing, you can just quality gate with an LLM reviewer that has been instructed to be brutal and do a manual pass when the code gets by the LLM reviewer.
- LtWorf 1y ago
  Because juniors learn while LLMs don't and you must explain the same thing over and over forever.
  - adastra22 1y ago
    If you are explaining things more than once, you are doing it wrong. Which is not on you as the tools currently suck big time. But it is quite possible to have LLM agents “learn” by intelligently matching context (including historical lessons learned) to conversation.

MangoCoffee 1y ago

I've been vibe coding a couple of personal projects. I've found that test-driven development fits very well with vibe coding, and it's just as you said break up the problem into small, testable chunks, get the AI to write unit tests first, and then implement the actual code

yodsanklai 1y ago
Actually, all good engineering principles which reduce cognitive load for humans work for AI as well.
- colordrops 1y ago
  This is the big secret. Keep code modular, small, single purpose, encapsulated, and it works great with vibe coding. I want to write a protocol/meta language similar to the markdown docs that Claude et al create that is per module, and defines behavior, so you actually program and compose modules with well defined interfaces in natural language. I'm surprised someone hasn't done it already.
  - drzaiusx11 1y ago
    Isn't what you're describing exactly what Kiro aims to solve?
    - colordrops 1y ago
      Possibly, I've never heard of Kiro, will look into it.
  - adastra22 1y ago
    My set of Claude agent files have an explicit set of interface definitions. Is that what you’re talking about?
    - colordrops 1y ago
      Are Claude agent files per module? If so, then I guess so.

rmonvfer 1y ago

I’d like to add: keep some kind of development documentation where you describe in detail the patterns and architecture of your application and it’s components. I’ve seen incredible improvements just by doing this and using precise prompting to get Claude to implement full services by itself, tests included. Of course it requires manual correction later but just telling Claude to check the development documentation before starting work on a feature prevents most hallucinations (that and telling it to use the Context7 MCP for external documentation), at least in my experience. The downside to this is that 30% of your context window will be filled with documentation but hey, at least it won’t hallucinate API methods or completely forget that it shouldn’t reimplement something. Just my 2 cents.

deleted 1y ago

[deleted]

nostrademons 1y ago

I've found that an effective tactic for larger, more complex tasks is to tell it "Don't write any code now. I'm going to describe each of the steps of the problem in more detail. The rough outline is going to be 1) Read this input 2) Generate these candidates 3) apply heuristics to score candidates 4) prioritize and rank candidates 5) come up with this data structure reflecting the output 6) write the output back to the DB in this schema". Claude will then go and write a TODO list in the code (and possibly claude.md if you've run /init), and prompt you for the details of each stage. I've even done this for an hour, told Claude "I have to stop now. Generate code for the finished stages and write out comments so you can pick up where you left off next time" and then been able to pick up next time with minimal fuss.

hex4def6 1y ago
FYI: You can force "Plan mode" by pressing shift-tab. That will prevent it from eagerly implementing stuff.
- jaggederest 1y ago
  > That will prevent it from eagerly implementing stuff. In theory. In practice, it's not a very secure sandbox and Claude will happily go around updating files if you insist / the prompt is bad / it goes off on a tangent. I really should just set up a completely sandboxed VM for it so that I don't care if it goes rm-rf happy.
  - adastra22 1y ago
    Plan mode disabled the tools, so I don’t see how it would do that. A sandboxed devcontainer is worth setting up though. Lets me run it with —dangerously-skip-permissions
    - jaggederest 1y ago
      I don't know either but I've seen it write to files in plan mode. Very confusing.

dontlaugh 1y ago

At that point, why not just write the code yourself?

lucasyvas 1y ago
I reached this conclusion pretty quickly. With all the hand holding I can write it faster - and it’s not bragging, almost anyone experienced here could do the same. Writing the code is the fast and easy part once you know what you want to do. I use AI as a rubber duck to shorten that cycle, then write it myself.
- 2muchcoffeeman 1y ago
  I’ve been trapped in a hole of “can I get the agent to do this?” And the change would have taken me 1/10th the time. Choosing the battles to pick is part of the skill at the moment. I use AI for a lot of boiler plate, tedious tasks I can’t quite do a vim recording for, small targeted scripts.
  - skydhash 1y ago
    How many of these boilerplate do you actually have to do? Any script or complicated command that I had to write was worthy to be recorded in some bash alias or preserved somewhere. But they mostly live in my bash history or right next to the project. The boilerplate argument is becoming quite old.
    - indiosmo 1y ago
      One recent example of boilerplate for me is I’ve been writing dbt models and I get it to write the schema.yml file for me based on the sql. It’s basically just a translation, but with dozens of tables, each with dozens of columns it gets tedious pretty fast. If given other files from the project as context it’s also pretty good at generating the table and column descriptions for documentation, which I would probably just not write at all if doing it by hand.
    - 2muchcoffeeman 1y ago

rco8786 1y ago

I feel like I do all of this stuff and still end up with unusable code in most cases, and the cases where I don't I still usually have to hand massage it into something usable. Sometimes it gets it right and it's really cool when it does, but anecdotally for me it doesn't seem to be making me any more efficient.

jaggederest 1y ago
The key is prompting. Prompt to within an inch of your life. Treat prompts as source code - edit them in files, use @ notation to bring them into the console. Use Claude to generate its own prompts - https://github.com/wshobson/commands/ https://github.com/wshobson/commands/ and https://github.com/wshobson/agents/ https://github.com/wshobson/agents/ are very handy, they include a prompt-engineer persona. I'm at the point now where I have to yell at the AI once in a while, but I touch essentially zero code manually, and it's acceptable quality. Once I stopped and tried to fully refactor a commit that CC had created, but I was only able to make marginal improvements in return for an enormous time commitment. If I had spent that time improving my prompts and running refactoring/cleanup passes in CC, I suspect I would have come out ahead. So I'm deliberately trying not to do that. I expect at some point on a Friday (last Friday was close) I will get frustrated and go build things manually. But for now it's a cognitive and effort reduction for similar quality. It helps to use the most standard libraries and languages possible, and great tests are a must. Edit: Also, use the "thinking" commands. think / think hard / think harder / ultrathink are your best friend when attempting complicated changes (of course, if you're attempting complicated changes, don't.)
- shaunxcode 1y ago
  I am convinced that this comment once read aloud in the cadence of Ginsberg is a work of art!
  - jaggederest 1y ago
    Now I'm trying to find a text-to-Ginsberg translator. Maybe he's who I sound like in my head.

plaguuuuuu 1y ago

I've been using a few LLMs/agents for a while and I still struggle with getting useful output from it. In order for it not to do useless stuff I need to expend more energy on prompting than writing stuff myself. I find myself getting paranoid about minutia in the prompt, turns of phrase, unintended associations in case it gives shit-tier code because my prompt looked too much like something off experts-exchange or whatever. What I really want is something like a front-end framework but for LLM prompting, that takes away a lot of the fucking about with generalised stuff like prompt structure, default to best practices for finding something in code, or designing a new feature, or writing tests..

Mars008 1y ago
> What I really want is something like a front-end framework but for LLM prompting It's not simple to even imagine ideal solution. The more you think about it the more complicated your solution becomes. Simple solution will be restricted to your use cases. Generic is either visual or a programming language. I's like to have visual constructor, graph of actions, but it's complicated. The language is more powerful.

salty_frog 1y ago

This is my algorithm for wetware llms.

ants_everywhere 1y ago

IMO by far the best improvement would be to make it easier for the agent to force the agent to use a success criterion. Right now it's not easy prompting claude code (for example) to keep fixing until a test suite passes. It always does some fixed amount of work until it feels it's most of the way there and stops. So I have to babysit to keep telling it that yes I really mean for it to make the tests pass.

com2kid 1y ago

> 1) Don't ask for large / complex change. Ask for a plan but ask it to implement the plan in small steps and ask the model to test each step before starting the next. I asked Claude Code to read a variable from a .env file. It proceeded to write a .env parser from scratch. I then asked it to just use Node's built in .env file parsing.... This was the 2nd time in the same session that it wrote a .env file parser from scratch. :/ Claude Code is amazing, but it'll goes off and does stupid even for simple requests.

NitpickLawyer 1y ago
Check your settings, they might be unable to read .env files as a guardrail.
- com2kid 1y ago
  I just reminded it to use the built in .env support and it did the right thing. If you ignore that I had to pay for its initial failure...
theshrike79 1y ago
It doesn't say no. For me it built a full-ass YAML parser when it couldn't use Viper to parse the configuration correctly :) It was a fully vibe-coded project (I like playing stupid and seeing what the LLM does), but it got caught when the config got a bit more complex and its shitty regex-yaml-parser didn't work anymore. :)

biggc 1y ago

Thin sounds a lot like making a change yourself.

therein 1y ago
It appeals to some people because they'd rather manage a bot and get it to do something they told it to do rather than do it themselves.

whateveracct 1y ago

that sounds like just coding it yourself with extra steps

baq 1y ago
Exactly, then you launch ten copies of yourself and write code to manage that yourself, maybe.

ccorcos 1y ago

Seems like this logic could all be represented in Claude.md and some agents. Has anyone done this? I’d love to just import that into my project because I’m using some of these tactics but it’s fairly manual and tedious.

renegat0x0 1y ago

Huh, I thought that AI was made to be magic. Click and it generates code. Turns out it is like magic, but you are an apprentice, and still have to learn how to wield it.

dotancohen 1y ago
All sufficiently advanced technology...