6 ms·

The biggest challenge i found with LLMs on large codebase is making the same mistakes again and again How do keep track of the architecture decisions in context

The biggest challenge i found with LLMs on large codebase is making the same mistakes again and again How do keep track of the architecture decisions in context of every tasks on the large codebase ?

tom_m 1y ago
Very very clear, unambiguous, prompts and agent rules. Use strong language like "must" and "critical" and "never" etc. I would also try working on smaller sections of a large codebase at a time too if things are too inaccurate. The AI coding tools are going to be looking at other files in the project to help with context. Ambiguity is the death of AI effectiveness. You have to keep things clear and so that may require addressing smaller sections at a time. Unless you can really configure the tools in ways to isolate things. This is why I like tools that have a lot of control and are transparent. If you ask a tool what the full system and user prompt is and it doesn't tell you? Run away from that tool as fast as you can. You need to have introspections here. You have to be able to see what causes a behavior you don't want and be able to correct it. Any tool that takes that away from you is one that won't work.
- sarchertech 1y ago
  > Use strong language like "must" and "critical" and "never" etc. Truly we live in the stupidest timeline. Imagine if you had a domestic robot but when you asked it make you breakfast you had to preface your request with “it’s critical that you don’t kill me.” Or when you asked it to do the laundry you had to remember to tell it that it “must not make its own doors by knocking holes in the wall” and hope that it listens.
  - phs318u 1y ago
    > Truly we live in the stupidest timeline. Wholeheartedly agree. Truly, I look around and see no shortage of evidence for this assertion. EDITED to make it clear I am agreeing with parent.
  - sally_glance 1y ago
    There is even a chance our timeline might include that robot too at some point... Book recommendation no one asked for but which is essentially about some guy living through multiple more or less stupid timelines: Count to Eschaton series by John C. Wright
  - filoeleven 1y ago
    "Better living through prompt fondling."
- mattmanser 1y ago
  I always chuckle at these rain dance posts. INVOKE THE ULTRATHINK OH MIGHTY CLAUDE AND BLESS MY CODE. Have you tried kissing the keyboard before you press enter? It makes the code 123% more flibbeled.
  - alecco 1y ago
    Cool. Added it to the top of all my CLAUDE.md files and my web UI preferences. But I'll kiss the mic instead, as I'm a voice-only vivecoder.
hyperadvanced 1y ago
In some sense “the same mistakes again and again” is either a prompting problem or a “you” problem insofar as your expectations differ from the machine overlords.
loandbehold 1y ago
Whenever I see Claude Code make same mistake multiple times I add instructions to clade.md to avoid it in the future.
athrowaway3z 1y ago
`opencode` will read any amount of `!cmd` output. I start my sessions with something like `!cat ./docs/*` and I can start asking questions. Make sure you regularly ask it to point out any inconsistencies or ambiguity in the docs.
- eddywebs 1y ago
  nice !

The biggest challenge i found with LLMs on large codebase is making the same mistakes again and again How do keep track of the architecture decisions in context of every tasks on the large codebase ?

tom_m 1y ago

Very very clear, unambiguous, prompts and agent rules. Use strong language like "must" and "critical" and "never" etc. I would also try working on smaller sections of a large codebase at a time too if things are too inaccurate. The AI coding tools are going to be looking at other files in the project to help with context. Ambiguity is the death of AI effectiveness. You have to keep things clear and so that may require addressing smaller sections at a time. Unless you can really configure the tools in ways to isolate things. This is why I like tools that have a lot of control and are transparent. If you ask a tool what the full system and user prompt is and it doesn't tell you? Run away from that tool as fast as you can. You need to have introspections here. You have to be able to see what causes a behavior you don't want and be able to correct it. Any tool that takes that away from you is one that won't work.

sarchertech 1y ago
> Use strong language like "must" and "critical" and "never" etc. Truly we live in the stupidest timeline. Imagine if you had a domestic robot but when you asked it make you breakfast you had to preface your request with “it’s critical that you don’t kill me.” Or when you asked it to do the laundry you had to remember to tell it that it “must not make its own doors by knocking holes in the wall” and hope that it listens.
- phs318u 1y ago
  > Truly we live in the stupidest timeline. Wholeheartedly agree. Truly, I look around and see no shortage of evidence for this assertion. EDITED to make it clear I am agreeing with parent.
- sally_glance 1y ago
  There is even a chance our timeline might include that robot too at some point... Book recommendation no one asked for but which is essentially about some guy living through multiple more or less stupid timelines: Count to Eschaton series by John C. Wright
- filoeleven 1y ago
  "Better living through prompt fondling."
mattmanser 1y ago
I always chuckle at these rain dance posts. INVOKE THE ULTRATHINK OH MIGHTY CLAUDE AND BLESS MY CODE. Have you tried kissing the keyboard before you press enter? It makes the code 123% more flibbeled.
- alecco 1y ago
  Cool. Added it to the top of all my CLAUDE.md files and my web UI preferences. But I'll kiss the mic instead, as I'm a voice-only vivecoder.

hyperadvanced 1y ago

In some sense “the same mistakes again and again” is either a prompting problem or a “you” problem insofar as your expectations differ from the machine overlords.

loandbehold 1y ago

Whenever I see Claude Code make same mistake multiple times I add instructions to clade.md to avoid it in the future.

athrowaway3z 1y ago

`opencode` will read any amount of `!cmd` output. I start my sessions with something like `!cat ./docs/*` and I can start asking questions. Make sure you regularly ask it to point out any inconsistencies or ambiguity in the docs.

eddywebs 1y ago
nice !