6 ms·

The problem, in my experience is: 1. It writes SO MUCH code, in minutes, that theres no way I as a human can review it. 2. I am not very motivated to read/rev

by ghoul2 20d ago

The problem, in my experience is: 1. It writes SO MUCH code, in minutes, that theres no way I as a human can review it. 2. I am not very motivated to read/review this code anyway: it was cheaply written, by some _thing_ that is not going to improve from my feedback. 3. If you DO review it conscientiously , it becomes a never ending thing: you keep finding issue upon issue. 4. If you report the issues for fixes, the fixes normally fix that immediate issue, and typically add another parallel path/another option/another 100-odd lines of code, instead of a structural fix. 5. If the code base is large, and if you let AI write a meaningful amount of code in it, its no longer _your_ code. You lose the depth+width of understanding needed to reason thru things mentally, cause you no longer KNOW enough about the code. 6. If you do review seriously, and either fix things yourself, or have the ai/harness fix the issues for you, the rounds of fixes take so long, if you look back you realize it would have been better to just do it yourself in the first place. My personal opinion there are really only two choices: A. If you want to build FAST, using an agent, just let the agent write tests, validations, extensively, have it keep running them (it likes to call them gates), and just let it loose. Give up on the idea that its your code, and that you understand it. The bottom line becomes: does it work, and WHEN it breaks in weird ways, just use the agent to find and fix the issue (probably breaking something else in the process). B. limit AI use for non trivial, prod-quality projects to limited research, very tiny targeted changes, write most of the code yourself still, and review every line. You won't get much speedup, maybe 20-30%, but it will still be YOUR code, and you will still be able to reason about it.

t-writescode 20d ago
Route A seems genuinely horrifying to me - maybe not for small and/or helper scripts; but for anything substantial. For anything that bites you or someone else hard when things go bad. I don’t want a vibe-coded, mass-produced diabetes tracking app, or banking tool, or tax management software. I don’t want a vibe-coded power grid analysis software. I don’t care if a person’s little, local scripting thing is vibe-coded. I don’t care if an artist’s 0->1 game code is vibe-coded; but that’s not what a lot of the things we’re talking about here really are.
- anon373839 20d ago
  Route A is good for rapid, disposable prototyping. If you ever have a stray thought, “I wonder how this would work if the whole paradigm were turned sideways”, you now have a chance to preview a “working” version of your idea. If you like it, discard the code and reimplement it correctly. In this way, I think it can be a good adjunct to sketches and other lofi prototyping techniques. Just don’t outsource the creative ideation to the LLM, because all you’ll get are the same solutions as everyone else.
  - t-writescode 20d ago
    Have you seen many of the replies in this thread and others? They’re not doing that. They’re putting it into production code. Their company’s thought leaders are trying to say that review itself is a waste of time, etc. That’s not “disposable prototyping” that’s whole versions of the codebase written with barely a human in sight.
    - anon373839 20d ago
      Yes; my point was not related to any of that.
- nektro 20d ago
  Route A is what concretely what is behind those "a select few are seeing 100x productivity increases" posts. agents bring enterprise-level engineering to the IC. your average enterprise PM has no idea how the code works either but knows who to call when it breaks or needs a new feature added. the industry split on agentic use lies in how much and to what degree you believe "but AI is different!" in this analogy
othmanosx 20d ago
You should be the one building the gates then, but those gates should be mechanical and deterministic so the AI doesn’t wprk around them. Lint rules, type errors, commit lint, anything that tells the AI to stop instead of allowing workarounds.
- ghoul2 20d ago
  > Lint rules, type errors, commit lint, anything These are basic: they work for human coders and are typically quick to setup. But gates at this level are far from enough for agentic coding. You need an extensive test suite/e2e suite/benchmark suite. Typically much larger than the code base itself: This is what enables those "i ported bun to rust in a weekend" headlines. Where ever such extensive 'gates', high quality ones, already exist, the agent can do a good job of making things work, cause it can automatically iterate. But: every hole, every gap in coverage, will be eventually found by the coding agent and 'exploited'. So you need full, extensive coverage. This suite itself is a LOT of work, much more than what it would be if you only had to worry about human coders. So if YOU are writing the gates yourself, you are going to fall between A and B: you still have a convoluted, unreliable, impossible-to-reason-about prod app, AND it still took you eons to build it cause you spent that time writing the gates. This is no win at all, on either front. Where gates, extensive ones, help in route A is they can significantly reduce the churn/spinning, fix-x-break-y, to just the parts that are gaps in coverage (which the agent will keep finding at a ridiculous rate). And your work would be to have it ALSO keep filing in those gaps with more tests/etc. One of the reasons go coding with agents works better than other langs is cause I even include AST based enforcements in my governance suite. Its still route A, but with somewhat more confidence, somewhat less frustration.
  - t-writescode 19d ago
    I mean, from my training long ago, a good test suite is usually / always a few multiples in size of the codebase. 100k lines of app code? 300-500k lines of test, sort of thing.
    - ghoul2 19d ago

by ghoul2 20d ago

t-writescode 20d ago

Route A seems genuinely horrifying to me - maybe not for small and/or helper scripts; but for anything substantial. For anything that bites you or someone else hard when things go bad. I don’t want a vibe-coded, mass-produced diabetes tracking app, or banking tool, or tax management software. I don’t want a vibe-coded power grid analysis software. I don’t care if a person’s little, local scripting thing is vibe-coded. I don’t care if an artist’s 0->1 game code is vibe-coded; but that’s not what a lot of the things we’re talking about here really are.

anon373839 20d ago
Route A is good for rapid, disposable prototyping. If you ever have a stray thought, “I wonder how this would work if the whole paradigm were turned sideways”, you now have a chance to preview a “working” version of your idea. If you like it, discard the code and reimplement it correctly. In this way, I think it can be a good adjunct to sketches and other lofi prototyping techniques. Just don’t outsource the creative ideation to the LLM, because all you’ll get are the same solutions as everyone else.
- t-writescode 20d ago
  Have you seen many of the replies in this thread and others? They’re not doing that. They’re putting it into production code. Their company’s thought leaders are trying to say that review itself is a waste of time, etc. That’s not “disposable prototyping” that’s whole versions of the codebase written with barely a human in sight.
  - anon373839 20d ago
    Yes; my point was not related to any of that.
nektro 20d ago
Route A is what concretely what is behind those "a select few are seeing 100x productivity increases" posts. agents bring enterprise-level engineering to the IC. your average enterprise PM has no idea how the code works either but knows who to call when it breaks or needs a new feature added. the industry split on agentic use lies in how much and to what degree you believe "but AI is different!" in this analogy

othmanosx 20d ago

You should be the one building the gates then, but those gates should be mechanical and deterministic so the AI doesn’t wprk around them. Lint rules, type errors, commit lint, anything that tells the AI to stop instead of allowing workarounds.

ghoul2 20d ago
> Lint rules, type errors, commit lint, anything These are basic: they work for human coders and are typically quick to setup. But gates at this level are far from enough for agentic coding. You need an extensive test suite/e2e suite/benchmark suite. Typically much larger than the code base itself: This is what enables those "i ported bun to rust in a weekend" headlines. Where ever such extensive 'gates', high quality ones, already exist, the agent can do a good job of making things work, cause it can automatically iterate. But: every hole, every gap in coverage, will be eventually found by the coding agent and 'exploited'. So you need full, extensive coverage. This suite itself is a LOT of work, much more than what it would be if you only had to worry about human coders. So if YOU are writing the gates yourself, you are going to fall between A and B: you still have a convoluted, unreliable, impossible-to-reason-about prod app, AND it still took you eons to build it cause you spent that time writing the gates. This is no win at all, on either front. Where gates, extensive ones, help in route A is they can significantly reduce the churn/spinning, fix-x-break-y, to just the parts that are gaps in coverage (which the agent will keep finding at a ridiculous rate). And your work would be to have it ALSO keep filing in those gaps with more tests/etc. One of the reasons go coding with agents works better than other langs is cause I even include AST based enforcements in my governance suite. Its still route A, but with somewhat more confidence, somewhat less frustration.
- t-writescode 19d ago
  I mean, from my training long ago, a good test suite is usually / always a few multiples in size of the codebase. 100k lines of app code? 300-500k lines of test, sort of thing.
  - ghoul2 19d ago