8 ms·

You should be the one building the gates then, but those gates should be mechanical and deterministic so the AI doesn’t wprk around them. Lint rules, type erro

You should be the one building the gates then, but those gates should be mechanical and deterministic so the AI doesn’t wprk around them. Lint rules, type errors, commit lint, anything that tells the AI to stop instead of allowing workarounds.

ghoul2 19d ago
> Lint rules, type errors, commit lint, anything These are basic: they work for human coders and are typically quick to setup. But gates at this level are far from enough for agentic coding. You need an extensive test suite/e2e suite/benchmark suite. Typically much larger than the code base itself: This is what enables those "i ported bun to rust in a weekend" headlines. Where ever such extensive 'gates', high quality ones, already exist, the agent can do a good job of making things work, cause it can automatically iterate. But: every hole, every gap in coverage, will be eventually found by the coding agent and 'exploited'. So you need full, extensive coverage. This suite itself is a LOT of work, much more than what it would be if you only had to worry about human coders. So if YOU are writing the gates yourself, you are going to fall between A and B: you still have a convoluted, unreliable, impossible-to-reason-about prod app, AND it still took you eons to build it cause you spent that time writing the gates. This is no win at all, on either front. Where gates, extensive ones, help in route A is they can significantly reduce the churn/spinning, fix-x-break-y, to just the parts that are gaps in coverage (which the agent will keep finding at a ridiculous rate). And your work would be to have it ALSO keep filing in those gaps with more tests/etc. One of the reasons go coding with agents works better than other langs is cause I even include AST based enforcements in my governance suite. Its still route A, but with somewhat more confidence, somewhat less frustration.
- t-writescode 19d ago
  I mean, from my training long ago, a good test suite is usually / always a few multiples in size of the codebase. 100k lines of app code? 300-500k lines of test, sort of thing.
  - ghoul2 19d ago
    Exactly. I am now looking at 9 to 10x the app code, for non trivial stuff. And this is not just the usual unit/integration/e2e suites too. Extensive deadcode, field use, code shape tests (ast walking) - god objects, 15-param helpers, wrapper-piled-upon-wrapper, badly named (a skill) functions/methods/objects, path-dependent artifacts, etc. There is no way I could have written all this manually. It doesn't guarentee high quality OR reliability or readability or maintainability, but does reduce churn and makes me feel a bit more confident about deploying route A work product in prod.
  - othmanosx 19d ago
    Tests are also a good mechanical and deterministic guardrails, but they're written by the same AI that wrote the code then what's the difference?

ghoul2 19d ago

> Lint rules, type errors, commit lint, anything These are basic: they work for human coders and are typically quick to setup. But gates at this level are far from enough for agentic coding. You need an extensive test suite/e2e suite/benchmark suite. Typically much larger than the code base itself: This is what enables those "i ported bun to rust in a weekend" headlines. Where ever such extensive 'gates', high quality ones, already exist, the agent can do a good job of making things work, cause it can automatically iterate. But: every hole, every gap in coverage, will be eventually found by the coding agent and 'exploited'. So you need full, extensive coverage. This suite itself is a LOT of work, much more than what it would be if you only had to worry about human coders. So if YOU are writing the gates yourself, you are going to fall between A and B: you still have a convoluted, unreliable, impossible-to-reason-about prod app, AND it still took you eons to build it cause you spent that time writing the gates. This is no win at all, on either front. Where gates, extensive ones, help in route A is they can significantly reduce the churn/spinning, fix-x-break-y, to just the parts that are gaps in coverage (which the agent will keep finding at a ridiculous rate). And your work would be to have it ALSO keep filing in those gaps with more tests/etc. One of the reasons go coding with agents works better than other langs is cause I even include AST based enforcements in my governance suite. Its still route A, but with somewhat more confidence, somewhat less frustration.

t-writescode 19d ago
I mean, from my training long ago, a good test suite is usually / always a few multiples in size of the codebase. 100k lines of app code? 300-500k lines of test, sort of thing.
- ghoul2 19d ago
  Exactly. I am now looking at 9 to 10x the app code, for non trivial stuff. And this is not just the usual unit/integration/e2e suites too. Extensive deadcode, field use, code shape tests (ast walking) - god objects, 15-param helpers, wrapper-piled-upon-wrapper, badly named (a skill) functions/methods/objects, path-dependent artifacts, etc. There is no way I could have written all this manually. It doesn't guarentee high quality OR reliability or readability or maintainability, but does reduce churn and makes me feel a bit more confident about deploying route A work product in prod.
- othmanosx 19d ago
  Tests are also a good mechanical and deterministic guardrails, but they're written by the same AI that wrote the code then what's the difference?