4 ms·
Headless agentic coding only works when: * The scope is constrained and clearly defined * You articulate an exhaustive and objective set of success criteria
by 10000truths 23d ago
Headless agentic coding only works when:
* The scope is constrained and clearly defined
* You articulate an exhaustive and objective set of success criteria
* You provide a deterministic tool for the AI to evaluate its output against those criteria and receive feedback (compiler errors, unit tests, regression tests, etc.)
You're essentially writing a comprehensive spec for the AI to follow. The time suck was always from making the spec and the QA bulletproof.
- apatheticonion 23d agoEven then, it tends to code itself into a corner. Code tends to be ridged and difficult to expand. One shot benchmarks that have exhaustive prompts are impressive, but benchmarks that emulate real software development (iterative changes over time with changing requirements) have close-to-zero pass rates.
- 10000truths 23d agoWell yes, if the thing you're working on is subject to changes in requirements, then you fundamentally can't one-shot it, AI or no AI. That's why the waterfall process fell out of fashion long ago. But there are cases where there are no changes, or only backwards-compatible changes. Take the core WebAssembly specification, for example. Extremely detailed, formally verified, comes with a large test suite. Someone else has done the hard work of specifying the behavior of every possible edge case. A WASM runtime implementer can point their agent to that spec, plus some prior art. The remaining manual work is in constructing a test harness and collecting a corpus of test WASM binaries that exercises decent coverage.