4 ms·
We created a self-improving, spec-driven delivery harness to test whether frontier models can port complete systems.
by TylerJewell 15d ago
We created a self-improving, spec-driven delivery harness to test whether frontier models can port complete systems.
- TylerJewell 15d agoThere were a few interesting findings: 1. We were able to execute complete, unattended rewrites of existing systems that pass the original's unit and integration tests. 2. Low effort models were more efficient than high effort models. 3. The delivery harness and its structure had a bigger impact on consistency than model capability. 4. We saw a LOC or performance improvement on 57 of 65 ports.