8 ms·
That kind of one shot capability is impressive but how does it work for my typical work style? The way I work is to build a huge roadmap with goals and hand it
by coder-pm 15d ago
That kind of one shot capability is impressive but how does it work for my typical work style? The way I work is to build a huge roadmap with goals and hand it to my agent to execute (often over night). I don't care that much about the benchmarks, what I care about is how often Fable 5.1 is making a baffling decision and destroys my plan, not respecting stop conditions or goals. I would seek for behavioral reliability over long autonomous runs, not eval scores. Anyone have that kind of feedback and observations?
- visarga 15d agoYou can engineer loops that have it, but it depends on a case by case basis. Does your loop have strong validation? if it's all vibes nothing can stop it from diverging.
- coder-pm 14d agoAgree on the validation, my loops are already gated. My concerns are about the cases when model is passing validation and quietly abandoning the goal. The second scenario is rewriting the plan to fit what was already done.