5 ms·
Can you share what tools and processes you're using to do this? I've been using Pi to build custom extensions and wrapping workflows in shell processes to make
by tempoponet 8d ago
Can you share what tools and processes you're using to do this?
I've been using Pi to build custom extensions and wrapping workflows in shell processes to make it more deterministic and enforce certain validations, all guided by Fable. This isn't production work, though, just playing llm factorio at home.
- pimeys 7d agoWhat you want is a bunch of sessions to replay. Something anonymized if it's not yours, and something that's not depending on state. You replay all your sessions against your harness, and then store all logs all output, everything to a safe place. Finally use a blind judge to check everything, and score the output. Then fix your harness, iterate again until better until you are in a point where it's just the model's weakness. If you get to that, use a bigger model.