9 ms·
How do you prevent the agent from reward-hacking the hidden eval? e.g. writing training data that effectively leaks the eval distribution rather than teaching a
by azertyvode 2mo ago
How do you prevent the agent from reward-hacking the hidden eval? e.g. writing training data that effectively leaks the eval distribution rather than teaching a general skill?
- Danau5tin 2mo agoGreat question. Today the answer is that there is nothing stopping this, but you are right this is a 100% valid reward hacking possibility. No signs of it in the run I did, but for any future larger scale runs, it would be wise to dedicate some focus to prevent this.