mrifaki

5 ms·

1. 1.▲
   by mrifaki 5mo ago

   the adaptive thinking complaints in this thread are interesting because they are basically the same verifier quality problem showing up in a different costume the model has to decide how hard to think before knowing how hard the problem is
2. 2.▲
   by mrifaki 5mo ago

   this is atctually he reward hacking problem from RL showing up in evaluation infra which is not surprising but worth naming clearly, an interesting question raised here is whether agents start doing this on their own and from an RL perspect
3. 3.▲
   by mrifaki 5mo ago

   finding vulns in a large codebase is a search problem with a huge negative space and what aisle measured is classification accuracy on ground-truth positives, those are different tasks so a model that correctly labels a pre-isolated vulnera