11 ms·
I mean, I'm interested to know if the frontier models also get to see all questions at once. Then it's more fair game than if they just see one question at a ti
by docheinestages 15d ago
I mean, I'm interested to know if the frontier models also get to see all questions at once. Then it's more fair game than if they just see one question at a time.
- JBits 15d agoThey look at each problem individually. As the ARC AGI guidelines [1] state, 'a core design principle of ARC-AGI is that the test taker must not know what the test will be.' [1] https://arcprize.org/policy#dataset-security https://arcprize.org/policy#dataset-security