6 ms·
why is your actual benchmark question so secret?
by _zoltan_ 12d ago
why is your actual benchmark question so secret?
- dotancohen 12d agoProbably so it does not become a target to meet.
- cogman10 12d agoBingo. I know that people that work on LLMs read sites like HN. Already my question is losing it's usefulness as most models pass it now a days, but not every model does. The "car wash" question is a good example of this happening. Pretty much every model now correctly answers that question because it gained enough notoriety that the LLM authors now train to avoid looking silly on it.