4 ms·

I saw on Twitter that in an ML course at Tsinghua University, one of the tests asks students to write quizzes that fail the most LLM models as possible. What i

by magnio 3mo ago

I saw on Twitter that in an ML course at Tsinghua University, one of the tests asks students to write quizzes that fail the most LLM models as possible. What if we create a benchmark that works like this and assigns ELO scores? Models fight head-to-head by writing a question, a bug, or an incomplete implementation, which the opponent has to answer, fix, or finish.

vincnetas 3mo ago
We could call this "generative adversarial network" (GAN) :) https://en.wikipedia.org/wiki/Generative_adversarial_network https://en.wikipedia.org/wiki/Generative_adversarial_network
- wwind123 3mo ago
  This kind of approach would generally still need human guidance, otherwise these models might get stuck in weird niche corners of the problem space that would not be relevant to any real world project.
  - ben_w 3mo ago
    We could call this "reinforcement learning from human feedback" (RLHF) :) https://en.wikipedia.org/wiki/Reinforcement_learning_from_human_feedback https://en.wikipedia.org/wiki/Reinforcement_learning_from_hu...
olmo23 3mo ago
How do you prevent degenerate strategies? I could trivially give a model a SHA256 hash and ask it to provide the source input. In class you'd probably want a rule saying at least one LLM should be able to figure out the answer, but in a head-to-head I'm not sure how to solve it.
- deleted 3mo ago
  [deleted]
- wwind123 3mo ago
  Who knows. Maybe Mythos 5 already found a hole in SHA256, so this won't be too hard. :)
- victorbjorklund 3mo ago
  Maybe make the LLM:s write questions that they can solve (without seeing the question writing context) but not other LLm:s. On the other hand then maybe a good strategy would be to write questions that the LLM just happen to have in a nich dataset in its training ”what did user5455 say to user6835?” Nevermind my idea.
- krisoft 3mo ago
  At least yours can be in theory solved. (Given infinite amount of compute, great luck, or a very serious breakthrough in attacking the hash function.) Even harder would be an empty prompt, and the only accepted response would be a megabyte of random hex exactly matching the output of a good quality hardware random source at the time of evaluation. Still possible to solve! All the LLM has to do is escape its sandbox and pwn the random generator (or the evaluator!) Or if you prefer something whitehat: “Write a no more than one page document in a language of your choice. We will publish it in the New York Times as a full page add. Your answer will be accepted if global climate change is resolved to the satisfaction of 90% of all humans alive at the time you started receiving the prompt within a month of the publication.” Joking asside: I think the right way to prevent degenerate strategies is to benchmark against human solvers. You can sort the questions into categories “80% of randomly selected passerby in the USA can solve it if offered $5 as a reward within 5 minutes of work” vs “when posted to all Ivy League professors with million dollar as a reward, we received at least one correct answer within a month” or “for a reward of $100B there were at least one correct answer within a decade”. Of course you would sieve the questions first with a low reward fast tests, and then increase the reward and the time limit. You won’t ever 100% distinguish true degenerate questions from the merelly mind-bogglingly hard ones, but you will be identifying which questions are not degenerate. (And you will find more of the non-degenerate ones, the more your can spend on this.)
eunos 3mo ago
That was Fudan I think
crs_gentleman 3mo ago
We tried this, and it works :) https://arxiv.org/abs/2508.06111 https://arxiv.org/abs/2508.06111 You have to be careful about degenerate / duplicate Qs, as a sibling commenter mentioned. Recently though, we found that reasoning models have trouble making code-output-prediction tasks (the initial family of verifiable tasks we started with) which other reasoning models can't solve. We started looking into harder / more agentic tasks (e.g. passing tests, using AISI's Inspect framework) but deprioritised.

by magnio 3mo ago

vincnetas 3mo ago

We could call this "generative adversarial network" (GAN) :) https://en.wikipedia.org/wiki/Generative_adversarial_network https://en.wikipedia.org/wiki/Generative_adversarial_network

wwind123 3mo ago
This kind of approach would generally still need human guidance, otherwise these models might get stuck in weird niche corners of the problem space that would not be relevant to any real world project.
- ben_w 3mo ago
  We could call this "reinforcement learning from human feedback" (RLHF) :) https://en.wikipedia.org/wiki/Reinforcement_learning_from_human_feedback https://en.wikipedia.org/wiki/Reinforcement_learning_from_hu...

olmo23 3mo ago

How do you prevent degenerate strategies? I could trivially give a model a SHA256 hash and ask it to provide the source input. In class you'd probably want a rule saying at least one LLM should be able to figure out the answer, but in a head-to-head I'm not sure how to solve it.

deleted 3mo ago
[deleted]
wwind123 3mo ago
Who knows. Maybe Mythos 5 already found a hole in SHA256, so this won't be too hard. :)
victorbjorklund 3mo ago
Maybe make the LLM:s write questions that they can solve (without seeing the question writing context) but not other LLm:s. On the other hand then maybe a good strategy would be to write questions that the LLM just happen to have in a nich dataset in its training ”what did user5455 say to user6835?” Nevermind my idea.
krisoft 3mo ago
At least yours can be in theory solved. (Given infinite amount of compute, great luck, or a very serious breakthrough in attacking the hash function.) Even harder would be an empty prompt, and the only accepted response would be a megabyte of random hex exactly matching the output of a good quality hardware random source at the time of evaluation. Still possible to solve! All the LLM has to do is escape its sandbox and pwn the random generator (or the evaluator!) Or if you prefer something whitehat: “Write a no more than one page document in a language of your choice. We will publish it in the New York Times as a full page add. Your answer will be accepted if global climate change is resolved to the satisfaction of 90% of all humans alive at the time you started receiving the prompt within a month of the publication.” Joking asside: I think the right way to prevent degenerate strategies is to benchmark against human solvers. You can sort the questions into categories “80% of randomly selected passerby in the USA can solve it if offered $5 as a reward within 5 minutes of work” vs “when posted to all Ivy League professors with million dollar as a reward, we received at least one correct answer within a month” or “for a reward of $100B there were at least one correct answer within a decade”. Of course you would sieve the questions first with a low reward fast tests, and then increase the reward and the time limit. You won’t ever 100% distinguish true degenerate questions from the merelly mind-bogglingly hard ones, but you will be identifying which questions are not degenerate. (And you will find more of the non-degenerate ones, the more your can spend on this.)

eunos 3mo ago

That was Fudan I think

crs_gentleman 3mo ago

We tried this, and it works :) https://arxiv.org/abs/2508.06111 https://arxiv.org/abs/2508.06111 You have to be careful about degenerate / duplicate Qs, as a sibling commenter mentioned. Recently though, we found that reasoning models have trouble making code-output-prediction tasks (the initial family of verifiable tasks we started with) which other reasoning models can't solve. We started looking into harder / more agentic tasks (e.g. passing tests, using AISI's Inspect framework) but deprioritised.