7 ms·
I think this is an open question still and very interesting. Ilya discussed this on the Dwarkesh podcast. But the capabilities of LLMs is clearly exponential an
by HPMOR 9mo ago
I think this is an open question still and very interesting. Ilya discussed this on the Dwarkesh podcast. But the capabilities of LLMs is clearly exponential and perhaps super exponential. We went from something that could string together incoherent text in 2022 to general models helping people like Terrance Tao and Scott Aaronson write new research papers. LLMs also beat IMO and the ICPC. We have entered the John Henry era for intellectual tasks...
- llmslave2 9mo ago> But the capabilities of LLMs is clearly exponential and perhaps super exponential By what metric?
- utopiah 9mo agoBS metric... /s
- jennyholzer3 9mo agoChat GPT told him it was true
- aspenmartin 9mo ago- Scaling laws (Chinchilla type) - METR task horizon It's a mix, performance gains are bursty but we have been getting a lot of bursts (RLVR, test-time compute, agentic breakthroughs)
- tsimionescu 9mo ago> LLMs also beat IMO and the ICPC Very spurious claims, given that there was no effort made to check whether the IMO or ICPC problems were in the training set or not, or to quantify how far problems in the training set were from the contest problems. IMO problems are supposed to be unique, but since it's not at the frontier of math research, there is no guarantee that the same problem, or something very similar, was not solved in some obscure manual.