5 ms·
LLM capability improvement, eg as measured by METR task time.
by theptip 20d ago
LLM capability improvement, eg as measured by METR task time.
- fluoridation 20d agoSo it's an exponential with a negative exponent? The thing about those kinds of curves is that since they have asymptotes to zero, it's hard to tell when you've plateaued.
- theptip 20d agoNo, the metric is “p50 task duration”. 7-month doubling time, recently 4-month: https://metr.org/ https://metr.org/ Plenty of other exponential metrics too, compute built, AI revenue, etc.