fmaccomber

8 ms·

1. 1.▲
   by fmaccomber 2mo ago

   That's not an accurate characterization of the ruling
2. 2.▲
   Benchmark accuracy retention is the wrong metric(joshuahedtke.com)

   2 points by fmaccomber 3mo ago|1 comments
3. 3.▲
   by fmaccomber 3mo ago

   Whether model routing works is an empirical problem. Existing empirical efforts rely on benchmark accuracy retention, i.e. how does a model routing system score compared to a sophisticated model like Opus 4.7 on a complex task benchmark lik