thomasliao

7 ms·

1. 1.▲
   by thomasliao 3mo ago

   >equally "frontier" models A key point I want to make is that the notion of "frontier" is somewhat fictive in the sense that a model which dominates all others on a given eval is not guaranteed to be number one on ano
2. 2.▲
   by thomasliao 3mo ago

   It's an important question! If you are paying a lot of money to use AI models, you care that you are using the best for your task. And it turns out that figuring out which AI models is best for your task is not trivial and requires som
3. 3.▲
   by thomasliao 3mo ago

   (Author) It's short for "evaluation", a test for an AI model. Specifically, an AI evaluation comprises (1) a dataset of prompts (as questions / tasks / queries), (2) some way to score model performance on each promp