jmeaden

5 ms·

1. 1.▲
   by jmeaden 1y ago

   Hi, I’m one of the authors. Happy to answer questions about the dataset (LLM coding performance compared to 390k+ human submissions), the scoring approach, or the methodology behind COMPASS. Feedback and critique are welcome.
2. 2.▲
   Why most AI coding benchmarks are misleading (COMPASS paper)(arxiv.org)

   13 points by jmeaden 1y ago|2 comments
3. 3.▲
   by jmeaden 1y ago

   agree
4. 4.▲
   by jmeaden 1y ago

   This "taste" concept has overlap with conscientiousness, a trait that companies look for when hiring. But, it is hard to develop.