6 ms·

This thought process is pretty baffling to me, and this is at least the second time I've encountered it on HN. What's the value of a secret benchmark to anyone

by grog454 9mo ago

This thought process is pretty baffling to me, and this is at least the second time I've encountered it on HN. What's the value of a secret benchmark to anyone but the secret holder? Does your niche benchmark even influence which model you use for unrelated queries? If LLM authors care enough about your niche (they don't) and fake the response somehow, you will learn on the very next query that something is amiss. Now that query is your secret benchmark. Even for niche topics it's rare that I need to provide more than 1 correction or knowledge update.

Turskarama 9mo ago
The point is that it's a litmus test for how well the models do with niche knowledge _in general_. The point isn't really to know how well the model works for that specific niche. Ideally of course you would use a few of them and aggregate the results.
nl 9mo ago
I have a bunch of private benchmarks I run against new models I'm evaluating. The reason I don't disclose isn't generally that I think an individual person is going to read my post and update the model to include it. Instead it is because if I write "I ask the question X and expect Y" then that data ends up in the train corpus of new LLMs. However, one set of my benchmarks is a more generalized type of test (think a parlor-game type thing) that actually works quite well. That set is the kind of thing that could be learnt via reinforcement learning very well, and just mentioning it could be enough for a training company or data provider company to try it. You can generate thousands of verifiable tests - potentially with verifiable reasoning traces - quite easily.
- grog454 9mo ago
  Ok, but then your "post" isn't scientific by definition since it cannot be verified. "Post" is in quotes because I don't know what you're trying to but you're implying some sort of public discourse. For fun: https://chatgpt.com/s/t_694361c12cec819185e9850d0cf0c629 https://chatgpt.com/s/t_694361c12cec819185e9850d0cf0c629
  - eru 9mo ago
    I didn't see anyone claiming any 'science'? Did I miss something?
    - grog454 9mo ago
      I guess there's two things I'm still stuck on: 1. What is the purpose of the benchmark? 2. What is the purpose of publicly discussing a benchmark's results but keeping the methodology secret? To me it's in the same spirit as claiming to have defeated alpha zero but refusing to share the game.
      - nl 9mo ago
        1. The purpose of the benchmark is to choose what models I use for my own system(s). This is extremely common practice in AI - I think every company I've worked with doing LLM work in the last 2 years has done this in some form. 2. I discussed that up-thread, but https://github.com/microsoft/private-benchmarking https://github.com/microsoft/private-benchmarking and https://arxiv.org/abs/2403.00393 https://arxiv.org/abs/2403.00393 discuss some further motivation for this if you are interested. > To me it's in the same spirit as claiming to have defeated alpha zero but refusing to share the game. This is an odd way of looking at it. There is no "winning" at benchmarks, it's simply that it is a better and more repeatable evaluation than the old "vibe test" that people did in 2024.
        grog454 9mo ago
        I see the potential value of private evaluations. They aren't scientific but you can certainly beat a "vibe test". I don't understand the value of a public post discussing their results beyond maybe entertainment. We have to trust you implicitly and have no way to validate your claims. > There is no "winning" at benchmarks, it's simply that it is a better and more repeatable evaluation than the old "vibe test" that people did in 2024. Then you must not be working in an environment where a better benchmark yields a competitive advantage.
        9mo ago
akoboldfrying 9mo ago
I actually think "concealing the question" is not only a good idea, but a rather general and powerful idea that should be much more widely deployed (but often won't be, for what I consider "emotional reasons"). Example: You are probably already aware that almost any metric that you try to use to measure code quality can be easily gamed. One possible strategy is to choose a weighted mixture of metrics and conceal the weights. The weights can even change over time. Is it perfect? No. But it's at least correlated with code quality -- and it's not trivially gameable, which puts it above most individual public metrics.
- grog454 9mo ago
  It's hard to have any certainty around concealment unless you are only testing local LLMs. As a matter of principle I assume the input and output of any query I run in a remote LLM is permanently public information (same with search queries). Will someone (or some system) see my query and think "we ought to improve this"? I have no idea since I don't work on these systems. In some instances involving random sampling... probably yes! This is the second reason I find the idea of publicly discussing secret benchmarks silly.
  - grog454 9mo ago
    I learned in another thread there is some work being done to avoid contamination of training data during evaluation of remote models using trusted execution environments (https://arxiv.org/pdf/2403.00393 https://arxiv.org/pdf/2403.00393). It requires participation of the model owner.
theshrike79 9mo ago
Because it encompasses the very specific way I like to do things. It's not of use to the general public.

by grog454 9mo ago

Turskarama 9mo ago

The point is that it's a litmus test for how well the models do with niche knowledge _in general_. The point isn't really to know how well the model works for that specific niche. Ideally of course you would use a few of them and aggregate the results.

nl 9mo ago

I have a bunch of private benchmarks I run against new models I'm evaluating. The reason I don't disclose isn't generally that I think an individual person is going to read my post and update the model to include it. Instead it is because if I write "I ask the question X and expect Y" then that data ends up in the train corpus of new LLMs. However, one set of my benchmarks is a more generalized type of test (think a parlor-game type thing) that actually works quite well. That set is the kind of thing that could be learnt via reinforcement learning very well, and just mentioning it could be enough for a training company or data provider company to try it. You can generate thousands of verifiable tests - potentially with verifiable reasoning traces - quite easily.

grog454 9mo ago
Ok, but then your "post" isn't scientific by definition since it cannot be verified. "Post" is in quotes because I don't know what you're trying to but you're implying some sort of public discourse. For fun: https://chatgpt.com/s/t_694361c12cec819185e9850d0cf0c629 https://chatgpt.com/s/t_694361c12cec819185e9850d0cf0c629
- eru 9mo ago
  I didn't see anyone claiming any 'science'? Did I miss something?
  - grog454 9mo ago
    I guess there's two things I'm still stuck on: 1. What is the purpose of the benchmark? 2. What is the purpose of publicly discussing a benchmark's results but keeping the methodology secret? To me it's in the same spirit as claiming to have defeated alpha zero but refusing to share the game.
    - nl 9mo ago
      1. The purpose of the benchmark is to choose what models I use for my own system(s). This is extremely common practice in AI - I think every company I've worked with doing LLM work in the last 2 years has done this in some form. 2. I discussed that up-thread, but https://github.com/microsoft/private-benchmarking https://github.com/microsoft/private-benchmarking and https://arxiv.org/abs/2403.00393 https://arxiv.org/abs/2403.00393 discuss some further motivation for this if you are interested. > To me it's in the same spirit as claiming to have defeated alpha zero but refusing to share the game. This is an odd way of looking at it. There is no "winning" at benchmarks, it's simply that it is a better and more repeatable evaluation than the old "vibe test" that people did in 2024.
      - grog454 9mo ago
        I see the potential value of private evaluations. They aren't scientific but you can certainly beat a "vibe test". I don't understand the value of a public post discussing their results beyond maybe entertainment. We have to trust you implicitly and have no way to validate your claims. > There is no "winning" at benchmarks, it's simply that it is a better and more repeatable evaluation than the old "vibe test" that people did in 2024. Then you must not be working in an environment where a better benchmark yields a competitive advantage.
        9mo ago

akoboldfrying 9mo ago

I actually think "concealing the question" is not only a good idea, but a rather general and powerful idea that should be much more widely deployed (but often won't be, for what I consider "emotional reasons"). Example: You are probably already aware that almost any metric that you try to use to measure code quality can be easily gamed. One possible strategy is to choose a weighted mixture of metrics and conceal the weights. The weights can even change over time. Is it perfect? No. But it's at least correlated with code quality -- and it's not trivially gameable, which puts it above most individual public metrics.

grog454 9mo ago
It's hard to have any certainty around concealment unless you are only testing local LLMs. As a matter of principle I assume the input and output of any query I run in a remote LLM is permanently public information (same with search queries). Will someone (or some system) see my query and think "we ought to improve this"? I have no idea since I don't work on these systems. In some instances involving random sampling... probably yes! This is the second reason I find the idea of publicly discussing secret benchmarks silly.
- grog454 9mo ago
  I learned in another thread there is some work being done to avoid contamination of training data during evaluation of remote models using trusted execution environments (https://arxiv.org/pdf/2403.00393 https://arxiv.org/pdf/2403.00393). It requires participation of the model owner.

theshrike79 9mo ago

Because it encompasses the very specific way I like to do things. It's not of use to the general public.