5 ms·

I'm a significant genAI skeptic. I periodically ask them questions about topics that are subtle or tricky, and somewhat niche, that I know a lot about, and fin

• TeodorDyakov 9mo ago
  Hi. I am curious what was the benchmark question? Cheers!
  • Turskarama 9mo ago
    The problem with publicly disclosing these is that if lots of people adopt them they will become targeted to be in the model and will no longer be a good benchmark.
    • lambda 9mo ago
      Yeah, that's part of why I don't disclose. Obviously, the fact that I've done Google searches and tested the models on these means that their systems may have picked up on them; I'm sure that Google uses its huge dataset of Google searches and search index as inputs to its training, so Google has an advantage here. But, well, that might be why Googles new models are so much better, they're actually taking advantage of some of this massive dataset they've had for years.
    • grog454 9mo ago
      This thought process is pretty baffling to me, and this is at least the second time I've encountered it on HN. What's the value of a secret benchmark to anyone but the secret holder? Does your niche benchmark even influence which model you use for unrelated queries? If LLM authors care enough about your niche (they don't) and fake the response somehow, you will learn on the very next query that something is amiss. Now that query is your secret benchmark. Even for niche topics it's rare that I need to provide more than 1 correction or knowledge update.
      • Turskarama 9mo ago
        The point is that it's a litmus test for how well the models do with niche knowledge _in general_. The point isn't really to know how well the model works for that specific niche. Ideally of course you would use a few of them and aggregate the results.
      • nl 9mo ago
        I have a bunch of private benchmarks I run against new models I'm evaluating. The reason I don't disclose isn't generally that I think an individual person is going to read my post and update the model to include it. Instead it is because if I write "I ask the question X and expect Y" then that data ends up in the train corpus of new LLMs. However, one set of my benchmarks is a more generalized type of test (think a parlor-game type thing) that actually works quite well. That set is the kind of thing that could be learnt via reinforcement learning very well, and just mentioning it could be enough for a training company or data provider company to try it. You can generate thousands of verifiable tests - potentially with verifiable reasoning traces - quite easily.
        • grog454 9mo ago
          Ok, but then your "post" isn't scientific by definition since it cannot be verified. "Post" is in quotes because I don't know what you're trying to but you're implying some sort of public discourse. For fun: https://chatgpt.com/s/t_694361c12cec819185e9850d0cf0c629 https://chatgpt.com/s/t_694361c12cec819185e9850d0cf0c629
  • kridsdale3 9mo ago
    If they told you, it would be picked up in a future model's training run.
    • jacobn 9mo ago
      Don't the models typically train on their input too? I.e. submitting the question also carries a risk/chance of it getting picked up? I guess they get such a large input of queries that they can only realistically check and therefore use a small fraction? Though maybe they've come up with some clever trick to make use of it anyway?
      • energy123 9mo ago
        Given they asked in on LMArena, yes.
        • lambda 9mo ago
          Yeah, probably asking on LMArena makes this an invalid benchmark going forward, especially since I think Google is particular active in testing models on LMArena (as evidenced by the fact that I got their preview for this question). I'll need to find a new one, or actually put together a set of questions to use instead of just a single benchmark.
      • jerojero 9mo ago
        they probably dont train on inputs from testing grounds. you dont train on your test data because you need to have that to compare if training is improving or not.
      • nl 9mo ago
        OpenAI and Anthropic don't train on your questions if you have pressed the opt-out button and are using their UI. LMArena is a different matter.
    • _heimdall 9mo ago
      Is that an issue if you now need a new question to ask?
  • Marazan 9mo ago
    Heres my old benchmark question and my new variant: "When was the last time England beat Scotland at rugby union" new variant "Without using search when was the last time England beat Scotland at rugby union" It is amazing how bad ChatGPT is at this question and has been for years now across multiple models. It's not that it gets it wrong - no shade, I've told it not to search the web so this is _hard_ for it - but how badly it reports the answer. Starting from the small stuff - it almost always reports the wrong year, wrong location and wrong score - that's the boring facts stuff that I would expect it to stumble on. It often creates details of matches that didn't exist, cool standard hallucinations. But even within the text it generates itself it cannot keep it consistent with how reality works. It often reports draws as wins for England. It frequently states the team that it just said scored most points lost the match, etc. It is my ur example for when people challenge my assertion LLMs are stochastic parrots or fancy Markov chains on steroids.
• prettyblocks 9mo ago
  I don't think tricky niche knowledge is the sweet spot for genai and it likely won't be for some time. Instead, it's a great replacement for rote tasks where a less than perfect performance is good enough. Transcription, ocr, boilerplate code generation, etc.
  • ozim 9mo ago
    Second this. Basically making sense of unstructured data is super cool. I can get 20 people to write an answer the way they feel like it and model can convert it to structured data - something I would have to spend time on, or I would have to make form with mandatory fields that annoy audience. I am already building useful tools with the help of models. Asking tricky or trivia questions is fun and games. There are much more interesting ways to use AI.
  • lambda 9mo ago
    The thing is, I see people use it for tricky niche knowledge all the time; using it as an alternative to doing a Google search. So I want to have a general idea of how good it is at this. I found something that was niche, but not super niche; I could easily find a good, human written answer in the top couple of results of a Google search. But until now, all LLM answers I've gotten for it have been complete hallucinated gibberish. Anyhow, this is a single data point, I need to expand my set of benchmark questions a bit now, but this is the first time that I've actually seen progress on this particular personal benchmark.
    • mikepurvis 9mo ago
      And Google themselves obviously believe that too as they happily insert AI summaries at the top of most serps now.
      • ComputerGuru 9mo ago
        Or maybe Google knows most people search inane, obvious things?
• fragmede 9mo ago
  Even the most magical wonderful auto-hammer is gonna be bad at driving in screws. And, in this analogy I can't fault you because there are people trying to sell this hammer as a screwdriver. My opinion is that it's important to not lose sight of the places where it is useful because of the places where it isn't.
  • pretzellogician 9mo ago
    Funny, I grew up using what's called a "hand impact screwdriver"... turns out a hammer can be used to drive in screws!
• andai 9mo ago
  So this is an interesting benchmark, because if the answer is actually in the top 3 google results, then my python script that runs a google search, scrapes the top n results and shoves them into a crappy LLM would pass your benchmark too! Which also implies that (for most tasks), most of the weights in a LLM are unnecessary, since they are spent on memorizing the long tail of Common Crawl... but maybe memorizing infinite trivia is not a bug but actually required for the generalization to work? (Humans don't have far transfer though... do transformers have it?)
  • lambda 9mo ago
    I've tried doing this query with search enabled in LLMs before, which is supposed to effectively do that, and even with that they didn't give very good answers. It's a very physical kind of thing, and its easy to conflate with other similar descriptions, so they would frequently just conflate various different things and give some horrible mash-up answer that wasn't about the specific thing I'd asked about.
    • andai 9mo ago
      So it's a difficult question for LLMs to answer even when given perfect context? Kinda sounds like you're testing two things at the same time then, right? The knowledge of the thing (was it in the training data and was it memorized?) and the understanding of the thing (can they explain it properly even if you give them the answer in context).
• vitaflo 9mo ago
  I also have my own tricky benchmark that up til now only Deepseek has been able to answer. Gemini 3 Pro was the second. Every other LLM fail horribly. This is the main reason I started looking at G3pro more seriously.
• arisAlexis 9mo ago
  can you give us an example of this niche knowledge? I highly doubt there is knowledge that is not inside some internet training material.
• jve 9mo ago
  Counter point about general knowledge that is documented/discussed in different spots on the internet. Today I had to resolve performance problems for some sql server statement. Been doing it years, know the regular pitfalls, sometimes have to find "right" words to explain to customer why X is bad and such. I described the issue to GPT5.2, gave the query, the execution plan and asked for help. It was spot on, high quality responses and actionable items and explanations on why this or that is bad, how to improve it and why particularly sql may have generated such a query plan. I could instantly validate the response given my experience in the field. I even answered with some parts of chatgpt on how well it explained. However I did mention that to customer and I did tell them I approve the answer. Asked high quality question and receive a high quality answer. And I am happy that I found out about an sql server flag where I can influence particular decision. But the suggestion was not limited to that, there were multiple points given that would help.