7 ms·

Just use an LLM to weed them out. What’s so hard about that?

by novalis78 8mo ago

eqvinox 8mo ago
At this point it's impossible to tell if this is sarcasm or not. Brave new world we got there.
vee-kay 8mo ago
Set a thief to catch a thief.
bootsmann 8mo ago
If AI can't be trusted to write bug reports, why should it be trusted to review them?
GalaxyNova 8mo ago
Because LLMs are bad at reviewing code for the same reasons they are bad at making it? They get tricked by fancy clean syntax and take long descriptions / comments for granted without considering the greater context.
- colechristensen 8mo ago
  I don't know, I prompted Opus 4.5 "Tell me the reasons why this report is stupid" on one of the example slop reports and it returned a list of pretty good answers.[1] Give it a presumption of guilt and tell it to make a list, and an LLM can do a pretty good job of judging crap. You could very easily rig up a system to give this "why is it stupid" report and then grade the reports and only let humans see the ones that get better than a B+. If you give them the right structure I've found LLMs to be much better at judging things than creating them. Opus' judgement in the end: "This is a textbook example of someone running a sanitizer, seeing output, and filing a report without understanding what they found." 1. https://claude.ai/share/8c96f19a-cf9b-4537-b663-b1cb771bfe3f https://claude.ai/share/8c96f19a-cf9b-4537-b663-b1cb771bfe3f
  - nprateem 8mo ago
    And if you ask why it's accurate it'll spaff out another list of pretty convincing answers.
    - colechristensen 8mo ago
      It does indeed, but at the end added: >However, I should note: without access to the actual crash file, the specific curl version, or ability to reproduce the issue, I cannot verify this is a valid vulnerability versus expected behavior (some tools intentionally skip cleanup on exit for performance). The 2-byte leak is also very small, which could indicate this is a minor edge case or even intended behavior in certain code paths. Even biased towards positivity it's still giving me the correct answer. Given a neutral "judge this report" prompt we get "This is a low-severity, non-security issue being reported as if it were a security vulnerability." with a lot more detail as to why So positive, neutral, or negative biased prompts all result in the correct answer that this report is bogus.
      - Draiken 8mo ago
        Yet this is not reproducible. This is the whole issue with LLMs: they are random. You cannot trust that it'll do a good job on all reports so you'll have to manually review the LLMs reports anyways or hope that real issues didn't get false-negatives or fake ones got false-positives. This is what I've seen most LLM proponents do: they gloss over the issues and tell everyone it's all fine. Who cares about the details? They don't review the gigantic pile of slop code/answers/results they generate. They skim and say YOLO. Worked for my narrow set of anecdotal tests, so it must work for everything! IIRC DOGE did something like this to analyze government jobs that were needed or not and then fired people based on that. Guess how good the result was? This is a very similar scenario: make some judgement call based on a small set of data. It absolutely sucks at it. And I'm not even going to get into the issue of liability which is another can of worms.
        colechristensen 8mo ago
        Is it not reproducable? Someone up thread reproduced it and expanded on it. It worked for me the first time I prompted. Did you try it or are you just guessing that it's not reproducable because that's what you already think? I'm not talking about completely replacing humans, the goal of this exercise was demonstrating how to use an LLM to filter out garbage. Low quality semi-anonymous reports don't deserve a whole lot of accuracy and being conservative and rejecting most reports even when you throw out legitimate ones is fine. You seem like regardless of evidence presented, your prejudices will lead you to the same conclusions, so what's the point discussing anything? I looked for, found, and shared evidence, you're sharing your opinion. >IIRC DOGE did something like this to analyze government jobs that were needed or not and then fired people based on that. Guess how good the result was? I'm talking about filtering spammy communication channels, that has nothing like the care required in making employment decisions. Your comment is plainly just bad faith and prejudice.
f311a 8mo ago
How would it work if LLMs provide incorrect reports in the first place? Have a look at the actual HackerOne reports and their comments. The problem is the complete stupidity of people. They use LLMs to convince the author of the curl that he is not correct about saying that the report is hallucinated. Instead of generating ten LLM comments and doubling down on their incorrect report, they could use a bit of brain power to actually validate the report. It does not even require a lot of skills, you have to manually tests it.
- fc417fc802 8mo ago
  Let the reporter duke it out with the project's gatekeeping LLM. If it keeps going on for long enough a human can quickly skim the exchange. It should be immediately obvious if the reporter is making sensible rebuttals or just throwing more slop at the wall. I think fighting fire with fire is likely the correct answer here.

Just use an LLM to weed them out. What’s so hard about that?

eqvinox 8mo ago

At this point it's impossible to tell if this is sarcasm or not. Brave new world we got there.

vee-kay 8mo ago

Set a thief to catch a thief.

bootsmann 8mo ago

If AI can't be trusted to write bug reports, why should it be trusted to review them?

GalaxyNova 8mo ago

Because LLMs are bad at reviewing code for the same reasons they are bad at making it? They get tricked by fancy clean syntax and take long descriptions / comments for granted without considering the greater context.

colechristensen 8mo ago
I don't know, I prompted Opus 4.5 "Tell me the reasons why this report is stupid" on one of the example slop reports and it returned a list of pretty good answers.[1] Give it a presumption of guilt and tell it to make a list, and an LLM can do a pretty good job of judging crap. You could very easily rig up a system to give this "why is it stupid" report and then grade the reports and only let humans see the ones that get better than a B+. If you give them the right structure I've found LLMs to be much better at judging things than creating them. Opus' judgement in the end: "This is a textbook example of someone running a sanitizer, seeing output, and filing a report without understanding what they found." 1. https://claude.ai/share/8c96f19a-cf9b-4537-b663-b1cb771bfe3f https://claude.ai/share/8c96f19a-cf9b-4537-b663-b1cb771bfe3f
- nprateem 8mo ago
  And if you ask why it's accurate it'll spaff out another list of pretty convincing answers.
  - colechristensen 8mo ago
    It does indeed, but at the end added: >However, I should note: without access to the actual crash file, the specific curl version, or ability to reproduce the issue, I cannot verify this is a valid vulnerability versus expected behavior (some tools intentionally skip cleanup on exit for performance). The 2-byte leak is also very small, which could indicate this is a minor edge case or even intended behavior in certain code paths. Even biased towards positivity it's still giving me the correct answer. Given a neutral "judge this report" prompt we get "This is a low-severity, non-security issue being reported as if it were a security vulnerability." with a lot more detail as to why So positive, neutral, or negative biased prompts all result in the correct answer that this report is bogus.
    - Draiken 8mo ago
      Yet this is not reproducible. This is the whole issue with LLMs: they are random. You cannot trust that it'll do a good job on all reports so you'll have to manually review the LLMs reports anyways or hope that real issues didn't get false-negatives or fake ones got false-positives. This is what I've seen most LLM proponents do: they gloss over the issues and tell everyone it's all fine. Who cares about the details? They don't review the gigantic pile of slop code/answers/results they generate. They skim and say YOLO. Worked for my narrow set of anecdotal tests, so it must work for everything! IIRC DOGE did something like this to analyze government jobs that were needed or not and then fired people based on that. Guess how good the result was? This is a very similar scenario: make some judgement call based on a small set of data. It absolutely sucks at it. And I'm not even going to get into the issue of liability which is another can of worms.
      - colechristensen 8mo ago
        Is it not reproducable? Someone up thread reproduced it and expanded on it. It worked for me the first time I prompted. Did you try it or are you just guessing that it's not reproducable because that's what you already think? I'm not talking about completely replacing humans, the goal of this exercise was demonstrating how to use an LLM to filter out garbage. Low quality semi-anonymous reports don't deserve a whole lot of accuracy and being conservative and rejecting most reports even when you throw out legitimate ones is fine. You seem like regardless of evidence presented, your prejudices will lead you to the same conclusions, so what's the point discussing anything? I looked for, found, and shared evidence, you're sharing your opinion. >IIRC DOGE did something like this to analyze government jobs that were needed or not and then fired people based on that. Guess how good the result was? I'm talking about filtering spammy communication channels, that has nothing like the care required in making employment decisions. Your comment is plainly just bad faith and prejudice.

f311a 8mo ago

How would it work if LLMs provide incorrect reports in the first place? Have a look at the actual HackerOne reports and their comments. The problem is the complete stupidity of people. They use LLMs to convince the author of the curl that he is not correct about saying that the report is hallucinated. Instead of generating ten LLM comments and doubling down on their incorrect report, they could use a bit of brain power to actually validate the report. It does not even require a lot of skills, you have to manually tests it.

fc417fc802 8mo ago
Let the reporter duke it out with the project's gatekeeping LLM. If it keeps going on for long enough a human can quickly skim the exchange. It should be immediately obvious if the reporter is making sensible rebuttals or just throwing more slop at the wall. I think fighting fire with fire is likely the correct answer here.