7 ms·

Google gives a pittance even for full ossfuzz integration. Which is why many projects just have the bare minimum fuzz tests. My original point was that even wit

Google gives a pittance even for full ossfuzz integration. Which is why many projects just have the bare minimum fuzz tests. My original point was that even with these bare minimum tests ossfuzz has found way more than "AI" has.

tptacek 9mo ago
Another weird assumption you've got here is that fuzzing outcomes scale linearly with funding, which, no. Further, the field of factory-scale fuzzing and triage is one Google security engineers basically invented, so it's especially odd to hold Google out as a bad actor here. At any rate, Google didn't employ "AI" to find this vulnerability, and Google fuzzing probably wouldn't have outcompeted these researchers for this particular bug (totally different methods of bugfinding), so it's really hard to find a coherent point you'd be making about "fuzzers", "AI", and "Google" here.
- hedgehog 9mo ago
  My guess is the main "AI" contribution here is to automate some of the work around the actual fuzzing. Setting up the test environment and harness, reading the code + commit history + published vulns for similar projects, identifying likely trouble spots, gathering seed data, writing scripts to generate more seed data reaching the identified trouble spots, adding instrumentation to the target to detect conditions ASan etc don't, writing PoC code, writing draft patches... That's a lot of labor and the coding agents can do a mediocre job of all of it for the cost of compute.
  - tptacek 9mo ago
    If it's finding exploitable bugs prior factory-scale fuzzing of ffmpeg hasn't, seems like a pretty big win to me.
    - hedgehog 9mo ago
      For sure, and I think it expands the scope of what factory scale efforts can find. The big question of course being how to handle remediation because more bugs without more maintainer capacity is a recipe for tears.
- bgwalter 9mo ago
  [flagged]
  - tptacek 9mo ago
    I am a professional software developer and have been since the 1990s.

tptacek 9mo ago

Another weird assumption you've got here is that fuzzing outcomes scale linearly with funding, which, no. Further, the field of factory-scale fuzzing and triage is one Google security engineers basically invented, so it's especially odd to hold Google out as a bad actor here. At any rate, Google didn't employ "AI" to find this vulnerability, and Google fuzzing probably wouldn't have outcompeted these researchers for this particular bug (totally different methods of bugfinding), so it's really hard to find a coherent point you'd be making about "fuzzers", "AI", and "Google" here.

hedgehog 9mo ago
My guess is the main "AI" contribution here is to automate some of the work around the actual fuzzing. Setting up the test environment and harness, reading the code + commit history + published vulns for similar projects, identifying likely trouble spots, gathering seed data, writing scripts to generate more seed data reaching the identified trouble spots, adding instrumentation to the target to detect conditions ASan etc don't, writing PoC code, writing draft patches... That's a lot of labor and the coding agents can do a mediocre job of all of it for the cost of compute.
- tptacek 9mo ago
  If it's finding exploitable bugs prior factory-scale fuzzing of ffmpeg hasn't, seems like a pretty big win to me.
  - hedgehog 9mo ago
    For sure, and I think it expands the scope of what factory scale efforts can find. The big question of course being how to handle remediation because more bugs without more maintainer capacity is a recipe for tears.
bgwalter 9mo ago
[flagged]
- tptacek 9mo ago
  I am a professional software developer and have been since the 1990s.