11 ms·

SQLite Critical CVEs or LLM Slop?

• deleted 2mo ago
  [deleted]
• inigyou 2mo ago
  This is going to be fun for organizations that are mandated to patch all CVEs, isn't it?
  • ape4 2mo ago
    Create the referenced but non-existent file and then fix it /s
  • ymir_e 2mo ago
    This was my first thought, this could be terrible if used offensively. The best defense I can imagine is to have an agent reproduce the issues before a human sees it, but even that will cost money.
    • inigyou 2mo ago
      If by "terrible" you mean "amazing". Imagine being able to tear down an entire Jenga tower of bullshit corpospeak, and replace it with actual security.
  • lucideer 2mo ago
    I'm very curious what organisations would have such a policy. I can't imagine it being viable for any size of org without significant self-deception (or banning the use of all open source at which point CVEs are moot anyway).
    • ignore_prev 2mo ago
      [flagged]
    • SirFatty 2mo ago
      ITAR
      • lucideer 2mo ago
        ITAR has no such hard requirements. Might be some orgs that tell themselves they're attempting this under ITAR but they're not doing it in any comprehensive way. The only thing within ITAR that I'm aware of concerning itself with software supply chain is SP 800-218 requirements & that's just a load of open-to-interpretation weasel words about having CVE detection & automations in place & some defined plans for reducing the number of vulns. Pretty sure that component of it is even eligible for self-assessment.
        • bluGill 2mo ago
          Yes and no. ITAR (and other laws like it) are self assessment and don't specifically say thing thing. However your interpretation / self-assessment is subject to various reviews/audits. These days the reviewers are not going to be kind to someone who just says "not an issue", they will demand strong justification. Most organizations take the view that is is easier to fix all CVEs than try to pass audits. Thus by the letter of the law you are correct. However to meet the letter of the law without fixing CVEs is generally seen as harder than thus fixing CVEs. So the effect is ITAR (and similar laws) force you to fix CVEs.
        • SirFatty 2mo ago
          Whatever you say, chief. I worked in that environment for quite a while, and maybe on a technicality you're right, the effect outcome is that you will do it if not for all your customers that will require it.
    • clbrmbr 2mo ago
      Many orgs (esp w ISO27000) have a vulnerability management policy that involves patching at least critical CVEs within a short timeline. Tools like trivvy make it possible to do the scans…
      • lucideer 2mo ago
        I've been in such an org, & I've led initiatives to set up automated detection at very large scale. We started by issuing tickets to teams to resolve CVEs within varying timelines - ranging from a 24hr fix to 6 months - connected to the CVSS score. It wasn't viable. - Firstly, you quickly realise how irrelevant CVSS scores are - initiatives like First's EPSS are designed to fix this but they aren't there yet - Secondly, you need to begin implementing localised heuristics to determine exploitable code paths. This has generally been incredibly difficult to do reliably - LLMs have started to make it easier, but it's expensive. - Lastly, you need to factor in consideration of actionable remediation pathways. A dependency upgrade for critical infrastructure might contain breaking changes that take months to fix, or two competing CVEs might be present in interdependent versions of transitive dependencies in your sbom tree. Most orgs aren't applying any of the above three filters to reduce their CVE remediation burden, & even if they are, it's still too high to make zero a viable target. In reality, most orgs aren't doing comprehensive detection to begin with - if you haven't discovered all of your CVEs, your remediation burden is going to be a lot more manageable.
        • mr_mitm 2mo ago
          > - Firstly, you quickly realise how irrelevant CVSS scores are Even if you factor in the environmental score? I realize it's a lot more work, but it basically allows you to tune the score to get any value you want.
          • michaelt 2mo ago
            Imagine a YAML parsing library that can cause an out-of-memory exception if you give it a YAML file greater than 3 megabytes. If you're an online service where untrusted users can submit arbitrary YAML, and an out-of-memory exception is a severe problem, then it's severity 10. If you're an online service that doesn't use yaml in any way, but your web framework bundled the library as a transitive dependency because yaml is one of their five supported configuration options, then it's severity 2. The problem is figuring out which of those situations you're in takes a load of time - and the flow of CVEs is endless, as CVE numbers are given out like candy at halloween. Often it's quicker to just update to the latest version of the YAML library.
    • anygivnthursday 2mo ago
      If I remember correctly, we had to patch or provide justification for CVEs flagged by tools like AWS Inspector for SOC2 as well.
      • YeahThisIsMe 2mo ago
        So you didn't have to patch all of them.
        • bluGill 2mo ago
          No, but if you don't patch them you need to convince an auditor that they are not a problem. Often patching is easier. I'm working on such a problem now - we are using an old web browser (no longer supported) to show help on one system. That is web pages were generate internally, with no links elsewhere, and no provision for the user to enter a URL. It is still easier port to a newer supported browser than to convince the auditors that that we are not exploitable. Sure it is obvious that everything is internal and we won't write html that exploits bugs, but nobody wants to convince an auditor of that.
    • jeltz 2mo ago
      Many large organizations like banks have requirements like this and they solve it through a mix of automatic scanners, e.g. Trivvy, and self-deception as not all systems are actually scanned in any sufficiently large org.
    • traceroute66 2mo ago
      > I'm very curious what organisations would have such a policy. I would humbly suggest any org of any size that has insurance cover that covers anything tech related (e.g. data loss/recovery, cyber etc.) has a very good look at the small print. Over the last few years insurers have aggressively been adding "no vulnerability patch, no claim" exclusion clauses.
      • saghm 2mo ago
        Yeah, policies like this are often not coming from engineering directly but often through other parts of the company like legal, or even sales from contract negotiations. Not that it's entirely comparable, but I was at AWS when the big log4j vulnerability happened, and the handling for it was not left up to individual engineering teams, which I don't think would surprise anyone. At a large enough company, processes for handling things like security vulnerabilities will have a lot of stakeholders with incentives that are not necessarily perfectly aligned.
      • dns_snek 2mo ago
        > Over the last few years insurers have aggressively been adding "no vulnerability patch, no claim" exclusion clauses. If you take an even closer look- what exactly does that mean? It should stipulate patching real vulnerabilities in your system and not "all CVEs in all dependencies irrespective of their usage or applicability to your system", that's madness. It's like voiding your health insurance policy because a smoker moved in across the street. Most vulnerabilities in your dependencies don't become vulnerabilities in your system, and most vulnerabilities in your system (probably) don't originate from your dependencies.
        • traceroute66 1mo ago
          > If you take an even closer look- what exactly does that mean? Here is one example: "Critical Vulnerability Exclusion We will not pay you under the cyber and data risks section of cover where your legal liability or any loss that you suffer arises from a cyber attack that exploits a critical vulnerability within your computer equipment. However this exclusion will only apply where a patch or fix for any critical vulnerability exploited has been available for 21 days prior to the date of the incident and has not been applied to your computer equipment. Critical Vulnerability means a common vulnerability and exposure (CVE) in the National Vulnerability Database operated by the National Institute of Standards and Technology which has a score of 8.0 or higher on the Common Vulnerability Scoring System (CVSS)"
    • vrighter 2mo ago
      I have been given a list by security. "We had an automated tool scan that machine. It reported these. Fix anything medium severity and above. Never mind that some of them involved vulnerabilities in some part of the bluetooth stack (servers in our datacenter don't even have bluetooth). But they just didn't care
      • ptx 2mo ago
        This does make some sense if it's considered a valid fix to document that you have verified that Bluetooth is disabled on the servers and therefore not vulnerable. But that assumes that the scanning tool can be told about this kind of fix, so that it stops warning about it, which I guess it might not.
      • SoftTalker 2mo ago
        So run apt full-upgrade and get the new bluetooth driver. Why bother with a fight over something that isn't even used? Just do the quickest thing to get it off your plate.
        • icedchai 2mo ago
          This might cause other problems, problems of the "if it's not broken, don't fix it" variety. Upgrading everything only to break something else, in a previously stable configuration, isn't worth it.
          • pixl97 2mo ago
            Really the days of "Lets run this stable configuration forever" are gone. Getting rid of as much stuff in your OS and software stack as possible should be the security teams ultimate goal, so you have less to upgrade in the end. But actual security updates just come out at a tremendous rate, and you need a QA system that checks as much as it can before prod is upgraded.
            • icedchai 2mo ago
              I do agree with more frequent upgrades, but it has to be part of the culture. The longer you wait, the harder it gets. Unfortunately, I have worked in some heavily tech-debt-laden environments where upgrading anything required an act of god. I've logged on to production servers with 1500+ days of uptime at multiple companies. Nothing had been updated since well before that time. At one place, I recall encountering a 10+ year old dependency. On top of that, they were still using Python 2.7. This wasn't that long ago.
    • michaelt 2mo ago
      SOC2 CC7.1 [1] requires a vulnerability scanner, findings tracked with tickets, assigned severities according to a documented risk-based system, severity-based SLAs for remediation, and that the SLAs mostly be complied with or have tracked exceptions. However it doesn't mandate any particular SLA, or the details of how risks are to be evaluated. Organisations get to write their own policy, and they don't need to commit to patching every CVE within 24 hours or anything like that. [1] https://www.compliancebase.org/controls/soc-2/cc7-1 https://www.compliancebase.org/controls/soc-2/cc7-1
      • deleted 2mo ago
        [deleted]
    • regularfry 2mo ago
      Any org large enough to have separated the people responsible for the security exposure of the organisation from the developers with familiarity of what's deployed is likely to have done exactly this. The thing you have to remember is that CVEs can be a) scanned for without exerting mental effort, and b) counted.
    • agilob 2mo ago
      It's more common than many think https://old.reddit.com/r/kubernetes/comments/1vb3x2c/where_are_people_getting_the_best_cvefree_images/ https://old.reddit.com/r/kubernetes/comments/1vb3x2c/where_a...
    • swiftcoder 2mo ago
      > I'm very curious what organisations would have such a policy Do you provide SOC2, HIPAA, GDPR, or similar certifications to your b2b customers? Then your tech stack undergoes an annual audit, and in your audit you will need to provide a paper trail for every single vulnerability in your stack. In practice, this means that your audit compliance software (something like Vanta.com) is going to be setup to mandate every CVE in the whole stack is patched within SLA.
    • jmull 2mo ago
      It's quite common in enterprisey environments. For one thing, bigcorps in regulated areas like it a lot. They push hard to get it required by the regulations (in practice if not directly). Although it's quite inefficient, it becomes a regulatory moat. A cost they can bear that potential upstart competitors cannot.
  • whatevaa 2mo ago
    Those organizations will have to adapt to new reality, ie, that some CVEs are not real.
    • afarah1 2mo ago
      Nothing new about that
  • deleted 2mo ago
    [deleted]
  • cleansy 2mo ago
    All organisations also have exceptions to policies. This one would be one
    • inigyou 2mo ago
      You'd be surprised...
    • jeltz 2mo ago
      Yes, but getting an exception approved can be a huge pain in some places.
    • y-curious 2mo ago
      If you work with the government, you don’t really have this luxury
  • smitty1110 2mo ago
    It’s honestly not great. The security guys are completely exasperated at my job, we’re wasting time having with these. You take the scam really, investigate for a bit, write up a DNF with justification, they go and up date records, and we all just kinda hope that someone updates the scans so it stops showing up. Something is going to give, and I suspect that the optimistic open filling is going to get canceled.
  • lelanthran 2mo ago
    Not really. LLMs can hallucinate the patches too :-)
  • dns_snek 2mo ago
    Steps to patch a hallucinated CVE: git stash -m "sigh" git commit --allow-empty -m "Patch: CVE-2026-51302" git stash pop
  • dgellow 2mo ago
    One can hope that will put pressure on the industry to design a better system than CVEs. The signal noise ratio was already terrible before LLMs, I cannot imagine that will still be a meaningful system in 10y. But I’m too cynical to not consider all the middlemen who benefit from the status quo
    • pixl97 2mo ago
      It's going to be very very difficult to build this system as the exploitability of any particular CVE can massively vary depending on your system configuration. There are a lot of things that are bugs, but cannot be exploited in a standard configuration, most people would wonder why this is even a CVE. But then you have those users that would have the application, then something like a report module that was imported, and another 3rd party module that imports images, and maybe another one that arranges the images in a chart. And suddenly that isolated CVE is now exploitable by user written data because of the original bug, it just required a complex and unexpected chain to get there.
      • dns_snek 2mo ago
        > There are a lot of things that are bugs, but cannot be exploited in a standard configuration And this describes the vast majority of CVEs you see when you scan your dependencies. Typical case: Your regex library has a denial of service vulnerability for crafted regular expressions but your program never allows users to have any influence over the regex.
• ChrisMarshallNY 2mo ago
  The problem with this kind of thing, is that it reduces the S/N (Signal-to-Noise) ratio, so weeding out the legit CVEs becomes a lot more difficult. But, on the other hand, I do know that LLMs have been discovering a lot of legit CVEs, and I will lay odds that the blackhats are leveraging them to the max.
  • tcp_handshaker 2mo ago
    [dead]
  • ymir_e 2mo ago
    Edits for clarity: seems like this was a controversial take, which I find pretty interesting. I believe people took this comment as LLMs being better than security aware engineers who have the time to spend building solid systems. This wasn't the point, the reality outside "established" tech companies is that software security can be lackluster. There often simply isn't enough resources to check old software for basic vulnerabilities, outdated packages with known issues, there might be a manager who insists on a certain solution. Or that certain services are on "maintenance mode", but rarely get checked since they're on a certain part of the internal network. If they are able to properly scan their full software stack for CVE issues, they cannot deal with a flood of CVEs. ---- We're in a transition period where AI will eventually make software much more secure than it ever was. These noisy CVEs will probably lead to agents verifying vulnerabilities before humans review them. The problem with agent reviews from what I can think of is: - cost to use LLMs to review things - not necessarily easy to plug-and-play in repos: (domain knowledge + vulnerability knowledge) - especially with anthropic: able to use models defensively, without hitting guardrails The last one is the most interesting one to me. How does the AI providers know if you're a "good or bad" guy? And does it matter if open source models is catching up? We're in a kind of cyber arms race wether we like it or not.
    • rghammt 2mo ago
      Currently we either get AI promo vulnerability dumps like from Chrome with pretty graphs that no one checks or false positives. Where is this one now that was hyped everywhere? https://news.ycombinator.com/item?id=49133889 https://news.ycombinator.com/item?id=49133889 The GitHub submitter could no longer reproduce the issue and the LKML post has no replies: https://lore.kernel.org/all/CALCETrXbj__SFQMzPZhES5y6-sh4np-ZHY5T_=4QY5+Fn8BM4A@mail.gmail.com/ https://lore.kernel.org/all/CALCETrXbj__SFQMzPZhES5y6-sh4np-...
• BedVibe_Studios 2mo ago
  [flagged]
• dvh 2mo ago
  [flagged]
  • mike_hock 2mo ago
    It's not actually vulnerable. You're absolutely right. I made a critical error. It's NOT vulnerable. It' actually vulnerable. You're absolutely right. I made a critical error. It IS vulnerable. It's not actually vulnerable. You're absolutely right. I made a critical error. It's NOT vulnerable.
    • inigyou 2mo ago
      Hilariously, this comment tripped the LLM writing filter and was auto-deaded. I vouched for it.
      • mike_hock 1mo ago
        Thanks, I guess :D
• mlvljr 2mo ago
  Honest take, this is a critical CVE.
  • awkwardpotato 2mo ago
    How so? All 6 of the CVEs covered in the article did not actually exist when investigated
    • mlvljr 2mo ago
      You're absolutely right, I have hallucinated this. Would you like to find some real CVEs next?
    • mahogany 2mo ago
      The comment is making fun of a Claude-ism where it becomes super “honest” about stuff. It’s a joke but there is an underlying real effect where this type of language is psychologically manipulative and I would guess makes people believe LLMs output more than if it didn’t use “honest” (or “load bearing” or whatever super serious important sounding word).
      • ptx 2mo ago
        Or maybe they didn't train it that way to be manipulative (although it's certainly a plausible explanation) but simply as an accidental artifact of trying to make it give honest answers? LLM-generated images sometimes includes text from the prompt as literal text in the image, so perhaps this is the same sort of artifact? If they've told it to be honest, it responds by talking about being honest instead of actually being honest, because it has no actual understanding of anything.
        • inigyou 2mo ago
          The whole RLHF process is structured to train models to be manipulative, no matter what you thought you were training them for.
        • jimktrains2 2mo ago
• Spide_r 2mo ago
  Somewhat related: https://sqlite.org/cves.html https://sqlite.org/cves.html
• trueno 2mo ago
  just ran this article that ran a cve through gptzero.. through gptzero -GPTZero AI Detection -Model 4.8b -We are moderately confident this text is a mix of AI and human -63/88 Sentences likely AI generated
  • nearlyepic 2mo ago
    I mean you didn’t even have to do that, you could tell just by the AI generated header image.
  • vips7L 2mo ago
    Sad days ahead.
• Ekaros 2mo ago
  Not validating submissions seems like avenue for massive attack. Flood the whole system with endless false reports. Thus making it significantly less reliable.
  • masklinn 2mo ago
    That is exactly why many big projects are migrating to becoming CNA, so that randos can’t get assigned unqualified CVEs which nobody has looked at or validated. Apparently RedHat is a CNA of last resort, so it might be possible to get your project under Redhat’s scope and go through them without having to be a CNA yourself.
    • inigyou 2mo ago
      What are the requirements to become a CNA?
      • neitsa 2mo ago
        The OSSF (Open Source Security Foundation) has a nice guide on how to become a CNA [0]. It's pretty involved though. [0] https://github.com/ossf/wg-vulnerability-disclosures/blob/main/docs/guides/becoming-a-cna-as-an-open-source-org-or-project.md https://github.com/ossf/wg-vulnerability-disclosures/blob/ma...
  • insanitybit 2mo ago
    This is what the Linux kernel is currently attempting since becoming a CNA. edit: "This" being to sabotage the CVE system via DoS, to be clear.
    • inigyou 2mo ago
      What do you mean? It isn't a DoS to assign every single bug fix a CVE! Every single bug is making someone vulnerable in some way.
      • 2mo ago
• progval 2mo ago
  > All advisories in this repo seem AI generated when testing them with Gptzero I pasted this blog post from "Analysis Matrix" to the end in Gptzero, and it also says the blog post was AI-generated (71% chance of AI, 29% chance of AI-Human mix).
• sabot90260 2mo ago
  A critical rating on a vuln that doesn't exist is wild. The triage cost still lands on the maintainers.
• throwa356262 2mo ago
  In the mean time, my very real vulnerability reports are not even acknowledged because maintainers are busy handling this kind of nonsense.
  • chrisjj 2mo ago
    Does author rep not count for something?
• account42 2mo ago
  The irony of writing an article about slop reports and then defacing it with a giant unrelated slop image at the top.
  • pelasaco 2mo ago
    I had the impression that the code review done and text formatting by the jfrog was mostly driven by Claude too..
• trashb 2mo ago
  Could be just me but I would be quite careful running these PoC's, especially since the CVE's mostly seem nonsensical. I suppose these CVE's could be targeting some kind of PoC execution instead of actual bug reporting? (I didn't look at the code in depth). Why is the repo even mixing CVE's for "schreibfaul1 ESP32-audioI2S" and "SQLite"? Is mixing CVE's for different products in one repo common practice?
  • inigyou 2mo ago
    There is no specific practice for a repo full of bug reports. Having a repo full of bug reports is already unusual. Normally you report a bug against the software that has a bug, not by creating a file in your own repo.
    • _flux 2mo ago
      I've created bug reports to open source projects that have related files in my repo. Then I can just list the reproduction instructions like: 1) arrange xxx, such that yyy.. or git clone this repo where this is set up. 2) .. I thought that might have been quite helpful not just to the person I'm sending the bug report to, but also to myself when I need to evaluate if the bug has been fixed.
• effnorwood 2mo ago
  [dead]
• kasperset 2mo ago
  As a non-expert in this field, I feel we have introduced more “stochasticity”. Before, We had to deal with some human error/ misunderstanding for most part but now we have to deal with power of condensed human expertise along with homogenized flaws.
• gortok 2mo ago
  We can chalk this up as another example of over-exhuberance by what folks believe LLMs can accomplish vs. what they actually are. LLM-based “AI” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credibility massively when your LLM-based “AI” gets the probabilities wrong at best, or in this case, claims a line of code generates a vulnerability when it is, in fact, a code comment. LLMs are text-prediction engines. They are not Artificial Intelligence, and shouldn’t not be treated in any form or fashion as if they possess intelligence. What bothers me about this entire situation is that presumably the folks that relied on the LLM-based “AI” to generate these vulnerabilities knew (or should have known) enough about their tool to know this would happen, but did not. Now, we all pay the consequence, to the tune of hundreds of thousands if not millions of dollars of wasted productivity from teams that have to deal with the resulting fall-out of this usage of “AI”. A human must verify everything an LLM presents as fact. Everything. If you don’t, we all pay the price. LLMs do not remove the onus of responsibility on the human being, if anything they amplify it because LLMs can generate lots more output more quickly that needs to be verified than humans can.
  • tsunamifury 2mo ago
    [flagged]
  • geraneum 2mo ago
    Unfortunately people sometimes get defensive against this take. But I think treating the LLM as you described can make you a better LLM user and help get better output. It helps understand the failure modes better, and moderate one’s reliance on them. Just like how we should do for every tool we work with.
    • gr_norm 2mo ago
      Yes, I've found that reminding yourself of how they actually work helps keep you on guard against LLM-patterned mistakes. Especially things like carefully considering what parts of the current task likely fall outside the distribution of corpus + RL data (as much as that can be guessed).
• firesteelrain 2mo ago
  > This incident demonstrates a systemic issue with automated vulnerability ingestion But they make X-Ray which does automated vulnerability indexing and matching dependencies to CVEs.
• fl1pper 2mo ago
  Great article! We need to further emphasize the importance of responsibility when using LLM tools to produce output for others. It's great to use them for refactoring and bug discovery, but keep in mind that it's your responsibility to analyze it and iterate on it with AI. It makes your code better and develops technical expertise. The "Hey, analyze that codebase, find all dangerous CVEs, and write a README for the PoCs, so I can post it online for others to analyze, and if I'm lucky, I'll get a paycheck or a title to add to my resumé" approach might work in 0.1% of cases, but it will generate a ton of slop for the community to drown in. I also think GPTZero and other AI detectors have far more false positives than correct guesses. I tried it on several texts & messages I wrote before 2019, and it flagged them as 80% AI-generated. In that case, it's reasonable to assume that AI also generated the README text for each discovered CVE. In other cases, however, we should be more cautious.
• gste 2mo ago
  > The cited code didn't even exist in those versions or referenced unrelated logic. > When testing the PoC payloads they didn’t work (not triggering any crash). I think the future is pretty obvious, if this isn't being done on projects already: you need to automate these checks and reject automatically
  • tantalor 2mo ago
    Great idea! We can use LLM to do this. LLM: I ran the check and it repro'd > Did you really? LLM: You're absolutely right. I didn't actually run the check. Good catch! One sec let me do that now... yep I ran the check and it definitely repro'd this time. > I hate my job
    • timedude 2mo ago
      That is if it didn't completely delete the repo in the first place. Consider yourself lucky
• luciana1u 2mo ago
  [flagged]
• r_lee 2mo ago
  can anyone explain what value does the ai slop image at the top add to this blog post?
• wang_li 2mo ago
  There should be some sort of defamation lawsuit possible when people publicly publish trash repos or reports like this. If you were writing a competing product in a safe language you’d love to see such FUD.
  • inigyou 2mo ago
    It might be possible, but we'll only find out if a lawsuit actually takes place.
• leviyi 2mo ago
  [flagged]
• myshapeprotocol 2mo ago
  [flagged]
• myshapeprotocol 2mo ago
  [flagged]
• taf2 2mo ago
  Llm are great at things that are repeatable. Otherwise they are just like us - great at believable bullshit… a sign of true intelligence
• boundsconf 2mo ago
  [flagged]
• oxydite 2mo ago
  Damn, dunno why I've always assumed that if something gets a CVE, then some authority has reproduced and verified it. Shouldn't it be CNA's job? Why would anything get a number if it hadn't been verified?
  • insanitybit 2mo ago
    The current system is radically overwhelmed and has been seeing cuts over the last few years. > then some authority has reproduced and verified it. No, the system puts a lot of trust into the submitter.
  • inigyou 2mo ago
    It's really just a number allocation and that's all. You can get a number. That doesn't mean the number refers to something useful.
• mavleop 2mo ago
  Thankfully I’m seeing other people mention that this article itself seems AI-generated. While there's less proof, i suspect that the actual analysis was done by an llm as well. So while i like to dunk on hallucinations as much as the next guy, who's to say this analysis doesnt have them too? (for the record, i think the cve's are probably bogus, i just think there's an inherent problem with debunking slop with slop)
• BigTTYGothGF 2mo ago
  They couldn't even be bothered to use a non-AI image.
  • lexicality 2mo ago
    that's fine, I'm not sure they bothered to actually write the article either
• cocorebop 2mo ago
  > Over the past few days, a newly created GitHub repo (programmervuln/cveadvisory-) published a batch of SQLite vulnerability advisories (as part of other 50+ CVEs which we believe are also LLM slop except from one). NVD quickly flagged these as critical, and CISA's ADP agreed. These two sentences seem incompatible to me. A brand new actor can just say some stuff and cause major disruptions in this way?
• minraws 2mo ago
  I think this will give a lot of ammunition for people on both sides, but the thing is we have made software so complex, ofc a lot of it is inherent, that we can't even tell if the vulnerability is hallucinated or not. I find it getting closer and closer to the nonsensical 50 page mathematical proofs built with computer assited tools, that been driving reason for abstract proof systems so that we can break down such complicated examples into something simpler that we can feed into a system to help us verify. I feel like for some reason we have built the output machine before we could even build the actual output verification machine. And aren't ready to accept that we should all move towards building some kind of system to better help verify the "truth" produced by these AI systems. I am now very convinced that human built and proven to work verification systems for LLM outputs are more important than ever, but I just don't see any company or team working towards it. Trillion dollar systems shipping to production code that is hardly understood by any one without any verification what really could go wrong. Honestly I really truly hope nothing goes wrong. To note I am not anti-AI but I want to be able to have my LLM outputs be verified by more than just me under a tighter and tighter deadlines, accepting worse and worse slop just because it would less cost to me to accept the PR and then fix it in prod. LLMs aren't sustainable software development, they are delusional slop factories if we aren't able to actually make sure the outputs have any amount of correctness. With humans it worked because all 10 of us could in some ways verify each others outputs, and had trust in the fact that our timelines included the time for reviews.
• _superposition_ 2mo ago
  Taking security theatre to a whole new level.
• jedbrooke 2mo ago
  nonexistent functions being referenced is pretty bad, I haven’t had that issue since like opus 4. Shows either complete lack of oversight and/or using a bargain basement model (so no even having financial skin in the game). Most likely both
• linuxhansl 2mo ago
  I almost feel that we have a new generation of the "script-kiddie". People without much (or any?) knowledge of software engineering using outside tooling to do "stuff" they wouldn't be able to do themselves. Maybe it's a not a perfect analogy - in this case the intentions seemed to be laudable - but we will see much more of this, including from malicious actors.
• jokoon 2mo ago
  could this be used to create some panic and make some money through trading or something? no idea who submitted all of those, but it sounds like malicious intent
• umarcyber 2mo ago
  It really is a shame that funding cuts at NIST has led directly to this. I don't know if there is an incentive for a non-profit to come out of the woods to do the validation and enrichment process, but boy do we need it. Like gortok has said, LLMs are just probabilistic text-prediction engines. They cannot automate the vulnerability validation lifecycle because they cannot account for improbable edge cases. They may not even get the version of the software right...
  • wildfireday2 2mo ago
    Funding cuts at NIST did not lead directly to this. You must be new here or haven’t spent a lot of time directly interfaced with NIST on cybersecurity in decades past. And NIST’s role with CVE’s has always been purely ministerial/clerical. It has never been their job to do even a cursory investigation into the vulnerability itself.
    • jrochkind1 2mo ago
      OP suggests something at NIST changed in 2024, where they stopped doing as much verification as they did before. To be sure, the suggestion is not funding cuts, but an increase in workload with same funding. Here is the 2024 NIST announcement OP links to: https://nvd.nist.gov/general/news/nvd-program-transition-announcement https://nvd.nist.gov/general/news/nvd-program-transition-ann... > Currently, we are prioritizing analysis of the most significant vulnerabilities. In addition, we are working with our agency partners to bring on more support for analyzing vulnerabilities and have reassigned additional NIST staff to this task as well. The OP described this as "NIST effectively hit pause on deep analysis. " It does sound like they stopped doing something they used to do in 2024. I personally have definitely not spent a lot of time directly interfaced with NIST on cybersecurity in decades past, I know nothing about it, just what I read in OP. Are you saying the OP was wrong to call what NIST used to do "deep analysis", and/or that the thing NIST stopped doing was "purely ministerial/clerical" in a way that it would not have caught fake reports anyway, contradicting the OP? Or other? Again, to be sure, the OP's suggestion was not that this was caused by NIST funding cuts, but by "a massive surge in vulnerability reports,"
• hnjbx769kd 2mo ago
  [dead]
• nirbendavid 2mo ago
  [flagged]
• wbsun 2mo ago
  So a so-called vulnerability has: - cited code not exist, and - PoC not work. How would any serious system allow it become a big deal?
  • 1970-01-01 2mo ago
    Exactly. CVEs shouldn't be allowed to be anywhere near critical without a working PoC or other proof.. Trust+verify.
• jrochkind1 2mo ago
  The federal government ought to be funding NIST sufficiently to actually do sufficient analysis. Do we care about funding civil "cyber" defense or not? (Obviously the answer is not, we only care about funding offensive capabiltiies).
  • inigyou 2mo ago
    The federal government defunded the CVE program last year, I think, because it was woke.
  • Plutoberth 2mo ago
    This is obviously impractical. With the volume of bug reports that are generated and such a wide breadth of software no single agency will be able to handle all reports. And honestly? It doesn't matter, even today. CVE should serve as a reasonably deduped identifier of specific vulnerabilities. It was never interesting and practical to care about 100% of vulnerabilities in a specific deployment, and it's not interesting today.
• dpacmittal 2mo ago
  The duality of LLMs is real. On one hand, they are solving unsolved math problems, on the other hand they get fooled by the strawberry test or the car wash test. We see this all the time on reddit and HN, where some users are building crazy things with LLM and others are swearing by how useless LLMs are. Some people claim SWE will be dead in a few years, and others are claiming the AI bubble is going to burst anytime now.
  • GeorgeWBasic 2mo ago
    The "AI bubble" refers to the massive overinvestment in AI, and doesn't imply that LLMs are going to go away after it pops.
• bilater 2mo ago
  This isn't great but a negative side effect of an otherwise positive change. I know a lot of comments will try to make this into a gotcha but its really not. Decreased friction leads to misuse. Just like before we'll figure out ways (reputation, light filter pass) to deal with it.
• rib3ye 2mo ago
  > Because no step in today's system actually requires a proof-of-concept or bug reproduction, a plausible-sounding fake advisory can slide right through the pipeline and end up in GHSA, downstream databases, and enterprise scanners. I have no experience in the realm of security, but why isn't the submission process similar to any normal software shop (big or small) in terms of required bug repro steps?
• Joel_Mckay 2mo ago
  Sue them for defamation. Seriously, the hubris on these "AI" sloppy clowns needs consequences. =3
• Bitu79 2mo ago
  [dead]
• securitycipher 2mo ago
  [dead]
• mpfh 2mo ago
  [dead]
• ghostnshells 2mo ago
  I think this is expected with the increased usage of AI on both fronts (finding vulns and automating their remediation). Yes, there are and will be companies that help other companies validate CVEs before they raise any alarms, but that noise/signal battle has always been there. I work as a sec eng at a biotech firm and have a tool I built that pulls our ITIL and runs a search on NVD, GHSA, etc. for enrichment and classification. But the number of CVEs I see for products we house is astronomical compared to the actual version of that product we are running (and that's only one value). But that's also a challenge when working with so many sources and having a small team try to tackle them. The age-old validate, remediate, verify proves itself useful once again!
• beyondscaletech 2mo ago
  [flagged]
• Sattyamjjain 1mo ago
  [flagged]