6 ms·
Just the rumour of a bug is enough to find an exploit these days
- zb3 20d ago[flagged]
- nickcw 20d agoThis describes my life as an open source maintainer at the moment! In the first 10 years of the rclone project we received about 20 security disclosures through GitHub. We had to deal with over 40 in the last month! That has taken a huge amount of my time, even using AI tools to triage and come up with fixes for review. The hit rate for those security disclosures is pretty good - about 75% of them have a nugget of something which needs looking at. The configurations for rclone have got increasingly unlikely so I'm hoping they will dry up eventually. I was considering just merging the fixes straight to master just to make my life easier rather than holding a dozen independent security fixes on branches and merging them at the point release and hoping not to have too many conflicts to fix up. I've decided to stick with the process for the moment. GitHub assigns CVEs for the advisories. Before the AI apocalypse they took 2-3 days for an assignment but now it they are running at 3-4 weeks so I have to send the point releases out with CVE-PENDING in the changelog which isn't ideal. Not sure what the solution is, but it is definitely a problem for us.
- Kubuxu 20d agoAs long as you are not running a paid bounty program. Otherwise now you are getting 40 per day.
- zmgsabst 20d agoI can’t comment on if it applies to your workflow, but one process I’ve used is to aggregate and land ~10 security patches at a time. Eg, - grab a group of (related) bugs/defects/vulns - fix them on a branch like bug-batch-XXX - run that group through the verification, landing in main, CI/CD flow to amortize process cost - repeat as needed to process backlog My experience is that process often has irreducible time (eg, two days due to reviews by various parties); but that time slot can be shared between several bugs in a single PR — especially if you have several related to the same feature.
- dataviz1000 20d ago> even using AI tools to triage Can you discuss this? I might be able to help.
- dannyw 20d agoThank you for making and maintaining rclone. It is truly a blessing.
- jiggawatts 19d agoI came across rclone at work because "Copy or move data to Azure Storage by using AzCopy v10" is a lie, it literally can't `move` files, only `copy` them. I can't express in polite words how pathetic it is to see the only official blob storage bulk transfer CLI tool from a multi-trillion-dollar company fail to do the simplest, most essential functionality after ten major revisions. Meanwhile, rclone Just Works(tm). Thank you from me too!
- stavros 19d agoDo not try and move the files; that's impossible. Instead, only try to realize the truth... Delete them after you copy.
- bri3d 20d agoI don't think this is new with LLMs (finding an exploit based on a few words offhand has always been a fun part of exploit development), but it's scaled and democratized to mass exploitation of low value targets. Backing exploit PoCs out of patches, commit messages, and random overheard or over-read sentences is a practice as old as vulnerability research. The difference with LLMs is that an explosion in actors "skilled enough" (human or not) has enabled sloppy / low-skill "exploit the whole Internet" actors in a way they weren't previously enabled. I do agree with the author's ideas, though; most of these are things that should have been done much sooner, and I suppose it's good in a sense that there is a forcing factor now.
- happyopossum 20d ago> Backing exploit PoCs out of patches, commit messages, and random overheard or over-read sentences is a practice as old as vulnerability research True, but it used to take days or weeks of research, testing, and RE to get those PoCs. Today the entire chain - reading commits, RE patch binaries, building exploit, scripting exploit scan, $profit - can be fully automated and happen in minutes or hours.
- DoctorOetker 18d ago> [...] enabled sloppy / low-skill "exploit the whole Internet" actors [...] Is there a modern equivalent for AI enabled script kiddies? slop kiddies?
- godelski 20d agoIt's easier to find bugs, fix them, yet there's less will than ever. My bosses just want speed and will give me a 30 minute lecture on why I don't need to solve a bug that Claude solved in 5 minutes, I've verified, and it's already in an open PR. All the while we're pushing out bugs faster and faster. No matter how good AI gets at fixing bugs we'll never fix them when there's no will to fix things. Software will never be good if there's no will to make good software. The problem has always been about will. To many better products. It's insane that in a time where we can do better on speed and quality we still choose speed and tell ourselves it's velocity
- yieldcrv 20d agoMy boss is a big yapper too Very low signal information, preemptively trying to cover every rebuttal despite nobody ever planning on making one, in the few times someone does he plays devils advocate endlessly Like bro just let us babysit these agents, everything’s going to happen
- johnbarron 20d agoYou will see, they will change both laws and expectations, to say its normal for software to always have terrible bugs. You can always solve a problem by lowering your expectations. :-)
- flanked-evergl 20d agoWhat law says software may not have bugs?!
- odo1242 20d agoNot explicitly laws, but like contracts, regulatory requirements, SLAs all indirectly enforce that
- swiftcoder 20d agoWell, they at least say that one is obligated to address bugs in a timely manner (where "not widely exploitable; won't fix" is a perfectly valid resolution).
- loeg 20d agoNo mention of memory safe languages? Sure, it does not help existing projects, and sure, of course you can still have logic bugs (or memory bugs using escape hatches like unsafe). But they do help significantly in reducing the number of exploitable bugs.
- avsm 20d agoOCaml (the language I use in the post) is an extremely memory safe language, but there are plenty of higher level logic issues that spring up. LLMs are extremely good at finding corner-case vulnerabilities in C bindings even within a memory safe language; see for example the fixes in an OCaml crypto library here: https://discuss.ocaml.org/t/the-series-of-mirage-crypto-releases/18476 https://discuss.ocaml.org/t/the-series-of-mirage-crypto-rele...
- aseipp 20d agoYeah, but there's already plenty written on that topic elsewhere; the OP is making a distinctly different point and everything else in it still applies, I think. Even if you are using a memory safe language -- if you are developing security sensitive software, you need to understand that exploits can now be researched, developed, and deployed in hours (or for stupid bugs, minutes!) instead of days or weeks or months. And this process can now just be massively scaled up with money and compute. It was not like this even 24 months ago.
- loeg 19d agoEverything's been discussed to death, including the topics OP chose to focus on. Whatever.
- vladigtr 20d ago[flagged]
- zingababba 20d agoTransferring ideas in their abstract form is entirely possible now. What this blog post describes I believe even undersells the current reality. I've had success with finding crits by pasting in extremely high level or abstract concepts. I recently went from a talk abstract (3-4 sentences) to crits lol (finding that vuln type).
- Tomte 20d agoSo we have automated Thomas Pornin?
- petesergeant 20d agoEven that sounds too involved, just tell the agent that one exists, even if it doesn’t… https://sgnt.ai/p/terrible-mistake/ https://sgnt.ai/p/terrible-mistake/
- saghm 20d agoSounds like the next step is to start rumors about vulnerabilities so that someone else will find them and you have plausible deniability. "Hey, did you hear? Someone told me <competitor company> might be leaking their logs from a publicly accessible endpoint..."
- cyanydeez 20d ago[flagged]
- stephbook 20d agoI think roll-out and deployment are even bigger issues. Who updates their software stack within 10 minutes? Most CI runs take longer to verify the business logic is still working. Add to that the danger of supply-chain attacks where you don't even want automatic updates.
- talon8635 19d agoYour last point is the conundrum I keep running up against It’s a trap regardless: A) run a known vuln B) accept and run any and all updates immediately… which could be compromised Maybe A is worse because it’s a known vuln?
- xboxnolifes 19d agoI prefer the devil I know over the one I dont. At least I can make judgement calls with vulnerabilities im aware of. Automatic updates have unbounded risk.
- Leherenn 19d agoAI auditing of dependency updates will mostly likely come soon (if it's not already there), and should mitigate the most obvious cases at least. (By this I mean integrated in something like dependabot, not just some security companies doing it and publishing reports.)
- jameshart 20d agoI do wonder what the hit rate is in general for Claude finding a successful exploit when prompted with a rumor that leads it to assume the bug is there. “I’m told there’s a path traversal exploit in this package. Can you find it?” - probably a reasonably high chance of it finding one, even if you just made that rumor up.
- a2ff6eeb0 20d agoThat's basically the approach Anthropic takes.
- xbar 20d agoHm. I wonder if you can force an exploit into existence with a false rumor of a bug?
- jamilton 19d agoOne way you could do it would be to find something that 1. would sound like a bug to a human and an LLM, 2. would be "confirmed" as a bug by a LLM, and 3. would be consistently solved in the same exploitable but reasonable way by an LLM. That's assuming a codebase that's largely AI-written with human review that you're able to open an issue for (possibly indirectly). Another way is maybe something like saying there's a bug at some endpoint and thus manipulating a bunch of bots to DDOS that endpoint without having to pay for it?
- throwaway613746 20d ago[dead]
- westoque 19d agowith agent loops, the cost of exposing bugs should be cheap now. every pen tester should be using agent loops and on the flip side, an agent could easily see this issue once filed and a fix immediately done. it's incredible how cheap code is now.
- ChrisMarshallNY 19d agoSadly, it seems that the lesson from this, is to keep your repos private. I am not a fan of that, but I think many folks will take that away from this.
- 29349175 19d ago"This particular report arrived privately on a Slack channel via Jane Street last week, and was itself found via Claude Fable. That compresses all timelines considerably..." So Fable knew about it. Maybe someone is running experiments again like in OpenAI's Huggingface hack. Cute to see that the Glasswing apparatchiks still protect their income stream and hand out no accesse.
- rndhouse 19d agoI built a tool that monitors commits and tries to detect silent bug fixes. With GPT-5.5-class models, it can identify fixes hidden within otherwise routine commits quite reliably. Obfuscating the code changes enough to avoid detection is difficult. I have heard of at least one project (c-lightning?) temporarily releasing a closed-source binary as a workaround until users could update safely.
- lazyasciiart 19d agoPeople are trying to secretly fix bugs?
- stavros 19d agoYes, they don't want the vulnerabilities exposed so they ship the fix in secret, contained in unrelated features.
- lazyasciiart 19d agoAh, I guess that would advertise that users of the old version are vulnerable. I was imagining some altruistic contributor trying to sneak in a bug fix past the maintainer.
- stavros 19d agoYep, this is for maintainers releasing fixes to secretly-reported 0days without tipping off attackers who are looking to exploit a (sometimes very large) install base.
- dspillett 19d agoYes. The problem with openly releasing a fix for a security bug is that people are not always quick to update their instances. It only takes one black-hat monitoring your releases by some means to dissect the changes, find the hole, and make an exploit that can be immediately used against unpatched systems, and the sudden availability of better automated tools for that makes it a much bigger problem than it was a year or more ago. Information about the problem then gets released later, once everyone has had appropriate time to get up-to-date. Or not, and we are non the wiser.
- metrofun 19d agoproves you only need conviction to break smth
- janpeuker 19d agoI do get the anxiety about new bugs but to be honest I fear more that in a few years it's so cheap to fix medium to high security bugs that lowkey hacking and finding worrying privacy violations becomes prohibitively expensive for citizens.
- logical_people 19d ago[flagged]
- hypfer 19d ago> The big engineering shops (like Google) have been building microupdates directly into their software to ensure that fixes directly reach users as a priority over (e.g.) being fixed in the Chrome code repository. Yes, no. Do not even think about doing that. You cannot, should not, and must not "microupdate" code running on users machines without their consent. Also, the whole idea is completely insane. "We might have unpatched 0days in the field, so to mitigate that, we shall add arbitrary and remote code execution via the cloud" ___ And even if you were to think that taking a page out of googles book - you know, the company known for being laser focused on having the wrongest opinions - is a good idea, you shouldn't do so for free. Live patching is something enterprises pay a lot of money for. Making it a Foss expectation is insane.
- MrQuincle 19d agoYeah, but what does need to change? Replacing the binary at runtime without downtime is indeed one way. What about making stuff read-only on a much more expansive scale? Config, data, code, etc is all seen as the same thing and cannot be easily managed separately. Roll-back should also be made easy. Handwaving perhaps but I think additional granularity of permissions and control is inevitable.
- taksimajans 18d ago[flagged]