8 ms·
Microsoft says bug causes Copilot to summarize confidential emails
- tablets 7mo agoInitial date of issue 3rd Feb 2026
- dolphinscorpion 7mo agoA bug here and a bug there...
- p0w3n3d 7mo ago100 nasty bugs in the code 100 bugs in the code Take one down Patch it around -127 nasty bugs in the code
- wiredpancake 7mo ago[dead]
- childofhedgehog 7mo ago> However, this ongoing incident has been tagged as an advisory, a flag commonly used to describe service issues typically involving limited scope or impact. How is having Copilot breach trust and privacy an “advisory”? Am I missing something?
- dijit 7mo agoAdvisory doesn't have the same meaning in security research as it does in the english language. Unfortunately "Advisory" is a report written about a security incident, like an official statement about the bug, it's impact, and how to fix it -- which differs from the english meaning... it's not meant to mean to "advise" people or to "take something" under "advisory" (which, is a very soft statement typically).
- layer8 7mo agohttps://www.merriam-webster.com/dictionary/advise https://www.merriam-webster.com/dictionary/advise meaning 2: to give information or notice to : INFORM An advisory gives notice and/or warns about something, and may give recommendations on possible actions (but doesn’t have to).
- _verandaguy 7mo agoWords have multiple meanings depending on context, and here it's at best ambiguous. In the context of security incidents, logging, auditing, etc., "advisory" is often used as a severity level (and one of the lower ones at that). So, yes, technically, it's de-facto advisory to publish this information, but assigning "advisory" as a severity tag here is questionable.
- lich_king 7mo agoThe LLM that wrote this nearly content-free story doesn't know what it's talking about. The basic distinction in the infosec industry is that advisories are what you publish to tell customers that you had a bug in your product that might have exposed them or their data to attacks and you want them to take some specific action (e.g., upgrade a package, review logs); while an incident report is what you publish when you know that the damage happened, it involved your infrastructure, and you want to share some details about happened and how you're going to prevent it from happening again. Because the latter invites a lot more public attention and regulatory scrutiny, a company like Microsoft will go out of their way to stick to advisories whenever possible (or just keep incidents under wraps). It might have happened at some points in their history, but off the top of my head, I don't recall Microsoft ever publishing a first-party security incident report.
- bpodgursky 7mo agoIf you inflate severity, people simply ignore incident warnings. What's the actual action needed here by a security team? None. You can hate it or not care but the end of the day there's no remediation or imminent harm, just a potential issue with DLP policies. Don't make it look like a 0-day that they actually have to deal with.
- indiekitai 7mo ago[dead]
- hippo22 7mo agoHow is this different than any other access control system?
- ses1984 7mo agoWhen you frame it that way, it’s really not that different. The issue isn’t the access control system itself, more so that it’s really asking too much of people who don’t have the skills or understanding to manage it. Teams of trained professionals get it wrong when the scope is limited to a single application or suite of applications, and you think grandma is going to properly manage access control over her entire digital footprint?
- kakacik 7mo agoWell, its maintained by humans to start with, peer reviewed by humans. They fuck up from time to time in extremely limited scope, depending on how much given company is willing to invest into getting quality work, but nothing like this. Humans are clearly not the weak link to be automated away, in contrary. I work in one of the special legal jurisdictions, such fubar would normally mean banning such product from company for good. Its micro$oft so unfortunately not possible yet, but oh boy are they digging their grave with such public incompetence, with horrible handling of the situation on top of that. For many companies, this is top priority right behind assuring enough cash flow, not some marginal regulatory topic. Dumb greedy amateurs.
- deleted 7mo ago[deleted]
- jrjeksjd8d 7mo agoI think the fundamental tension is that AI produces a high volume of low quality output, and the human in the loop hates reviewing all the slop. So people want to just let the AI interface directly, but when you let slop into the real world there are consequences.
- codeulike 7mo agoReads to me like it is not accessing other users mailboxes, its just accessing the current user's mailbox (like its meant to) but its supposed to ignore current user's emails that have a 'confidential' flag and that bit had a bug
- layer8 7mo agoI think the issue is that the confidential information is being sent to cloud AI, against DLP policies.
- tremon 7mo agoI think that Microsoft would rather not acknowledge that one. It's much easier to hide behind a simple "bug" than to admit to such a massive security breach.
- layer8 7mo agoNot a bug, a “code issue”.
- doodlebugging 7mo agoIt's a feature now.
- SoftTalker 7mo agoI.e. LLM slop code that wasn't adequately tested.
- HeavyStorm 7mo agoExactly.
- gortok 7mo agoThere are two issues I see here (besides the obvious “Why do we even let this happen in the first place?”): 1. What happened to all the data Copilot trained on that was confidential? How is that data separated and deleted from the model’s training? How can we be sure it’s gone? 2. This issue was found; unfortunately without a much better security posture from Microsoft, we have no way of knowing what issues are currently lurking that are as bad as —- if not worse than —- what happened here. There’s a serious fundamental flaw in the thinking and misguided incentives that led to “sprinkle AI everywhere”, and instead of taking a step back and rethinking that approach, we’re going to get pieced together fixes and still be left with the foundational problem that everyone’s data is just one prompt injection away from being taken; whether it’s labeled as “secure” or not.
- doctorpangloss 7mo agoAll the vendors paraphrase user data, then use the paraphrased data for training. This is what their terms of service say. They have significant experience in this. Microsoft software since the 2014, for the most part, is also paraphrased from other people's code they find laying around online.
- benterix 7mo ago> All the vendors paraphrase user data, then use the paraphrased data for training. This is what their terms of service say. It depends. E.g. OpenAI says: "By default, we do not train on any inputs or outputs from our products for business users, including ChatGPT Team, ChatGPT Enterprise, and the API."[0] [0] https://openai.com/policies/how-your-data-is-used-to-improve-model-performance/ https://openai.com/policies/how-your-data-is-used-to-improve...
- shakna 7mo ago"By default" is a fantastic escape catch in the language used there. So... What are the exceptions?
- deleted 7mo ago
- deleted 7mo ago[deleted]
- 52-6F-62 7mo agoNone of this should surprise anyone by now. You are being lied to, continually. You guys need to read the actual manifestos these AI leaders have written. And if not them, then read the propagandist stories they have others write like The Overstory by Richard Powers which is an arrogant pile of trash that culminates in the moral: humans are horrible and obsolete and all should die and leave the earth for our new AI child Which is of course, horseshit. They just want most people to die off, not all. And certainly not themselves. They don't care about your confidential information, or anything else about you.
- josefritzishere 7mo agoAI is such garbage. There is considerable overlap between the security practices of AI and that of the slowest interns in the office.
- observationist 7mo agoSeems like every day there's another compelling reason to switch to Linux. Microsoft is doing truly incredible work this year!
- etchalon 7mo agoApple not doing much better, but from the other end. Microsoft releasing overly ambitious features with disastrous consequences. Apple releasing features so unambitious it's hard to remember they're there.
- observationist 7mo agoPerformance is also degrading on iphones as software bloats, and/or they're up to their old shenanigans and making older phones unbearable to force people to buy the newest ones. Big tech is reaping what they've sown in a very satisfying way.
- import 7mo agoWe can safely assume that Apple will do much better compared to MS until they put AI to the Finder and Dock.
- varispeed 7mo agoDon't forget Apple handwaving serious security issues of their devices - users still cannot even check if their devices are compromised and only thing Apple can do here is "lockdown mode" - which again, after compromise is likely useless anyway.
- marcosdumay 7mo agoThe problem with the Microsoft features is really not excessive ambition. Half of the time it's open user hostility and blatant incompetence. The other half it's just the incompetence. Ambition doesn't enter the picture at all.
- etchalon 7mo agoEh. I think it is ambition. It's a lot product managers coming up with ideas, I think, and teams with a mandate to release those ideas.
- _slih 7mo agocalling it a bug is generous. the whole point of these tools is to read everything you have access to. the 'bug' is that it worked exactly as designed but on the wrong emails
- pu_pe 7mo agoMicrosoft somehow sees a future where LLMs have access to everything in your screen. In that dystopia, adding "confidential" tags or prompt instructions to ignore some types of content is never going to be enough. If you don't want LLMs to exfiltrate content then they cannot have access to it, period.
- autoexec 7mo agoMicrosoft wants access to everything in your screen (as well as the contents of your personal files) and feeding that to an LLM just makes it easier for them to profit from that data
- steve1977 7mo agoI'm shocked. Shocked!
- tartoran 7mo agoOh, poor desperate Microsoft. No amount of bug fixing is going to fix Microsoft. Now that they've embarked on the LLM journey they're not going to know what's going to hit them next.
- asdefghyk 7mo agoWhy was this bug not found in testing?
- deleted 7mo ago[deleted]
- nickdothutton 7mo ago"...including messages that carry confidentiality labels." Trusted operating system Mandatory Access Control where art thou?
- merb 7mo agoI more and more see a bug in my mouth that tries to encourage my boss to cancel Microsoft 365. I did not find the root cause yet
- ok123456 7mo agoAll these government contractors are forced to pay astronomical cloud bills to get "GCC-High" because it passes the right security-theater checklist, and then it totally ignores the DLP settings anyway!
- wartywhoa23 7mo agoAn exemplar BaaF corporation (Bug as a Feature).
- bronlund 7mo agoMicrosoft deploying buggy software is hardly news.
- 8cvor6j844qw_d6 7mo agoIs this a real bug or is it a "lets train on more emails" by being careless? I assume that whatever that is processed by AI service are generally retained for product improvements (training).
- surcap526 7mo ago[dead]
- allthetime 7mo agoThis is one of many reasons we are taking all our current and future private repos off of GitHub.
- jaybyrd 7mo agomicrosoft may very well be the MOST sinking ship to ever sink.
- DecoPerson 7mo agoI wonder, is Microsoft doing “outsider trading”, where they covertly pipe analytical data to the executives’ independently-owned stock trading houses as “tips”? They’ve had access to so many corporate internal emails for so long, with MS365, but Copilot is the perfect way to mask such analysis. Also Copilot would be good at analysing emails and providing useful “tips”. Just my whacky conspiracy theory of the day!
- ghostclaw-cso 7mo ago[dead]
- gh2k 7mo agoThe article doesn't say if the confidentiality labels were created with encryption. I've been using the latter (with Preview DLP) to prevent emails leaking out to _external_ integrations, which can't access the keys. With MS internal tooling, it's feasible that it access to the key, in which case that would be even worse. Does anyone know if this happened?
- mikrotikker 7mo agoThis company is an absolute joke now, if you're not desperately trying to jump ship at this point then you will go down with it.
- Blackstrat 7mo agoJust one more reason to abandon Microsoft. If ever Linux had an opportunity to breakout on the desktop, the proliferation of "AI" and privacy intrusion from the likes of Microsoft would seem to have opened that window. Yes, it would mean giving up some applications, at least temporarily, but the benefit in control and privacy makes that a fair trade off. Unlike the majority here, I don't want ANY "AI" features on my desktop, phone, car, or any appliance that I own. This is true of the "cloud" as well. Trusting corporate entities to have your best interests in mind is naive at best.