21 ms·
The AI aspect of this is a red herring. The real problem is that they're secretly adding in a unique identifier into every image you create. If somebody does no
by weberer 23d ago
The AI aspect of this is a red herring. The real problem is that they're secretly adding in a unique identifier into every image you create. If somebody does not like your meme, they can just send a copyright subpoena to Microsoft to instantly get your full name, address, email, phone number, and any other data associated with your Microsoft account. Just like age verification, this is another weapon in the war against internet anonymity.
- nemomarx 23d agoDoes it trigger on non AI images? The post doesn't say so at least.
- Someone1234 23d agoIt does kind of say: The GUID is coming from the moderation endpoint, which is hit when you generate a local or cloud AI image based on your prompt. If there is no prompt, there is no endpoint, and likely no GUID. Obviously Paint could have been watermarking prior to AI though, but this specific AI watermarking appears to be only that.
- londons_explore 23d agoThe fact that local ai image generation uses an online moderation API is a bit worrying too.... Why not just mod the app to not call this API?
- nvme0n1p1 23d agoSince the watermarker runs locally, you could also do the reverse: generate some porn with a different open model, then run that code to tag it as "Content watermarked by Microsoft Responsible AI"
- autoexec 23d agoProbably both so they can change what they censor without waiting for you to run windows update and also so they can collect your information (IP, timestamp, etc) to associate with whatever you did
- szatkus 23d agoI don't think they use a model that would run smoothly on most hardware that normal people use. Besides, you need to sign-in and pay to use that feature. It's very obvious that's not local.
- iririririr 23d agodid you even read the article? you're wrong on all points.
- szatkus 23d agoFrankly, I only skimmed it. Now I see that "On Copilot+ PCs, image generation is local but prompt moderation remains remote". I don't have a Copilot PC, but if you still need to pay MS to run the model on your own hardware it's laughable.
- Scharkenberg 23d agoNo, you don’t need to pay for using on-device models.
- qurren 23d ago> address Do not tell Microsoft where you sleep. IANAL but they are not a government or financial institution and do not have a right to that information. Make sure you register mailing addresses with your credit card institutions in addition to your residential addresses, and make your mailing address your billing address so that you aren't forced to tell a thousand businesses like Microsoft where you sleep.
- BitwiseFool 23d agoWindows 11 effectively forces users to register with a Microsoft account. Once that's established, all it takes is for an unaware user to fill out an e-commerce form and save an address for auto-fill.
- clear0250 23d ago> Windows 11 effectively forces users to register with a Microsoft account. It takes zero effort to bypass that with Rufus, if you set up your own pc.
- BitwiseFool 23d agoMy comment was in regard to the average PC user. The kinds of folks who would never install an operating system and would likely agree to use Edge while signed in to their Microsoft account.
- 0cf8612b2e1e 23d agoMicrosoft has also been pushing hard for Recall and recording all local activity forever. Not impossible to imagine that anything looking like a home address “somehow” gets ingested in the telemetry.
- AngryData 23d agoA bypass existing is good, needing a bypass in the first place is still a problem.
- pjc50 23d agoThis really needs to be hit with the GDPR hammer. Microsoft have not obtained consent for this.
- baby_souffle 23d ago> This really needs to be hit with the GDPR hammer. Microsoft have not obtained consent for this. At best they'd just disable it for EU... assuming they didn't successfully argue "it was in the ToS ..."
- red_admiral 23d agoBut the EU mandates watermarking of AI content.
- jkaplowitz 23d agoAs the article explained, the EU does not mandate a prompt-specific GUID, only the ability to identify the content as AI-generated. The highly privacy-invasive level of provenance tracking which Microsoft has added goes beyond the EU’s new mandates.
- mosura 23d agoWhy would the EU possibly object to this? It is exactly what they want.
- GJim 23d agoUmmmmmmmm. No. Inserting a tracker that can personally identify me without my explicit opt-in consent is a GDPR breach. I'm surprised you do not not this.
- mosura 22d agoSo you will defeat chat control by sitting there going “but you don’t have muh gdpr consent for that!”? The whole EU vision is they know what the little people do all the time, think all the time, and spend their money on all the time. For the children, obviously.
- dagaci 23d agoThis a variant of the provenance scheme: C2PA its implemented by all major Camera maker, and Google it seems, Apple support it with 3rd party apps on iPhone, but they have something called "Apple Reference Image" brewing. Personally I think there is a good argument for being able to distinguish AI generated image and video...
- akersten 23d ago> I think there is a good argument for being able to distinguish AI generated image and video... Neat but that's not what's being built here. What's being built is "we can trace back this content to who made it" which is bad. Doesn't matter if today that it's limited to AI generated content. Won't be tomorrow. Your devices should not act against your best interests. No cop in my pocket please.
- herf 23d agoNo - there is a huge difference between "this AI created this image" and "this user did it" - the first we need more of, and the second is a big privacy concern. The article does not say anything about identifying a user.
- dmantis 23d agoArticle literally says it adds a guid, not a binary field indicating that the image is ai generated.
- refulgentis 23d agoA guid isn’t a user id, if it is in this case, it’s an abuse of how guid is used, at least colloquially. I haven’t read enough to understand if it is user / machine id, I.e. only globally unique in the sense it’s a globally unique entity identifier.
- themaninthedark 23d agoMicrosoft Can Track Users via a Windows Device ID https://news.ycombinator.com/item?id=48815196 https://news.ycombinator.com/item?id=48815196 Microsoft GDID telemetry includes full browsing and gaming history https://news.ycombinator.com/item?id=48787239 https://news.ycombinator.com/item?id=48787239
- deleted 23d ago[deleted]
- dylan604 23d agoThey stated the GUID is used to identify the prompt used to generate the image. How are you confused as to not being able to identify the user? "I haven’t read enough to understand". Oh, now I know the answer to my question
- refulgentis 23d ago
- frollogaston 23d agoWell yeah they know my John Doe info
- pizzafeelsright 23d agoadd your IP, location, provider, computer specs, dimensions, screen info, nearby devices, etc etc etc Ain't nobody anon anymore thanks to the image recording GPS radio in the pocket.
- fishfasell 23d agoExactly. Forget the AI aspect, this is entirely to identify users for any purposes they deem necessary. People are ignoring the surveillance state aspect of this. Reminds me of the device id debacle they have attached to their outbounds Windows network calls
- red_admiral 23d ago> every image you create *with the help of AI*. Does in fact make a difference.
- tavavex 23d agoWhile it does make a difference, their willingness to quietly integrate watermarking features into what people saw as simple apps for doing simple tasks is unnerving. It only takes a small change for them to start baking your identifiable information into all images they edit, or some government politely asking them to do that. I wonder if in ten years we'll have a horrifying world where everything that leaves a machine is imprinted with its permanent identifier. Every file comes with a verifiable history of who created it, what computers it passed through, who made edits. We're closer to that world than we think.
- Telemakhos 23d agoThere is a certain convergence of wills here: watermarking AI products on the one hand, and infusing AI into everything on the other hand. When you layer generative AI into Notepad, once just a raw text buffer, then you’re setting the stage for watermarking everything because AI touches everything.
- inigyou 22d agoAI watermarking is now the law.
- chrisjj 22d agoImagine the horrifying world where nothing is - and a so-called AI is thereby free to impersonate anyone.
- tavavex 22d agoWe already live in that world, and the sky hasn't fallen so far. Even in the worst case scenario, the world isn't going to be as horrifying as the alternative, because people react to the proliferation of scams, they will not keep falling for fake impersonation for the 10000th time a row. In this scenario, we simply will stop taking factual-looking images at their word in the same way how you don't take any piece of writing on the internet as factual, verifiable truth. Journalism will rely on a person vouching for their images with their reputation, and images with unknown sources will be looked at with suspicion. I am not willing to pay the price of sending every bit I create to some megacorporation's server or stamp it with my only identity so the computer can tell me what's AI and what isn't. Especially not while the average quality settles into an uncanny valley that's often discernible with the naked eye, both for text and images.
- flockonus 23d ago[flagged]
- Scaled 23d agoI am a human and I upvoted the comment immediately. Do not underestimate the wide ranging hate for age verification. Even my normie friends are against it. Meta/etc have been buying those laws but at the cost of more and more people waking up to privacy activism. I predict in a couple years it'll backlash as people first fight against AV, and then fight for more broad privacy reform in the US. At least for myself, I had never been actively calling my reps until the AV laws.
- Terr_ 23d ago> a unique identifier into every image you create Printing (even text) is also a risk: It's very likely your printer is secretly adding marks to the page that contain its serial number and the current timestamp. [0] Meanwhile Microsoft (and Apple) have "telemetry" harvesting those serial numbers of all internal and external devices you've ever had connected or reachable. Then they link them to your MS/Apple account, IP addresses, and the extended social-graph of all computers that were ever in the same room or shared the same bluetooth speaker. In short, your "anonymous" flyer critiquing The Regime and depicting Dear Leader as a clown could lead thugs straight to your door. Or to the door of whomever you're staying with. [0] https://www.eff.org/issues/printers https://www.eff.org/issues/printers
- varispeed 23d agoand then in few years a new mono-mustachioed or mono-browed leader will emerge who declares memes a blasphemy punishable by death and will call up all the telemetry gathered to punish all involved.
- Terr_ 23d ago"I don't worry because I have nothing to hide, based on my perfectly accurate perception of today's political and legal forces in my state and nation which will stay safely static forever."
- pbhjpbhj 23d agoYou know a certain dictator denies access to the country they control based on whether visitors "like" them? They use a private militia, funded by stealing from taxpayers, in contravention to that country's laws, to target citizens who are not supporters of them. Some of those citizens get shipped abroad to foreign prisons by these militia. Others get shot in summary executions in the streets - the controlling regime covers it up with false media reports, and uses corrupt judges to evade scrutiny ... a certain software company support that regime, even acting against foreign scrutiny on their behalf. Essentially all of that country's businesses that are close to the regime need to be considered hostile. We don't need to wait a few years, the revolution already happened; the insurrectionists were freed. These companies paid their tributes, in dollars, to the regime. Maybe next ML will be integrated into software suites to enforce that regime's lies? Most Western governments use such software, the distributers of which have already shown they'll act in the regimes interests against supposed allies...
- hiccuphippo 23d agoCan this also be weaponized? Get an innocuous image from someone you don't like, grab their GUID, add it to another image, sent it to the thin skinned politician in power.
- dbmnt 23d agoNo, because the GU in GUID stands for globally unique. Adding it to a second image would cause a detectable collision. Also, C2PA, another technology mentioned in the article, means tampering is easily detected.
- janc_ 22d agoCollision only detectable when you have both messages. Detecting a forgery seems more likely.
- chrisjj 22d ago> Adding it to a second image would cause a detectable collision So? Still the weapon worked.
- joshheitzman 22d agoIt depends on which version of GUID it is. Only version 3 and 5 are hash based. The other versions are random, time, and/or MAC address based. Version 3 uses MD5 and its cheap to get the same hash for different images: https://crypto.stackexchange.com/questions/1434/are-there-two-known-strings-which-have-the-same-md5-hash-value https://crypto.stackexchange.com/questions/1434/are-there-tw...
- pbhjpbhj 23d agoThat was largely my read of the situation too - it would be a colossal GDPR breech against every EU user, surely? That's got to be a €Billion fine?? Presumably USA are complicit in this spying on allied countries - did the countries know, is it a Five Eyes thing?
- altmanaltman 23d agoWhy would they need to subpoena Microsoft and why will Microsoft hand them the data if someone doesn't like my meme? I understand what you are trying to say but it does not make a whole lot of sense even from Microsoft pov. They are not bound by law to hand over personal data to anyone without a warrant issued. If a warrant is issued then it is their lawful duty to give that data, the same would be done by you if you were in their shoes or your office or your family. Law is not optional to follow. But Microsoft doesn't need to do watermarking on memes to track you or to do this as a war against "internet anonymity".
- adrianN 23d agoWatermarking is just one puzzle piece in the surveillance state; the legislation around it another.
- altmanaltman 23d agoI would say the watermarking is not really a puzzle piece at all when it comes to the surveillance state, which was my point. They don't need watermarking to know who you are or to get information about you. So why would they work on this really visibile method if their intention was to progress the survelliance state? Microsoft with all their resources (and assuming it is an evil corporation purely acting from the state's interest to spy on you) will use this to track you?
- pdntspa 23d agoYou can incentivize cooperation without having to compel action with a warrant. This sort of corrupt quid pro quo is quite common, in other countries at least...
- fchicken 23d agoIt's like Snowden said (paraphrasing): "If privacy comes from policy, the policy can be changed on a whim, and in effect you have no privacy at all"
- shevy-java 23d agoIndeed. This also means I can no longer use Microsoft apps, because they spy on me.
- benterix 23d ago> Microsoft to instantly get your full name, address, email, phone number, and any other data associated with your Microsoft account. Provided you bent the knee and created a Microsoft account.
- icantevenhold 23d agoI was forced to create multiple as part of the mojang migration - truly a nightmarish experience to use their AuthN, haven’t seen something quite as bad since
- bobim 23d agoMaybe you were "tricked into", but probably not forced. We are free to refuse, at cost, but free.
- inigyou 22d agoThe cost is that you can never play Minecraft again
- bobim 22d agoIt might be small cost in retrospect.
- icantevenhold 22d agoNah I sunk many thousands of hours into these accounts; definitely not a small cost. Sure I could get a new account and pay twice for the game I already purchased multiple times - that would’ve ground my gears too
- JacobKfromIRC 22d agoI think the new account would have to be a Microsoft account also, so you wouldn't have even gained anything by not migrating. Some servers allow joining without a Microsoft account even these days, but probably can't play on any of the servers you regularly play on, if you don't have a Microsoft account.
- m3047 22d agoHow can they send a copyright subpoena for work which cannot be copyrighted? There is a contradiction here between "no it's FAIR USE" and copyright.