8 ms·

MP Maria Miller wants AI 'nudifying' tool banned

38 points by unklefolk 5y ago

basisword 5y ago
It will be interesting to see how this kind of thing plays out. I’m sure it’s quite distressing if a tool like this is used on your photo and then potentially shared in your friendship group. Hopefully we very quickly get to the point that nobody will ever be able to know if a photo is real or fake and it’s just not considered an issue therefore. Policing it seems like it would be extremely difficult. Maybe we police the intent? In other words you can produce the images but if you use it maliciously against a person then there is a crime.
- have_faith 5y ago
  I'm wondering if there are existing laws that would cover the intent part. Defamation and harassment laws etc. Maybe they just need amending. Trying to police the apps exact functionality directly seems counter productive.
  - judge2020 5y ago
    I wonder as well, there have been people putting their friends or adversaries’ heads on nude model bodies ever since photoshop came out. Is that a crime and has it ever been prosecuted (maybe for defamation)?
- pjc50 5y ago
  > Hopefully we very quickly get to the point that nobody will ever be able to know if a photo is real or fake and it’s just not considered an issue therefore. The ongoing "fake news" crisis proves that we are, in general, bad at spotting fakes - that is the purpose of a fake - and will also vehemently disagree about what is fake. Especially when it can be used for political purposes. Expect the first world leader brought down by a deepfake in the next decade. I'm reminded of https://www.theverge.com/2017/7/12/15961354/pakistan-calibri-font-scandal-forged-documents https://www.theverge.com/2017/7/12/15961354/pakistan-calibri...
JulianMorrison 5y ago
More precisely, she wants the distribution of nudes without consent, real or faked, to be considered an offense. This isn't about the tool, but what's being done with it.
- xibalba 5y ago
  This is an important difference, and, for me at least, the difference between disagreeing (“ban the tool”) and agreeing (“make non-consensual nude distribution illegal”) with the proposal. It’s a shame the headline communicates the former when it seems the proposal is the latter.
  - st_goliath 5y ago
    > It’s a shame the headline communicates the former when it seems the proposal is the latter. Because a headline blubbing about AI-X-ray-specs grabs attention. Making people more likely to click the link. You can see the effect in action right here: the link made it to the top of the front page in less than an hour of being submitted. Besides that tough, the article does contain the typical misunderstanding of politicians about how software works ("software developers should be forced make $SOFTWARE so that it can't do $MY_SPECIAL_EXCEPTION_CASE"), which is problematic per se, unrelated to any machine learning aspect.
- buro9 5y ago
  Taking this to an extreme: If an image is 100% generated and looks to be a nude of a human, from whom should consent be obtained? Especially if "this image is similar to me" is factored in, what degree of similarity makes an image a representation of a real person? We're not far from this being less about privacy and dignity, and being more about whether the idea of nudity is permitted.
  - Joker_vD 5y ago
    From the person who appears to be depicted on it. Not that different from an image being a picture painted by an artist from memory/imagination, honestly. In the end, it basically boils down to having the rights on your appearance.
uCantCauseUCant 5y ago
I cant wait for this to be build into push-through augmented reality glasses. And you can sell the opposite in more prude countries.
prepend 5y ago
Looking forward to seeing how this resolves. I remember when deepfakes first was released there was a group who would deepfake coworkers, Facebook friends, etc for a really low cost (like $100) as long as the target had a few hundred public photos. This is without consent as well, but it’s also not real. It seems like the equivalent of imagining people nude. Kind of creepy if I know it’s happening but not truly a violation of my privacy.
- hardlianotion 5y ago
  It's blurry but I think in a way it is a violation of privacy in a similar way that other forms of harassment are. It's not quite the equivalent of imagining people nude as there is an artefact that can potentially cause its own pain when distributed.
  - prepend 5y ago
    That’s a good point in that the distribution is different. Maybe the better analogy for imagining nude would be to just make an image and not show it to anyone. So the issue is commercialization or distribution. In the future when we have digital consciousnesses running on Google or Amazon or whatnot, will we be prevented from imagining people nude without their consent because the mind will be replicated across multiple availability zones?
Nextgrid 5y ago
I think outlawing this tool would be counter-productive. One of the reasons nudes being released is damaging is because it's a rare enough and noteworthy event. If because of this tool everyone has nudes of them floating around then it would become a normal thing and would actually remove most of the damage around real nudes leaking by providing plausible deniability (assuming anyone ever cares at this point - if the world is drowning in nudes of everyone, the real thing will probably go unnoticed anyway). Outlawing the tool wouldn't actually stop malicious usage of it but because only criminals would use it it would make its (rarer) usage more damaging than if anyone could legally use such a tool and nudes stop being a noteworthy event.
Tycho 5y ago
Just wait till the AI visual paternity tests get here.
- tester34 5y ago
  wow, at first glance it seems like decent startup idea it removes all overhead of DNA tests
  - a_f 5y ago
    Yeah I think it's a great idea for a site/service much akin to the OP. I imagine there would be a big draw just sticking in famous parents and kids to see if there is a match.
- White_Wolf 5y ago
  I'm guessing that won't help even if it becomes a reality. These days even proper paternity tests are useless if one end up on a birth certificate without his knowledge. You're still liable even if not guilty. Edit: That being said... this naked faking app can cause of lot of problems in the workplace and home.
villgax 5y ago
Wow, next up Photoshop filters/brushes too?
- arbitrage 5y ago
  You're sort of twenty years late, photoshop is already used to create fake nudes. It's not like it's even hard. Where's the outcry over Photoshop?
- superkuh 5y ago
  I believe Virgina has already passed legislation so broad and abstract as to make photoshop and other image editing tools illegal. Like most laws of this type it's not about stopping an immoral or illegal activity. It's about control of people in power's public image. It will only be enforced if you piss off someone rich or powerful. https://law.lis.virginia.gov/vacode/title18.2/chapter8/section18.2-386.2/ https://law.lis.virginia.gov/vacode/title18.2/chapter8/secti...
isamuel 5y ago
In America, the federal child pornography law applies only to depictions of an actual child (and you have to know it, for possession offenses, though that’s another matter). But the Justice Department has long taken the position that an image of a clothed child that’s altered to then make the child look nude—-they used to call these “morphed” images—-counts. I don’t think it’s ever been definitively resolved by the Supreme Court, and I don’t know what the courts of appeals have said, but tools like DeepSukebe have made that argument way more appealing. I’d bet that this is where regulation will begin: images of children. That has always been a domain where American courts have been extremely reluctant to intervene; for example, any visual depiction of a seventeen year old engaged in sex is proscribable without resort to the ordinary inquiry into whether the work as a whole is “obscene,” etc. But under reigning American First Amendment law, it gets a lot harder to explain why a law like the one being proposed here would be acceptable. The Supreme Court has, for example, held that the distribution of animal-cruelty videos cannot be forbidden. And it’s not clear to me how one could proscribe the distribution of an imaginary visual depiction of an adult who was nude. You could call it defamatory, I suppose, but if it’s concededly fictional… I don’t know.
- hackeraccount 5y ago
  That is not my understanding of the law at all.
- dannyw 5y ago
  I mean, for decades you couldn't swear on TV (and still can't), so I feel like there's a "technically legal to possess, but illegal to send on the internet" avenue.
  - pjc50 5y ago
    Indeed. And we're talking about the UK with no right of free speech; for years there was an opposition politician who was banned from speaking on television. https://en.wikipedia.org/wiki/1988%E2%80%931994_British_broadcasting_voice_restrictions https://en.wikipedia.org/wiki/1988%E2%80%931994_British_broa... (granted that was in the context of links to a terrorist organisation, but it shows how different the landscape is to the US)
sorokod 5y ago
There is a continuum there between harmless to deeply offensive. The exact location in the continuum will at the very least depend on the person being subjected to this treatment and the cultural context. The "AI" aspect will amplify the offense because of how life-like the end result can be.
- dannyw 5y ago
  also its a cultural thing. if i make AI that turns political figures into burglars, that's legally okay. But something about nudity makes it different.
dannyw 5y ago
How effective could it be short of an international convention? Reality is, unless most countries ban it, it's gonna be on the internet.
- ben_w 5y ago
  Depends if they care about commercial or non-commercial stuff. Banning commercial use of software is fairly easy (tell the gatekeepers to disallow it, or at the very least to let the police find who did it); banning non-commercial isn’t strictly impossible, but I recon to be effective it would need governments to treat international internet connections like any other international border, and they are not prepared for the side-effects of doing that. Most people don’t grok computers, so a commercial ban would probably cover most people, yet Bush-bin-Laden photoshop I saw back in the early noughties would still get made and shared.
Joakal 5y ago
Wow, talk about technology making hijab and similar clothes obsolete which will be a massive culture shock.
knipster 5y ago
So... Streisand effect an accident that makes this capability more prevalent and disturbing? OR Streisand effect intentional to make this so common that it no longer draws attention
- laurowyn 5y ago
  First one, then the other.

38 points by unklefolk 5y ago

It will be interesting to see how this kind of thing plays out. I’m sure it’s quite distressing if a tool like this is used on your photo and then potentially shared in your friendship group. Hopefully we very quickly get to the point that nobody will ever be able to know if a photo is real or fake and it’s just not considered an issue therefore. Policing it seems like it would be extremely difficult. Maybe we police the intent? In other words you can produce the images but if you use it maliciously against a person then there is a crime.

have_faith 5y ago
I'm wondering if there are existing laws that would cover the intent part. Defamation and harassment laws etc. Maybe they just need amending. Trying to police the apps exact functionality directly seems counter productive.
- judge2020 5y ago
  I wonder as well, there have been people putting their friends or adversaries’ heads on nude model bodies ever since photoshop came out. Is that a crime and has it ever been prosecuted (maybe for defamation)?
pjc50 5y ago
> Hopefully we very quickly get to the point that nobody will ever be able to know if a photo is real or fake and it’s just not considered an issue therefore. The ongoing "fake news" crisis proves that we are, in general, bad at spotting fakes - that is the purpose of a fake - and will also vehemently disagree about what is fake. Especially when it can be used for political purposes. Expect the first world leader brought down by a deepfake in the next decade. I'm reminded of https://www.theverge.com/2017/7/12/15961354/pakistan-calibri-font-scandal-forged-documents https://www.theverge.com/2017/7/12/15961354/pakistan-calibri...

JulianMorrison 5y ago

More precisely, she wants the distribution of nudes without consent, real or faked, to be considered an offense. This isn't about the tool, but what's being done with it.

xibalba 5y ago
This is an important difference, and, for me at least, the difference between disagreeing (“ban the tool”) and agreeing (“make non-consensual nude distribution illegal”) with the proposal. It’s a shame the headline communicates the former when it seems the proposal is the latter.
- st_goliath 5y ago
  > It’s a shame the headline communicates the former when it seems the proposal is the latter. Because a headline blubbing about AI-X-ray-specs grabs attention. Making people more likely to click the link. You can see the effect in action right here: the link made it to the top of the front page in less than an hour of being submitted. Besides that tough, the article does contain the typical misunderstanding of politicians about how software works ("software developers should be forced make $SOFTWARE so that it can't do $MY_SPECIAL_EXCEPTION_CASE"), which is problematic per se, unrelated to any machine learning aspect.
buro9 5y ago
Taking this to an extreme: If an image is 100% generated and looks to be a nude of a human, from whom should consent be obtained? Especially if "this image is similar to me" is factored in, what degree of similarity makes an image a representation of a real person? We're not far from this being less about privacy and dignity, and being more about whether the idea of nudity is permitted.
- Joker_vD 5y ago
  From the person who appears to be depicted on it. Not that different from an image being a picture painted by an artist from memory/imagination, honestly. In the end, it basically boils down to having the rights on your appearance.

uCantCauseUCant 5y ago

I cant wait for this to be build into push-through augmented reality glasses. And you can sell the opposite in more prude countries.

prepend 5y ago

Looking forward to seeing how this resolves. I remember when deepfakes first was released there was a group who would deepfake coworkers, Facebook friends, etc for a really low cost (like $100) as long as the target had a few hundred public photos. This is without consent as well, but it’s also not real. It seems like the equivalent of imagining people nude. Kind of creepy if I know it’s happening but not truly a violation of my privacy.

hardlianotion 5y ago
It's blurry but I think in a way it is a violation of privacy in a similar way that other forms of harassment are. It's not quite the equivalent of imagining people nude as there is an artefact that can potentially cause its own pain when distributed.
- prepend 5y ago
  That’s a good point in that the distribution is different. Maybe the better analogy for imagining nude would be to just make an image and not show it to anyone. So the issue is commercialization or distribution. In the future when we have digital consciousnesses running on Google or Amazon or whatnot, will we be prevented from imagining people nude without their consent because the mind will be replicated across multiple availability zones?

Nextgrid 5y ago

I think outlawing this tool would be counter-productive. One of the reasons nudes being released is damaging is because it's a rare enough and noteworthy event. If because of this tool everyone has nudes of them floating around then it would become a normal thing and would actually remove most of the damage around real nudes leaking by providing plausible deniability (assuming anyone ever cares at this point - if the world is drowning in nudes of everyone, the real thing will probably go unnoticed anyway). Outlawing the tool wouldn't actually stop malicious usage of it but because only criminals would use it it would make its (rarer) usage more damaging than if anyone could legally use such a tool and nudes stop being a noteworthy event.

Tycho 5y ago

Just wait till the AI visual paternity tests get here.

tester34 5y ago
wow, at first glance it seems like decent startup idea it removes all overhead of DNA tests
- a_f 5y ago
  Yeah I think it's a great idea for a site/service much akin to the OP. I imagine there would be a big draw just sticking in famous parents and kids to see if there is a match.
White_Wolf 5y ago
I'm guessing that won't help even if it becomes a reality. These days even proper paternity tests are useless if one end up on a birth certificate without his knowledge. You're still liable even if not guilty. Edit: That being said... this naked faking app can cause of lot of problems in the workplace and home.

villgax 5y ago

Wow, next up Photoshop filters/brushes too?

arbitrage 5y ago
You're sort of twenty years late, photoshop is already used to create fake nudes. It's not like it's even hard. Where's the outcry over Photoshop?
superkuh 5y ago
I believe Virgina has already passed legislation so broad and abstract as to make photoshop and other image editing tools illegal. Like most laws of this type it's not about stopping an immoral or illegal activity. It's about control of people in power's public image. It will only be enforced if you piss off someone rich or powerful. https://law.lis.virginia.gov/vacode/title18.2/chapter8/section18.2-386.2/ https://law.lis.virginia.gov/vacode/title18.2/chapter8/secti...

isamuel 5y ago

In America, the federal child pornography law applies only to depictions of an actual child (and you have to know it, for possession offenses, though that’s another matter). But the Justice Department has long taken the position that an image of a clothed child that’s altered to then make the child look nude—-they used to call these “morphed” images—-counts. I don’t think it’s ever been definitively resolved by the Supreme Court, and I don’t know what the courts of appeals have said, but tools like DeepSukebe have made that argument way more appealing. I’d bet that this is where regulation will begin: images of children. That has always been a domain where American courts have been extremely reluctant to intervene; for example, any visual depiction of a seventeen year old engaged in sex is proscribable without resort to the ordinary inquiry into whether the work as a whole is “obscene,” etc. But under reigning American First Amendment law, it gets a lot harder to explain why a law like the one being proposed here would be acceptable. The Supreme Court has, for example, held that the distribution of animal-cruelty videos cannot be forbidden. And it’s not clear to me how one could proscribe the distribution of an imaginary visual depiction of an adult who was nude. You could call it defamatory, I suppose, but if it’s concededly fictional… I don’t know.

hackeraccount 5y ago
That is not my understanding of the law at all.
dannyw 5y ago
I mean, for decades you couldn't swear on TV (and still can't), so I feel like there's a "technically legal to possess, but illegal to send on the internet" avenue.
- pjc50 5y ago
  Indeed. And we're talking about the UK with no right of free speech; for years there was an opposition politician who was banned from speaking on television. https://en.wikipedia.org/wiki/1988%E2%80%931994_British_broadcasting_voice_restrictions https://en.wikipedia.org/wiki/1988%E2%80%931994_British_broa... (granted that was in the context of links to a terrorist organisation, but it shows how different the landscape is to the US)

sorokod 5y ago

There is a continuum there between harmless to deeply offensive. The exact location in the continuum will at the very least depend on the person being subjected to this treatment and the cultural context. The "AI" aspect will amplify the offense because of how life-like the end result can be.

dannyw 5y ago
also its a cultural thing. if i make AI that turns political figures into burglars, that's legally okay. But something about nudity makes it different.

dannyw 5y ago

How effective could it be short of an international convention? Reality is, unless most countries ban it, it's gonna be on the internet.

ben_w 5y ago
Depends if they care about commercial or non-commercial stuff. Banning commercial use of software is fairly easy (tell the gatekeepers to disallow it, or at the very least to let the police find who did it); banning non-commercial isn’t strictly impossible, but I recon to be effective it would need governments to treat international internet connections like any other international border, and they are not prepared for the side-effects of doing that. Most people don’t grok computers, so a commercial ban would probably cover most people, yet Bush-bin-Laden photoshop I saw back in the early noughties would still get made and shared.

Joakal 5y ago

Wow, talk about technology making hijab and similar clothes obsolete which will be a massive culture shock.

knipster 5y ago

So... Streisand effect an accident that makes this capability more prevalent and disturbing? OR Streisand effect intentional to make this so common that it no longer draws attention

laurowyn 5y ago
First one, then the other.