6 ms·

From the article: > The automated system would proactively alert a team of human reviewers if it believes illegal imagery is detected, who would then contact l

From the article: > The automated system would proactively alert a team of human reviewers if it believes illegal imagery is detected, who would then contact law enforcement if the material can be verified. The scheme will initially roll out only in the US. This is dystopian and needs to be opposed, not applauded.

shuckles 5y ago
This is how every single cloud service has worked for a decade. You’ll need to elaborate on how, exactly, you believe it’s dystopian. The article is clearly wrong, because Apple cannot know which jurisdiction to forward a report to. That is NCMEC’s responsibility.
- cm2187 5y ago
  This is on your device, not hosted by a third party (and therefore not a third party liability). Kind of like if your microwave was checking that there wasn't any illegal activity happening in your home and phoning home if it has a suspicion. As others mentioned, privacy erosion only goes one way. This is a frontier being breached, i.e. searching your physical device without a warrant, and as usual in the name of CP (the alternative typical pretext being terrorism). It won't take very long before this gets extended to any illegal activity. Gay dating was an illegal activity until not so long ago. This sort of chinese-style monitoring is not desirable in any way, and I find it appalling that Apple seems to want to be the sponsor.
  - shuckles 5y ago
    No it’s not. Read the article: the proposed mechanism is implemented as part of photo upload to iCloud.
    - cm2187 5y ago
      The title says "scan US phones", the article is behind a paywall.
      - shuckles 5y ago
        https://archive.is/https://www.ft.com/content/14440f81-d405-452f-97e2-a81458f5411f https://archive.is/https://www.ft.com/content/14440f81-d405-...
        cm2187 5y ago
        Thanks. The article confirms the title, this is not about content uploaded to the cloud: Apple’s system is less invasive in that the screening is done on the phone, and “only if there is a match is notification sent back to those searching”, said Alan Woodward, a computer security professor at the University of Surrey. “This decentralised approach is about the best approach you could adopt if you do go down this route.”
        __blockcipher__ 5y ago
        https://twitter.com/matthew_d_green/status/1423109002280513540 https://twitter.com/matthew_d_green/status/14231090022805135... and https://twitter.com/matthew_d_green/status/1423109458293628931 https://twitter.com/matthew_d_green/status/14231094582936289... kind of sums it up. Clearly there is an intent to progress to more than just what's been shared with iCloud. Otherwise there'd be no need for software on the devices themselves at all.
        shuckles 5y ago
        That is not the only explanation. Involving the device could be considered better because it doesn’t require Apple to build infrastructure to scan all your photos in the cloud, which could be more easily repurposed without user consent.
        pseudalopex 5y ago
        They scan photos in the cloud now. And do you think Apple will announce and let users opt out of client side scanning changes?
- res0nat0r 5y ago
  AWS doesn't do any automatic fingerprinting of files being uploaded to S3 and never has. The only time I've heard of anything like this was briefly when Microsoft was doing it, and I don't know if that was shutdown long ago due to outrage or not.
  - shuckles 5y ago
    AWS doesn’t have to because the entity which writes those files is obligated to do this scanning if there is reason to believe the files contain CSAM.
    - res0nat0r 5y ago
      I guess I should clear up: Amazon isn't doing any type of automatic file-scanning for anything being uploaded to S3, or feeding md5/sha1/sha256sums to some 3rd party service automatically for everything that touches S3. If they get a subpoena and then maybe send some kind of file list/checksums they've been required to hand over to some 3rd party / government? That's possible, but an entirely different situation.
      - shuckles 5y ago
        Sure, but whoever owns the account which wrote those files in is, in the United States, responsible for doing some version of what Apple has announced they will do with iCloud Photos today.
- __blockcipher__ 5y ago
  It's dystopian because a private company, who in the past has been committed to maintaining user privacy even when it inconveniences the government, is now regressing and going to scan the "private" contents of peoples' phones to see if they have any matches for images that have been added to a government-supplied blacklist. This blacklist is completely opaque to us as citizens; there is no way to audit or review the contents. Additionally because it's using photoDNA, if someone ever did get ahold of even a single entry in the database, they could craft malicious images that fool the perceptual hash algorithm into flagging an innocent image. (Yes, I get there will be manual review, etc). Furthermore in principle there's no reason this has to be, or will be, just CP. First it'll expand to blatant terroristic images, say ISIS beheading type videos. Then it will progress to subversive material: COVID "misinformation", "domestic terrorism" (read: anyone the feds don't like), the list goes on. This tool and its successors will be used to completely abolish the concept of device privacy and will be used to ruin the lives of anyone who sufficiently pisses off the feds. And that's just in America, god knows what they're going to do with this system in China.
  - shuckles 5y ago
    1. Apple has only stated a commitment to protecting user privacy to the extent it is legal. They make a lot of suspicious operational changes to how they operate in China, for example. Their opposition to the FBI case was because they believed the request placed on them was illegal. 2. This article only provides evidence to say Apple will scan photos you upload into iCloud. Scanning local photos is your speculation. 3. The operation of NCMEC and the blacklist is defined by the Federal government. American (citizens) have some mechanism to oppose this design. Apple could perhaps lobby against it, but their lobbying operations are famously minimal. 4. In principle, Apple could do anything. That doesn’t really inform what they’re likely to do. Just like in principle, anyone could slip a malicious content scanning patch into the Linux kernel (and this already has a POC!).
marderfarker2 5y ago
> will initially roll out only in the US. Ok so they’re planning to eventually roll out this spyware worldwide. I do not know how Americans can think China is bad in this while allowing this to happen.
- trasz 5y ago
  Never ceases to amaze how people believe that China is somehow more oppressive than the the country with world highest incarceration rate.
  - skinkestek 5y ago
    You forget that death sentences are carried out much more swiftly in China. Also, while there's a lot to be said about court systems, public defenders etc in US I wouldn't compare the two.
  - elzbardico 5y ago
    China have actual slaves. You can't even know the real rate of incarceration in China, and trying to figure it out is almost a sure way to get arrested.
  - ggreer 5y ago
    The US has a high incarceration rate because it has a high crime rate. If EU countries had US levels of crime, they'd have US levels of incarceration. Also China doesn't count Uyghurs as incarcerated despite the fact that the PRC is committing genocide against them.[1] The PRC imprisons almost 2 million of Uyghurs a year. Of those, around a quarter million die. Oh and there's the forced sterilizations, destruction of mosques, brainwashing, banning of muslim-sounding names[2], forced labor, organ harvesting, and mandatory quartering of government agents.[3] 1. https://en.wikipedia.org/wiki/Uyghur_genocide https://en.wikipedia.org/wiki/Uyghur_genocide 2. https://en.wikipedia.org/wiki/Naming_laws_in_China https://en.wikipedia.org/wiki/Naming_laws_in_China 3. https://en.wikipedia.org/wiki/Civil_Servant-Family_Pair_Up https://en.wikipedia.org/wiki/Civil_Servant-Family_Pair_Up
    - 5y ago

shuckles 5y ago

This is how every single cloud service has worked for a decade. You’ll need to elaborate on how, exactly, you believe it’s dystopian. The article is clearly wrong, because Apple cannot know which jurisdiction to forward a report to. That is NCMEC’s responsibility.

cm2187 5y ago
This is on your device, not hosted by a third party (and therefore not a third party liability). Kind of like if your microwave was checking that there wasn't any illegal activity happening in your home and phoning home if it has a suspicion. As others mentioned, privacy erosion only goes one way. This is a frontier being breached, i.e. searching your physical device without a warrant, and as usual in the name of CP (the alternative typical pretext being terrorism). It won't take very long before this gets extended to any illegal activity. Gay dating was an illegal activity until not so long ago. This sort of chinese-style monitoring is not desirable in any way, and I find it appalling that Apple seems to want to be the sponsor.
- shuckles 5y ago
  No it’s not. Read the article: the proposed mechanism is implemented as part of photo upload to iCloud.
  - cm2187 5y ago
    The title says "scan US phones", the article is behind a paywall.
    - shuckles 5y ago
      https://archive.is/https://www.ft.com/content/14440f81-d405-452f-97e2-a81458f5411f https://archive.is/https://www.ft.com/content/14440f81-d405-...
      - cm2187 5y ago
        Thanks. The article confirms the title, this is not about content uploaded to the cloud: Apple’s system is less invasive in that the screening is done on the phone, and “only if there is a match is notification sent back to those searching”, said Alan Woodward, a computer security professor at the University of Surrey. “This decentralised approach is about the best approach you could adopt if you do go down this route.”
        __blockcipher__ 5y ago
        https://twitter.com/matthew_d_green/status/1423109002280513540 https://twitter.com/matthew_d_green/status/14231090022805135... and https://twitter.com/matthew_d_green/status/1423109458293628931 https://twitter.com/matthew_d_green/status/14231094582936289... kind of sums it up. Clearly there is an intent to progress to more than just what's been shared with iCloud. Otherwise there'd be no need for software on the devices themselves at all.
        shuckles 5y ago
        That is not the only explanation. Involving the device could be considered better because it doesn’t require Apple to build infrastructure to scan all your photos in the cloud, which could be more easily repurposed without user consent.
        pseudalopex 5y ago
        They scan photos in the cloud now. And do you think Apple will announce and let users opt out of client side scanning changes?
res0nat0r 5y ago
AWS doesn't do any automatic fingerprinting of files being uploaded to S3 and never has. The only time I've heard of anything like this was briefly when Microsoft was doing it, and I don't know if that was shutdown long ago due to outrage or not.
- shuckles 5y ago
  AWS doesn’t have to because the entity which writes those files is obligated to do this scanning if there is reason to believe the files contain CSAM.
  - res0nat0r 5y ago
    I guess I should clear up: Amazon isn't doing any type of automatic file-scanning for anything being uploaded to S3, or feeding md5/sha1/sha256sums to some 3rd party service automatically for everything that touches S3. If they get a subpoena and then maybe send some kind of file list/checksums they've been required to hand over to some 3rd party / government? That's possible, but an entirely different situation.
    - shuckles 5y ago
      Sure, but whoever owns the account which wrote those files in is, in the United States, responsible for doing some version of what Apple has announced they will do with iCloud Photos today.
__blockcipher__ 5y ago
It's dystopian because a private company, who in the past has been committed to maintaining user privacy even when it inconveniences the government, is now regressing and going to scan the "private" contents of peoples' phones to see if they have any matches for images that have been added to a government-supplied blacklist. This blacklist is completely opaque to us as citizens; there is no way to audit or review the contents. Additionally because it's using photoDNA, if someone ever did get ahold of even a single entry in the database, they could craft malicious images that fool the perceptual hash algorithm into flagging an innocent image. (Yes, I get there will be manual review, etc). Furthermore in principle there's no reason this has to be, or will be, just CP. First it'll expand to blatant terroristic images, say ISIS beheading type videos. Then it will progress to subversive material: COVID "misinformation", "domestic terrorism" (read: anyone the feds don't like), the list goes on. This tool and its successors will be used to completely abolish the concept of device privacy and will be used to ruin the lives of anyone who sufficiently pisses off the feds. And that's just in America, god knows what they're going to do with this system in China.
- shuckles 5y ago
  1. Apple has only stated a commitment to protecting user privacy to the extent it is legal. They make a lot of suspicious operational changes to how they operate in China, for example. Their opposition to the FBI case was because they believed the request placed on them was illegal. 2. This article only provides evidence to say Apple will scan photos you upload into iCloud. Scanning local photos is your speculation. 3. The operation of NCMEC and the blacklist is defined by the Federal government. American (citizens) have some mechanism to oppose this design. Apple could perhaps lobby against it, but their lobbying operations are famously minimal. 4. In principle, Apple could do anything. That doesn’t really inform what they’re likely to do. Just like in principle, anyone could slip a malicious content scanning patch into the Linux kernel (and this already has a POC!).

marderfarker2 5y ago

> will initially roll out only in the US. Ok so they’re planning to eventually roll out this spyware worldwide. I do not know how Americans can think China is bad in this while allowing this to happen.

trasz 5y ago
Never ceases to amaze how people believe that China is somehow more oppressive than the the country with world highest incarceration rate.
- skinkestek 5y ago
  You forget that death sentences are carried out much more swiftly in China. Also, while there's a lot to be said about court systems, public defenders etc in US I wouldn't compare the two.
- elzbardico 5y ago
  China have actual slaves. You can't even know the real rate of incarceration in China, and trying to figure it out is almost a sure way to get arrested.
- ggreer 5y ago
  The US has a high incarceration rate because it has a high crime rate. If EU countries had US levels of crime, they'd have US levels of incarceration. Also China doesn't count Uyghurs as incarcerated despite the fact that the PRC is committing genocide against them.[1] The PRC imprisons almost 2 million of Uyghurs a year. Of those, around a quarter million die. Oh and there's the forced sterilizations, destruction of mosques, brainwashing, banning of muslim-sounding names[2], forced labor, organ harvesting, and mandatory quartering of government agents.[3] 1. https://en.wikipedia.org/wiki/Uyghur_genocide https://en.wikipedia.org/wiki/Uyghur_genocide 2. https://en.wikipedia.org/wiki/Naming_laws_in_China https://en.wikipedia.org/wiki/Naming_laws_in_China 3. https://en.wikipedia.org/wiki/Civil_Servant-Family_Pair_Up https://en.wikipedia.org/wiki/Civil_Servant-Family_Pair_Up
  - 5y ago