6 ms·

It's dystopian because a private company, who in the past has been committed to maintaining user privacy even when it inconveniences the government, is now regr

It's dystopian because a private company, who in the past has been committed to maintaining user privacy even when it inconveniences the government, is now regressing and going to scan the "private" contents of peoples' phones to see if they have any matches for images that have been added to a government-supplied blacklist. This blacklist is completely opaque to us as citizens; there is no way to audit or review the contents. Additionally because it's using photoDNA, if someone ever did get ahold of even a single entry in the database, they could craft malicious images that fool the perceptual hash algorithm into flagging an innocent image. (Yes, I get there will be manual review, etc). Furthermore in principle there's no reason this has to be, or will be, just CP. First it'll expand to blatant terroristic images, say ISIS beheading type videos. Then it will progress to subversive material: COVID "misinformation", "domestic terrorism" (read: anyone the feds don't like), the list goes on. This tool and its successors will be used to completely abolish the concept of device privacy and will be used to ruin the lives of anyone who sufficiently pisses off the feds. And that's just in America, god knows what they're going to do with this system in China.

shuckles 5y ago
1. Apple has only stated a commitment to protecting user privacy to the extent it is legal. They make a lot of suspicious operational changes to how they operate in China, for example. Their opposition to the FBI case was because they believed the request placed on them was illegal. 2. This article only provides evidence to say Apple will scan photos you upload into iCloud. Scanning local photos is your speculation. 3. The operation of NCMEC and the blacklist is defined by the Federal government. American (citizens) have some mechanism to oppose this design. Apple could perhaps lobby against it, but their lobbying operations are famously minimal. 4. In principle, Apple could do anything. That doesn’t really inform what they’re likely to do. Just like in principle, anyone could slip a malicious content scanning patch into the Linux kernel (and this already has a POC!).

shuckles 5y ago

1. Apple has only stated a commitment to protecting user privacy to the extent it is legal. They make a lot of suspicious operational changes to how they operate in China, for example. Their opposition to the FBI case was because they believed the request placed on them was illegal. 2. This article only provides evidence to say Apple will scan photos you upload into iCloud. Scanning local photos is your speculation. 3. The operation of NCMEC and the blacklist is defined by the Federal government. American (citizens) have some mechanism to oppose this design. Apple could perhaps lobby against it, but their lobbying operations are famously minimal. 4. In principle, Apple could do anything. That doesn’t really inform what they’re likely to do. Just like in principle, anyone could slip a malicious content scanning patch into the Linux kernel (and this already has a POC!).