10 ms·

This is a harmful distraction from the massive issues with Apple's proposal. If you wanted to frame someone for possession of CSAM, similar stunts can be pulled

This is a harmful distraction from the massive issues with Apple's proposal. If you wanted to frame someone for possession of CSAM, similar stunts can be pulled with Google, Facebook, Instagram, and Microsoft today. Yes, the scope here is broader and some people don't use any of those, but ....It's silly, and it makes the tech community look like a fringe minority of screeching conspiracy theorists. And this is a problem because Apple's proposal is really really awful. Apple is normalizing scanning your private phone for files and reporting them. They built the technical capability to do it for any photo, and they will be under enormous pressure to expand it both in the US and abroad. And the fact that they did it will be used to pressure other companies into doing the same and to legitimize laws that require scanning for any content the government can justify. Apple built a surveillance mechanism that is incredibly powerful. One no government could ever force a company to design and build. But once it's built, the only thing stoping it from being abused is Apple's pinky promise they won't let it happen. If you believe that legal norms, big tech companies and some quasi governmental nonprofit like NCMEC will stop such an abuse if it happens .... where have you been living the past few years? Because it sure isn't the US, the UK, Turkey, or China.

skybrian 5y ago
Is this actually different from other cloud photo apps? If you use Google Photos then your photos will likely be scanned by Google. If you use Apple’s photo app then their app will do the scanning. There seems to be a vague idea floating around that this is built into the OS or the device just because the scanning happens on the device, but it’s not clear that’s the case. Apple doesn’t make the distinction between OS and app clear either.
- shadowgovt 5y ago
  Ironically, the difference is that Apple is doing it at the client layer so that they can't do it at the server layer; the user's iCloud [edit: photos, not all of iCloud] is encrypted at rest against Apple accessing it. This approach makes mass-sweeping of all server-side stored data harder to accomplish (whereas in, say, Google Photos, Google can break-glass server side to get into someone's private data, so they could hypothetically do a mass-scan if the government demanded it).
  - nightski 5y ago
    Right but it's easier to just not use Google Photos. It's harder to opt out of your phone. I realize they "said" that device scanning will only be used if iCloud is enabled (right?). But ToS changes constantly and who knows what the future holds.
    - nicce 5y ago
      It is only applied for photos which are going to iCloud. If they change that, then we should be really worried. Current method is only pure improvement if leave all speculation out of it.
      - ribosometronome 5y ago
        I am generally in agreement with you (and have made similar arguments, if you look at my post history), but the "expand in unspecified ways" is a bit ominous. Committing to only scanning photos that are being synced to the cloud (effectively, keeping parity with what everyone does, just doing it on device at the time of upload instead of in the cloud) would be really welcome here.
        hypothesis 5y ago
        Will such commitment substantially change anything? First line of their privacy policy is: “Apple is committed to your privacy.” [1] https://www.apple.com/legal/privacy/ https://www.apple.com/legal/privacy/
    - skybrian 5y ago
      It's not at all hard to avoid using an app on your phone. I have an iPad and I use Google Photos. I've never used Apple's photo app. This is what I meant by mixing up (and blurring the lines between) app-level and OS-level capabilities. It might not actually be mixed up technically, but it seems to be the user perception.
  - Klonoar 5y ago
    iCloud as it currently stands is not encrypted to where Apple can’t access it.
- ianmiers 5y ago
  Factually, not yet. That will change and I will explain how in a moment. But first there's a major difference between doing it on device vs in the cloud. It changes how we think about privacy and builds a capability to scan phones (not particularly limited to iCloud) into the device. That's a capability no Western tech company could ever be forced to build for more illicit usages, but now it exists. Second, Apple almost assuredly will encrypt iCloud after this. So now we have the precedent of scanning encrypted messages. And that will then feed legislation that congress has been attempting to pass for years to kill any right to meaningful end to end encryption for messaging. https://blog.cryptographyengineering.com/2020/03/06/earn-it-is-an-attack-on-encryption/ https://blog.cryptographyengineering.com/2020/03/06/earn-it-...
- ummonk 5y ago
  The danger is that this is further down the slippery slope.
nicce 5y ago
> Apple is normalizing scanning your private phone for files and reporting them. ”Antivirus cries in corner as forgotten...” I know, iOS has no build-in AV (like MacOS) but still, it is a bit laughable that many existing tools provides this same power, and only now it is a concern. On a black box system. I am resilent, and I will join into mass of pitchforks and torches only when there is actual evidence of them expanding their promises or using these features for something else they are meant. They knew the risks when bringing this feature and know the cost when it is proved to be misused.
- ianmiers 5y ago
  yes, you can turn AV into the same thing. But no one has been advocating for that or passing laws that would make AV both mandatory and have to report you to the police. This has been going on for CSAM scanning for a while now. The latest version was called the EARN IT act [0] The argument is not this is a slippery slope where you might miss step. IF that was the case, yes AV would be analogous. Instead, its that people are actively trying to push you into the spikes at the bottom of the pit, don't build things at the edge of the pit where the handrail is a pinky promise not to let others push you. [0] https://blog.cryptographyengineering.com/2020/03/06/earn-it-is-an-attack-on-encryption/ https://blog.cryptographyengineering.com/2020/03/06/earn-it-...
  - nicce 5y ago
    > don't build things at the edge of the pit where the handrail is a pinky promise not to let others push you. People are afraid, that the use of these tools are expanded in secret (hidden) for more than they should. From that point of view, legislation motives and discussion does not matter, because capability is valid for many tools at any moment, hence same speculation has applied before. However, if you want to apply surveillance publicly, then we indeed need legal base for pushing specified tools as mandatory. To expand it for more than CSAM, it will be quite slow process, and implementing something like that publicly before legal base is path for descruction for any company, because people can change for other company. Is Apple now making that legal process faster? While it feels like Apple is now closer to the edge of the pit, from techical perpective there is no difference yet. Tools have existed and system is closed source. Question is still the same; ”would you spy for us”? I don’t think that answer has changed from Apple because they changed the location of the image scan. So, the question is, will legislation change towards more surveillance. Whatever the result is, I think it would have happened whether Apple added this feature or not, as it is not morale excuse. People find the way. In China, it is simply illegal to not install some app by muslims.

skybrian 5y ago

Is this actually different from other cloud photo apps? If you use Google Photos then your photos will likely be scanned by Google. If you use Apple’s photo app then their app will do the scanning. There seems to be a vague idea floating around that this is built into the OS or the device just because the scanning happens on the device, but it’s not clear that’s the case. Apple doesn’t make the distinction between OS and app clear either.

shadowgovt 5y ago
Ironically, the difference is that Apple is doing it at the client layer so that they can't do it at the server layer; the user's iCloud [edit: photos, not all of iCloud] is encrypted at rest against Apple accessing it. This approach makes mass-sweeping of all server-side stored data harder to accomplish (whereas in, say, Google Photos, Google can break-glass server side to get into someone's private data, so they could hypothetically do a mass-scan if the government demanded it).
- nightski 5y ago
  Right but it's easier to just not use Google Photos. It's harder to opt out of your phone. I realize they "said" that device scanning will only be used if iCloud is enabled (right?). But ToS changes constantly and who knows what the future holds.
  - nicce 5y ago
    It is only applied for photos which are going to iCloud. If they change that, then we should be really worried. Current method is only pure improvement if leave all speculation out of it.
    - ribosometronome 5y ago
      I am generally in agreement with you (and have made similar arguments, if you look at my post history), but the "expand in unspecified ways" is a bit ominous. Committing to only scanning photos that are being synced to the cloud (effectively, keeping parity with what everyone does, just doing it on device at the time of upload instead of in the cloud) would be really welcome here.
      - hypothesis 5y ago
        Will such commitment substantially change anything? First line of their privacy policy is: “Apple is committed to your privacy.” [1] https://www.apple.com/legal/privacy/ https://www.apple.com/legal/privacy/
  - skybrian 5y ago
    It's not at all hard to avoid using an app on your phone. I have an iPad and I use Google Photos. I've never used Apple's photo app. This is what I meant by mixing up (and blurring the lines between) app-level and OS-level capabilities. It might not actually be mixed up technically, but it seems to be the user perception.
- Klonoar 5y ago
  iCloud as it currently stands is not encrypted to where Apple can’t access it.
ianmiers 5y ago
Factually, not yet. That will change and I will explain how in a moment. But first there's a major difference between doing it on device vs in the cloud. It changes how we think about privacy and builds a capability to scan phones (not particularly limited to iCloud) into the device. That's a capability no Western tech company could ever be forced to build for more illicit usages, but now it exists. Second, Apple almost assuredly will encrypt iCloud after this. So now we have the precedent of scanning encrypted messages. And that will then feed legislation that congress has been attempting to pass for years to kill any right to meaningful end to end encryption for messaging. https://blog.cryptographyengineering.com/2020/03/06/earn-it-is-an-attack-on-encryption/ https://blog.cryptographyengineering.com/2020/03/06/earn-it-...
ummonk 5y ago
The danger is that this is further down the slippery slope.

nicce 5y ago

> Apple is normalizing scanning your private phone for files and reporting them. ”Antivirus cries in corner as forgotten...” I know, iOS has no build-in AV (like MacOS) but still, it is a bit laughable that many existing tools provides this same power, and only now it is a concern. On a black box system. I am resilent, and I will join into mass of pitchforks and torches only when there is actual evidence of them expanding their promises or using these features for something else they are meant. They knew the risks when bringing this feature and know the cost when it is proved to be misused.

ianmiers 5y ago
yes, you can turn AV into the same thing. But no one has been advocating for that or passing laws that would make AV both mandatory and have to report you to the police. This has been going on for CSAM scanning for a while now. The latest version was called the EARN IT act [0] The argument is not this is a slippery slope where you might miss step. IF that was the case, yes AV would be analogous. Instead, its that people are actively trying to push you into the spikes at the bottom of the pit, don't build things at the edge of the pit where the handrail is a pinky promise not to let others push you. [0] https://blog.cryptographyengineering.com/2020/03/06/earn-it-is-an-attack-on-encryption/ https://blog.cryptographyengineering.com/2020/03/06/earn-it-...
- nicce 5y ago
  > don't build things at the edge of the pit where the handrail is a pinky promise not to let others push you. People are afraid, that the use of these tools are expanded in secret (hidden) for more than they should. From that point of view, legislation motives and discussion does not matter, because capability is valid for many tools at any moment, hence same speculation has applied before. However, if you want to apply surveillance publicly, then we indeed need legal base for pushing specified tools as mandatory. To expand it for more than CSAM, it will be quite slow process, and implementing something like that publicly before legal base is path for descruction for any company, because people can change for other company. Is Apple now making that legal process faster? While it feels like Apple is now closer to the edge of the pit, from techical perpective there is no difference yet. Tools have existed and system is closed source. Question is still the same; ”would you spy for us”? I don’t think that answer has changed from Apple because they changed the location of the image scan. So, the question is, will legislation change towards more surveillance. Whatever the result is, I think it would have happened whether Apple added this feature or not, as it is not morale excuse. People find the way. In China, it is simply illegal to not install some app by muslims.