8 ms·

I’m not sure I understand Apple’s logic here. Are iCloud Photos in their data centers not scanned? Isn’t everything by default for iCloud users sent there autom

by _m8fo 3y ago

I’m not sure I understand Apple’s logic here. Are iCloud Photos in their data centers not scanned? Isn’t everything by default for iCloud users sent there automatically to begin with? Doesn’t the same logic around slippery slope also apply to cloud scans? This is not to say they should scan locally, but my understanding of CSAM was that it would only be scanned on its way to the cloud anyways, so users who didn’t use iCloud would’ve never been scanned to begin with. Their new proposed set of tools seems like a good enough compromise from the original proposal in any case.

theshrike79 3y ago
In my opinion their goal was to get stuff to a state where they could encrypt everything on iCloud so that even they can't access it. To counter the "think of the children" -argument governments use to justify surveillance, Apple tried scanning stuff on-device but the internet got a collective hissy-fit of intentionally misunderstanding the feature and it was quickly scrapped.
- Shank 3y ago
  > In my opinion their goal was to get stuff to a state where they could encrypt everything on iCloud so that even they can't access it. They basically did. If you turn on Advanced Data Protection, you get all of the encryption benefits, sans scanning. The interesting thing is that if you turn on ADP though, binary file hashes are unencrypted on iCloud, which would theoretically allow someone to ask for those hashes in a legal request. But it's obviously not as useful for CSAM detection, as, say, PhotoDNA hashes. See: https://support.apple.com/en-us/HT202303 https://support.apple.com/en-us/HT202303
  - chatmasta 3y ago
    Nice! TIL this exists. For anyone else wondering, to enable it just go to iOS Settings -> iCloud and you'll see "Advanced Data Protection." Toggle it to enabled to create a recovery key, which you'll then be prompted to input correctly after saving it somewhere safe, and then return to the iCloud Settings page, toggle it one more time and enter your recovery key again to confirm.
- bryan_w 3y ago
  > so that even they can't access it. > scanning stuff on-device What do you think they were going to do once the scanning turned up a hit? Access the photos? Well that negates the first statement.
  - theshrike79 3y ago
    Who is this "they" who will access the photos on-device?
  - judge2020 3y ago
    > What do you think they were going to do once the scanning turned up a hit? Access the photos? Well that negates the first statement. In the whitepaper, the cryptography required that Apple have multiple different photodna (or whatever the name was for the on-device one) matches before they could unwrap the user's message containing these suspected CSAM photos and to then send them to NCMEC.
    - barsonme 3y ago
      Also, IIRC, it wasn’t the raw photos. It was small thumbnails of them.
      - theshrike79 3y ago
        "reduced-quality copy" was the wording in the whitepaper IIRC. So the resolution most likely would've been the same, but the detail blurred so that the poor human agent wouldn't have to see actual CSAM, just enough to make a call whether it is or isn't a likely match.
        FabHK 3y ago
        It was called a “visual derivative”. https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
  - MBCook 3y ago
- kobalsky 3y ago
  > but the internet got a collective hissy-fit of intentionally misunderstanding the feature how was it misunderstood? your device would scan your photos and notify apple or whoever if something evil was found. wasn't that what they were trying to do?
  - spacedcowboy 3y ago
    Your device would scan your photo at the point of you uploading it to the cloud and then it could encrypt it before sending it to the cloud. That meant that Apple's cloud servers didn't need to be able to scan it to comply with US Govt "recommendations" for cloud providers. Whereas right now all the other cloud providers just send the photo as-is and scan it on the cloud servers. With Apple's approach, the cloud servers don't get to look at every single one of your photos like cloud vendors do today, scanning happens within the privacy of your own phone, and only known-kiddy-porn signatures are flagged. Apple came up with a way to make things way more private, but the concept of your own device working "against" you if you happen to be a pedophile was too much of a leap.
  - theshrike79 3y ago
    Your device would've scanned your photos ONLY if you would've uploaded them to Apple's cloud service anyway. And it wouldn't have notified Apple of "something evil", just specifically known and human-verified actual real child abuse photos. And not even that, it would have needed multiple matches of those very real and verified abuse photos before it flagged them so that a real human could see a "visual derivative" of the photos. Only if those multiple matches of derivatives were deemed as actual, very real, child pornography the authorities would've been called. But nope. Now they just scan ALL your data in the cloud when the authorities demand it. And that's somehow better according to the internet in a way I still can't understand.
no_time 3y ago
> I’m not sure I understand apples logic here. Are iCloud Photos in their data centers not scanned? Isn’t everything by default for iCloud users sent there automatically to begin with? Doesn’t the same logic around slippery slope also apply to cloud scans? I don’t see the problem with this status quo. There is a clear demarcation between my device and their server. Each serving the interests of their owner. If I have a problem with their policy, I can choose not to entrust my data to them. And luckily, the data storage space has heaps of competitive options.
- chatmasta 3y ago
  > the data storage space has heaps of competitive options The generic space does, yes. But if you want native integration with iOS, your only choice is iCloud. It would certainly be nice if this was an open protocol where you could choose your own storage backend. But I think the chances of that ever happening are pretty much zero.
- judge2020 3y ago
  This status quo is that a lot of countries want to use the CSAM argument to push privacy-invasive technology (cough UK) like e.g. forcing companies to allow the government to break E2EE to catch CSAM distributors. Apple made this feature while planning to move iCloud Photos to E2EE so that they could argue "look, we still catch x CSAM distributors with n < 0.x% false positive rate, even with E2EE photos. therefore you don't need to pass these laws that break E2EE."
  - no_time 3y ago
    I know "give them an inch, they take a mile" is a reductive comparison but I really can't see this way of thinking going any other way in the long term.
    - salawat 3y ago
      It isn't reductive. At the end of the day, that's exactly what it comes down to.
- 3y ago
Moldoteck 3y ago
so users who didn’t use iCloud would’ve never been scanned to begin with. - so why not implement csam for icloud only without local scanning?
- Gigachad 3y ago
  Because the idea is that the iCloud data would be encrypted so their servers couldn’t scan it. With the plan being they would do on device scanning of photos that were marked as being stored on iCloud. It’s objectively better than what google does but I’m glad we somehow ended up with no scanning at all.
  - Moldoteck 3y ago
    that sounds strange, I mean i'm not sure what's the big difference. If data is scanned on icloud, this means it's not encrypted, got it, if scanned on devices, data is fully encrypted on icloud, but apple has access by scanning it on devices and can send unencrypted matches, so it behaves as an unencrypted system, that can be altered at apple's will, just like icloud... but still, why scanning locally only if icloud is enabled? why not scan regardless? Since policy is meant to 'catch bad ppl', why limit to icloud option and not scan all the time
    - nemothekid 3y ago
      Apple doesn’t want to scan period. However if Apple does E2ee icloud, the biggest political issue will be that of CSAM. So in order to reserve CSAM, they came up with this scheme. Apple doesn’t want to expand their power which is why they don’t scan locally. They weren’t doing it before and they don’t want to offer it now.
    - kalleboo 3y ago
      > Since policy is meant to 'catch bad ppl', why limit to icloud option and not scan all the time The policy is meant to ensure Apple's servers are not storing and distributing CSAM, not that Apple wants to become a police investigative force.
turquoisevar 3y ago
> Are iCloud Photos in their data centers not scanned? No outright statement confirming or denying this has ever made to my knowledge, but the implication, based both on Apple's statements and the statement of stakeholders, is that this isn't currently the case. This might come as a surprise to some, because many companies scan for CSAM, but that's done voluntarily because the government can't force companies to scan for CSAM. This is because based on case law, companies forced to scan for CSAM would be considered deputized and thus subsequently it would be a breach of the 4th amendments safeguards against "unreasonable search and seizure". The best the government can do is to force companies to report "apparent violations" of CSAM laws, this seems like a distinction without a difference, but the difference is between required to actively search for it (and thus becoming deputized) v. reporting when you come across it. Even then, the reporting requirement is constructed in such a way as to avoid any possible 4th amendment issues. Companies aren't required to report it to the DOJ, but rather to the NCMEC. The NCMEC is a semi-government organization, autonomous from the DOJ, albeit almost wholly funded by the DOJ, and they are the ones that subsequently report CSAM violations to the DOJ. The NCMEC is also the organization that maintains the CSAM database and provides the hashes that companies, who voluntarily scan for CSAM, use. This construction has proven to be pretty solid against 4th amendment concerns, as courts have historically found that this separation between companies and the DOJ and the fact that only confirmed CSAM making its way to the DOJ after review by the NCMEC, creates enough of a distance between the DOJ and the act of searching through a person's data, that there aren't any 4th amendment concerns. The Congressional Research Service did a write up on this last year for the ones that are interested in it[0]. Circling back to Apple, as it stands there's nothing indicating that they already scan for CSAM server-side and most comments both by Apple and child safety organizations seem to imply that this in fact is currently not happening. Apple's main concerns however, as stated in the letter by Apple, echo the same concerns by security experts back when this was being discussed. Namely that it creates a target for malicious actors, that it is technically not feasible to create a system that can never be reconfigured to scan for non-CSAM material and that governments could pressure/regulate it to reconfigure it for other materials as well (and place a gag order on them, prohibiting them to inform users of this). At the time, some of these arguments were brushed off as slippery slope FUD, and then the UK started considering something that would defy the limits of even the most cynical security researcher's nightmare, namely a de facto ban on security updates if it just so happens that the UK's intelligence services and law enforcement services are currently exploiting the security flaw that the update aims to patch. Which is what Apple references in their response. 0: https://crsreports.congress.gov/product/pdf/LSB/LSB10713 https://crsreports.congress.gov/product/pdf/LSB/LSB10713
matwood 3y ago
You are correct, the original method would only have scanned items destined to iCloud and only transmitted some hash of matching hashes. And yes, similar slippery arguments exist with any providers that store images unencrypted. They are all scanned today, and we have no idea what they are matched against. I speculated (and now we know) when this new scanning announced, that it was in preparation for full E2EE. Apple came up with a privacy preserving method of trying to keep CSAM off their servers while also giving E2EE. The larger community arguments swayed Apple from going forward with their new detection method, but did not stop them from moving forward with E2EE. At the end of the day they put the responsibility back on governments to pass laws around encryption - where they should be, though we may not like the outcome.
- gonehome 3y ago
  There are also ways to detect matches even with e2ee iirc and I suspect they found doing that instead easier than dealing with the previous approach. At the time I also thought it was obvious it was in preparation for e2ee (despite loud people on HN who disagreed). I do wonder if they had intended to have it be default on though, maybe not since probably better for most users to have a recovery option.
  - judge2020 3y ago
    > There are also ways to detect matches even with e2ee By definition, encryption (with unique user keys) means you can't infer nor check what the content of the message is. Not without client cooperation, which is what this feature would have been.
    - gonehome 3y ago
      This is what I was recalling, this method gives you a clever way to do it using the file itself as the key: > “Convergent encryption solves this problem in a very clever way: “The way to make sure that every unique user with the same file ends up with an encrypted version of that file that is also identical is to ensure they use the same key. However, you can’t share keys between users, because that defeats the entire point; you need a common reference point between users that is unknown to anyone but those users. “The answer is to use the file itself: the system creates a hash of the file’s content, and that hash (a long string of characters derived from a known algorithm) is the key that is used to encrypt said file. “If every iCloud user uses this technique — and given that Apple implements the system, they do — then every iCloud user with the same file will produce the same encrypted file, given that they are using the same key (which is derived from the file itself); that means that Apple only needs to store one version of that file even as it makes said file available to everyone who “uploaded” it (in truth, because iCloud integration goes down to the device, the file is probably never actually uploaded at all — Apple just includes a reference to the file that already exists on its servers, thus saving a huge amount of money on both storage costs and bandwidth). “There is one huge flaw in convergent encryption, however, called “confirmation of file”: if you know the original file you by definition can identify the encrypted version of that file (because the key is derived from the file itself). When it comes to CSAM, though, this flaw is a feature: because Apple uses convergent encryption for its end-to-end encryption it can by definition do server-side scanning of files and exploit the “confirmation of file” flaw to confirm if CSAM exists, and, by extension, who “uploaded” it. Apple’s extremely low rates of CSAM reporting suggest that the company is not currently pursuing this approach, but it is the most obvious way to scan for CSAM given it has abandoned its on-device plan.” https://stratechery.com/2022/apple-icloud-encryption-csam-scanning-and-convergent-encryption/ https://stratechery.com/2022/apple-icloud-encryption-csam-sc...

by _m8fo 3y ago

theshrike79 3y ago

In my opinion their goal was to get stuff to a state where they could encrypt everything on iCloud so that even they can't access it. To counter the "think of the children" -argument governments use to justify surveillance, Apple tried scanning stuff on-device but the internet got a collective hissy-fit of intentionally misunderstanding the feature and it was quickly scrapped.

Shank 3y ago
> In my opinion their goal was to get stuff to a state where they could encrypt everything on iCloud so that even they can't access it. They basically did. If you turn on Advanced Data Protection, you get all of the encryption benefits, sans scanning. The interesting thing is that if you turn on ADP though, binary file hashes are unencrypted on iCloud, which would theoretically allow someone to ask for those hashes in a legal request. But it's obviously not as useful for CSAM detection, as, say, PhotoDNA hashes. See: https://support.apple.com/en-us/HT202303 https://support.apple.com/en-us/HT202303
- chatmasta 3y ago
  Nice! TIL this exists. For anyone else wondering, to enable it just go to iOS Settings -> iCloud and you'll see "Advanced Data Protection." Toggle it to enabled to create a recovery key, which you'll then be prompted to input correctly after saving it somewhere safe, and then return to the iCloud Settings page, toggle it one more time and enter your recovery key again to confirm.
bryan_w 3y ago
> so that even they can't access it. > scanning stuff on-device What do you think they were going to do once the scanning turned up a hit? Access the photos? Well that negates the first statement.
- theshrike79 3y ago
  Who is this "they" who will access the photos on-device?
- judge2020 3y ago
  > What do you think they were going to do once the scanning turned up a hit? Access the photos? Well that negates the first statement. In the whitepaper, the cryptography required that Apple have multiple different photodna (or whatever the name was for the on-device one) matches before they could unwrap the user's message containing these suspected CSAM photos and to then send them to NCMEC.
  - barsonme 3y ago
    Also, IIRC, it wasn’t the raw photos. It was small thumbnails of them.
    - theshrike79 3y ago
      "reduced-quality copy" was the wording in the whitepaper IIRC. So the resolution most likely would've been the same, but the detail blurred so that the poor human agent wouldn't have to see actual CSAM, just enough to make a call whether it is or isn't a likely match.
      - FabHK 3y ago
        It was called a “visual derivative”. https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
- MBCook 3y ago
kobalsky 3y ago
> but the internet got a collective hissy-fit of intentionally misunderstanding the feature how was it misunderstood? your device would scan your photos and notify apple or whoever if something evil was found. wasn't that what they were trying to do?
- spacedcowboy 3y ago
  Your device would scan your photo at the point of you uploading it to the cloud and then it could encrypt it before sending it to the cloud. That meant that Apple's cloud servers didn't need to be able to scan it to comply with US Govt "recommendations" for cloud providers. Whereas right now all the other cloud providers just send the photo as-is and scan it on the cloud servers. With Apple's approach, the cloud servers don't get to look at every single one of your photos like cloud vendors do today, scanning happens within the privacy of your own phone, and only known-kiddy-porn signatures are flagged. Apple came up with a way to make things way more private, but the concept of your own device working "against" you if you happen to be a pedophile was too much of a leap.
- theshrike79 3y ago
  Your device would've scanned your photos ONLY if you would've uploaded them to Apple's cloud service anyway. And it wouldn't have notified Apple of "something evil", just specifically known and human-verified actual real child abuse photos. And not even that, it would have needed multiple matches of those very real and verified abuse photos before it flagged them so that a real human could see a "visual derivative" of the photos. Only if those multiple matches of derivatives were deemed as actual, very real, child pornography the authorities would've been called. But nope. Now they just scan ALL your data in the cloud when the authorities demand it. And that's somehow better according to the internet in a way I still can't understand.

no_time 3y ago

> I’m not sure I understand apples logic here. Are iCloud Photos in their data centers not scanned? Isn’t everything by default for iCloud users sent there automatically to begin with? Doesn’t the same logic around slippery slope also apply to cloud scans? I don’t see the problem with this status quo. There is a clear demarcation between my device and their server. Each serving the interests of their owner. If I have a problem with their policy, I can choose not to entrust my data to them. And luckily, the data storage space has heaps of competitive options.

chatmasta 3y ago
> the data storage space has heaps of competitive options The generic space does, yes. But if you want native integration with iOS, your only choice is iCloud. It would certainly be nice if this was an open protocol where you could choose your own storage backend. But I think the chances of that ever happening are pretty much zero.
judge2020 3y ago
This status quo is that a lot of countries want to use the CSAM argument to push privacy-invasive technology (cough UK) like e.g. forcing companies to allow the government to break E2EE to catch CSAM distributors. Apple made this feature while planning to move iCloud Photos to E2EE so that they could argue "look, we still catch x CSAM distributors with n < 0.x% false positive rate, even with E2EE photos. therefore you don't need to pass these laws that break E2EE."
- no_time 3y ago
  I know "give them an inch, they take a mile" is a reductive comparison but I really can't see this way of thinking going any other way in the long term.
  - salawat 3y ago
    It isn't reductive. At the end of the day, that's exactly what it comes down to.
3y ago

Moldoteck 3y ago

so users who didn’t use iCloud would’ve never been scanned to begin with. - so why not implement csam for icloud only without local scanning?

Gigachad 3y ago
Because the idea is that the iCloud data would be encrypted so their servers couldn’t scan it. With the plan being they would do on device scanning of photos that were marked as being stored on iCloud. It’s objectively better than what google does but I’m glad we somehow ended up with no scanning at all.
- Moldoteck 3y ago
  that sounds strange, I mean i'm not sure what's the big difference. If data is scanned on icloud, this means it's not encrypted, got it, if scanned on devices, data is fully encrypted on icloud, but apple has access by scanning it on devices and can send unencrypted matches, so it behaves as an unencrypted system, that can be altered at apple's will, just like icloud... but still, why scanning locally only if icloud is enabled? why not scan regardless? Since policy is meant to 'catch bad ppl', why limit to icloud option and not scan all the time
  - nemothekid 3y ago
    Apple doesn’t want to scan period. However if Apple does E2ee icloud, the biggest political issue will be that of CSAM. So in order to reserve CSAM, they came up with this scheme. Apple doesn’t want to expand their power which is why they don’t scan locally. They weren’t doing it before and they don’t want to offer it now.
  - kalleboo 3y ago
    > Since policy is meant to 'catch bad ppl', why limit to icloud option and not scan all the time The policy is meant to ensure Apple's servers are not storing and distributing CSAM, not that Apple wants to become a police investigative force.

turquoisevar 3y ago

> Are iCloud Photos in their data centers not scanned? No outright statement confirming or denying this has ever made to my knowledge, but the implication, based both on Apple's statements and the statement of stakeholders, is that this isn't currently the case. This might come as a surprise to some, because many companies scan for CSAM, but that's done voluntarily because the government can't force companies to scan for CSAM. This is because based on case law, companies forced to scan for CSAM would be considered deputized and thus subsequently it would be a breach of the 4th amendments safeguards against "unreasonable search and seizure". The best the government can do is to force companies to report "apparent violations" of CSAM laws, this seems like a distinction without a difference, but the difference is between required to actively search for it (and thus becoming deputized) v. reporting when you come across it. Even then, the reporting requirement is constructed in such a way as to avoid any possible 4th amendment issues. Companies aren't required to report it to the DOJ, but rather to the NCMEC. The NCMEC is a semi-government organization, autonomous from the DOJ, albeit almost wholly funded by the DOJ, and they are the ones that subsequently report CSAM violations to the DOJ. The NCMEC is also the organization that maintains the CSAM database and provides the hashes that companies, who voluntarily scan for CSAM, use. This construction has proven to be pretty solid against 4th amendment concerns, as courts have historically found that this separation between companies and the DOJ and the fact that only confirmed CSAM making its way to the DOJ after review by the NCMEC, creates enough of a distance between the DOJ and the act of searching through a person's data, that there aren't any 4th amendment concerns. The Congressional Research Service did a write up on this last year for the ones that are interested in it[0]. Circling back to Apple, as it stands there's nothing indicating that they already scan for CSAM server-side and most comments both by Apple and child safety organizations seem to imply that this in fact is currently not happening. Apple's main concerns however, as stated in the letter by Apple, echo the same concerns by security experts back when this was being discussed. Namely that it creates a target for malicious actors, that it is technically not feasible to create a system that can never be reconfigured to scan for non-CSAM material and that governments could pressure/regulate it to reconfigure it for other materials as well (and place a gag order on them, prohibiting them to inform users of this). At the time, some of these arguments were brushed off as slippery slope FUD, and then the UK started considering something that would defy the limits of even the most cynical security researcher's nightmare, namely a de facto ban on security updates if it just so happens that the UK's intelligence services and law enforcement services are currently exploiting the security flaw that the update aims to patch. Which is what Apple references in their response. 0: https://crsreports.congress.gov/product/pdf/LSB/LSB10713 https://crsreports.congress.gov/product/pdf/LSB/LSB10713

matwood 3y ago

You are correct, the original method would only have scanned items destined to iCloud and only transmitted some hash of matching hashes. And yes, similar slippery arguments exist with any providers that store images unencrypted. They are all scanned today, and we have no idea what they are matched against. I speculated (and now we know) when this new scanning announced, that it was in preparation for full E2EE. Apple came up with a privacy preserving method of trying to keep CSAM off their servers while also giving E2EE. The larger community arguments swayed Apple from going forward with their new detection method, but did not stop them from moving forward with E2EE. At the end of the day they put the responsibility back on governments to pass laws around encryption - where they should be, though we may not like the outcome.

gonehome 3y ago
There are also ways to detect matches even with e2ee iirc and I suspect they found doing that instead easier than dealing with the previous approach. At the time I also thought it was obvious it was in preparation for e2ee (despite loud people on HN who disagreed). I do wonder if they had intended to have it be default on though, maybe not since probably better for most users to have a recovery option.
- judge2020 3y ago
  > There are also ways to detect matches even with e2ee By definition, encryption (with unique user keys) means you can't infer nor check what the content of the message is. Not without client cooperation, which is what this feature would have been.
  - gonehome 3y ago
    This is what I was recalling, this method gives you a clever way to do it using the file itself as the key: > “Convergent encryption solves this problem in a very clever way: “The way to make sure that every unique user with the same file ends up with an encrypted version of that file that is also identical is to ensure they use the same key. However, you can’t share keys between users, because that defeats the entire point; you need a common reference point between users that is unknown to anyone but those users. “The answer is to use the file itself: the system creates a hash of the file’s content, and that hash (a long string of characters derived from a known algorithm) is the key that is used to encrypt said file. “If every iCloud user uses this technique — and given that Apple implements the system, they do — then every iCloud user with the same file will produce the same encrypted file, given that they are using the same key (which is derived from the file itself); that means that Apple only needs to store one version of that file even as it makes said file available to everyone who “uploaded” it (in truth, because iCloud integration goes down to the device, the file is probably never actually uploaded at all — Apple just includes a reference to the file that already exists on its servers, thus saving a huge amount of money on both storage costs and bandwidth). “There is one huge flaw in convergent encryption, however, called “confirmation of file”: if you know the original file you by definition can identify the encrypted version of that file (because the key is derived from the file itself). When it comes to CSAM, though, this flaw is a feature: because Apple uses convergent encryption for its end-to-end encryption it can by definition do server-side scanning of files and exploit the “confirmation of file” flaw to confirm if CSAM exists, and, by extension, who “uploaded” it. Apple’s extremely low rates of CSAM reporting suggest that the company is not currently pursuing this approach, but it is the most obvious way to scan for CSAM given it has abandoned its on-device plan.” https://stratechery.com/2022/apple-icloud-encryption-csam-scanning-and-convergent-encryption/ https://stratechery.com/2022/apple-icloud-encryption-csam-sc...