10 ms·

Who needs SWATing when you can send a CP pic (either real or with hash collision as per the thread few days ago) from a virtual overseas number/service and get

Who needs SWATing when you can send a CP pic (either real or with hash collision as per the thread few days ago) from a virtual overseas number/service and get FBI van to show up as well? What about injecting code into a public website to download same pic into local browser cache without user’s knowledge? The simplicity of the attack vectors here that would trigger the “manual” investigation is just dumbfounding and ripe for abuse/misuse.

sschueller 5y ago
My public Wifi captivity portal...
bpoyner 5y ago
You could completely wreck somebody's life with this. SWATing will look trivial in comparison.
- scbrg 5y ago
  To be fair, SWATing kills people. Death is generally considered a non trivial and also life wrecking event.
  - headmelted 5y ago
    Honestly I'd rather get shot dead by a SWAT team than implicated for something as atrocious as what this tool is looking for. I imagine many people with a family would feel the same way. It's an abomination that will destroy innocent people. The engineers behind this no doubt think it's fool-proof because they believe they're leagues smarter than any of those pesky naysayers ("hey, we're Apple"). If we've learned anything about Apple this year (as if we needed the reminder) is that their software is nowhere close to as flawless as they seem to think it is.
- headmelted 5y ago
  This. Combined with the unpatched remote-root-via-phone-number disclosed in the Pegasus leak this boils down to a single-click "destroy this person's life" tool.
- donkeyd 5y ago
  If you assume that cops will just arrest people without doing any further research... Then yeah.
  - bpoyner 5y ago
    Oh, the police will get a search warrant, and find exactly what they were told would be on your device. The police aren't in the business of discovering your innocence. It's then up to you and your lawyer to prove you didn't put it on your device. Meanwhile your life will fall apart as you get fired, your wife divorces you, you lose all custody of your kids, etc.
  - shakna 5y ago
    > If you assume that cops will just arrest people without doing any further research... Then yeah. Like when they arrested & charged someone for a poor facial recognition match that never had a hope of passing human review? [0] Just glancing at the original photo would have stopped that. Or checking his rock-solid alibi. Neither of those things happened. [0] https://www.wired.com/story/flawed-facial-recognition-system-sent-man-jail/ https://www.wired.com/story/flawed-facial-recognition-system...
- dzhiurgis 5y ago
  You can already do the same. Send message then call the cops. Just because it's auto-detected now doesn't mean it wasn't possible before.
hughrr 5y ago
This is why I just noped the fuck out of the Apple ecosystem. I won’t support anything which relies on opaque blacklisting to ruin lives. In this example as well on iCloud shared galleries you can upload to other people’s ones you have been invited to. What could possibly go wrong?
- jstx1 5y ago
  Maybe wait to see how it's implemented and how it works first? I really think that the HN crowd is having a giant knee-jerk reaction to all of this.
  - bilekas 5y ago
    Its harder to take back policies like this than it is to object and get them stopped initially. Also people have a habit of 'forgetting' about it later. Until stories of how it is misused are found. And then it's another attack vector we need to be conscious of.
    - ElFitz 5y ago
      And that’s how France still has VAT & revenue taxes. Revenue tax? Have to pay for that expensive WWI war effort, you understand? For all the good it did. Same with the VAT. Have to rebuild after WWII, you understand. We also have an "Exceptional and Temporary Contribution" (CET), recently renamed to "Technical Equilibrium Contribution" (still CET. Smart one, that one). A funny one, for a change? When the Germans invaded in WWII, they changed France's timezone to theirs. After the war, we still called it "the German time". There were talks of going back for a few years… Guess who still has noon at 2pm in the summer, decades later? Change, no matter how ridiculously small or sensical, even when nobody benefits from the status quo (ie the damn timezone) is horrendously difficult. Thus one should always assume that once it’s here, whatever "it" is, it’s here to stay.
      - candiodari 5y ago
        There is still one constant: how the state system cares for victims of child abuse is still the same as in WW2. https://www.kansascity.com/news/special-reports/article238206754.html https://www.kansascity.com/news/special-reports/article23820... You would think money would go into the "backend": caring for kids where the state is responsible for everything BEFORE more money goes into the frontend: finding more kids to throw into the hellhole that is child services. Without the "backend" being in order and working well, raising well-educated, stable kids, the frontend is completely immoral. "Saving" kids from abuse, only to throw them into a slightly different kind of abuse ... if any person did that (e.g. a guy marrying a woman (or I guess vice-versa) with that resulting in that person abusing their new spouse's kids) would be considered a despicable crime. Somehow child services, who do the exact same thing (and they use violence to do it) is not a despicable crime. Somehow just because the state does it, makes such things all a-okay. But frankly this is merely the hole in the justification, all this should merely tell you one thing: any government that doesn't work hard to fix the child services backend does not have children's interests at heart when making these sorts of laws (and mostly they're making budget cuts in the backend, of course). Because fundamentally these laws throw children into the child services system. THAT is the real effect these efforts have on the actual children behind this. THAT is what is meant by "saving kids". And if that system is full of abuse, how is that any better than what paedophiles do? It's not. Which means the state is not attempting to help abused or disadvantaged children. In fact, they're doing the opposite.
jacquesm 5y ago
No code required. <img width=0 height=0> would do the job.
- mimsee 5y ago
  I wonder how long it takes until they add a feature to Safari to scan all the <img> <video> <canvas> elements for possibly illegal content. Would be very convenient considering Safari is the only browser engine on iOS.
- oleganza 5y ago
  That's fine™. You are just going to redirect blame on the original source, provided you got enough Apple Cash on balance to pay the lawyers and stay out of jail while sorting this out.
- paulcole 5y ago
  How is that going to get the image into your iCloud photo storage?
  - jacquesm 5y ago
    It doesn't, but it does get the image into your browser cache and onto your machine.
    - paulcole 5y ago
      So what does that do in the context of this conversation about Apple and iCloud?
  - schoolornot 5y ago
    Content-Disposition: attachment; hit the wrong button, done. It's in your iCloud/Downloads folder.
- madmoose 5y ago
  > <img width=0 height=0> would do the job. No, that's not how the Apple's system works.
mimsee 5y ago
Yes. This reminds me of when typing or receiving certain text would make an iPhone crash. But now having your account deleted makes it a feature. For example Whatsapp automatically downloads media to the camera roll which then get uploaded to iCloud. Of course that can be turned off prior, but this is like what happens with backing up. People want to backup, but don't invest the time in it. That's until it's too late, they lost their data and now want their stuff back.
- laurent92 5y ago
  Backup is a good point: - Apple: “Backup your phone to iCloud, it will be safe there.” - 5 minutes later: “We’ve wiped your account because of a photos of (porn actor here) which is not CP but technically minor at the time she filmed.” - “Also we’ve wiped your iPhone because we couldn’t knowingly let you keep that. Good luck contacting your parents, we’ve deleted your contacts. Good luck! PS: We’ve reported you to the police.” - Also you can’t connect to your iMac now.
  - annamargot 5y ago
    Or photos of your own children. We have a Tumblr set up for family to view pics of the kids. Several photos and videos of our kids when they were under 2 were taken down either temporarily or permanently by their CP algo. These were a pic or video of kids in the bath or without a shirt. In none of them could you see bum or bits. Just a semi naked baby. Algorithms like this get things wrong all the time
    - tpush 5y ago
      This is not the kind of algorithm that Apple is be using. That one only scans for already known CSAM in NCMEC's database.
      - shakna 5y ago
        Which may contain the hashes of their photos, because they've been taken down in the past, which means they probably have been added to certain blacklists that may have been integrated into the blackbox of NCMEC's database.
simondotau 5y ago
None of those attacks would work against the system as described by Apple. The only photos scanned are items in your photo library prior to upload to iCloud. Your browser cache is not scanned. Hash collisions would fail human review. About the only consequence I can think of for hash collisions is that the person at Apple who performs the human review step has a slightly nicer day because they were about to look at an image... and then it wasn't CSAM.
- soziawa 5y ago
  > Hash collisions would not pass the human review. About the only consequence I can think of for hash collisions is that the person at Apple who performs the human review step has a slightly nicer day because they were about to look at an image... and then it wasn't CSAM. The whitepapers provided by Apple do not say what the human reviews consists of. They could just look at the hashes to make sure there isn‘t a bug in their system.
  - simondotau 5y ago
    > The whitepapers provided by Apple do not say what the human reviews consists of. At minimum what we know is that each flagged image generates a "safety voucher" which consists of metadata, plus a low-resolution greyscale version of the image. The human review process involves viewing the metadata and thumbnail content enclosed in each safety voucher which cumulatively caused that account to be flagged.
    - foobar33333 5y ago
      A human at Apple likely doesn't get access to anything. I assume it would be part of the police group under strict restrictions checking these.
      - simondotau 5y ago
        The data is not sent to a "police group", it is sent to NCMEC. From Apple's FAQ: Will CSAM detection in iCloud Photos falsely flag innocent people to law enforcement? No. The system is designed to be very accurate, and the likelihood that the system would incorrectly flag any given account is less than one in one trillion per year. In addition, any time an account is flagged by the system, Apple conducts human review before making a report to NCMEC. As a result, system errors or attacks will not result in innocent people being reported to NCMEC.
2OEH8eoCRo0 5y ago
Don't worry. I'm sure the police will believe you and help you out. /s What you've described is pretty much the scariest thing I can imagine as far as computer crime goes.
ratww 5y ago
I remember WhatsApp used to save each received image to the iCloud Photo Album. I remember one day going to my album and seeing several memes and pics I had received but never saved. Having 3rd party apps that have access to the photo album being able to do that makes it a bit risky to have iCloud.
- EtienneK 5y ago
  WhatsApp was my first thought as well. Any app that automatically saves photos to iCloud without user interaction is a huge risk.
jbverschoor 5y ago
It's a good method of protecting important documents. Simply add some stamps on top of all documents in case someone steals them
sylens 5y ago
You don't even need to inject code into a public website. There have been no shortage of zero-click exploits for iMessage
modernerd 5y ago
The reported response from Apple offers little reassurance: > The executives acknowledged that a user could be implicated by malicious actors who win control of a device and remotely install known child abuse material. But they said they expected any such attacks to be very rare and that in any case a review would then look for other signs of criminal hacking. What triggers them to look for signs of criminal hacking? Does every manual review process involve such checks? Are they searching device backups for indicators of compromise [IoC]? What if there's no device backup or device image to scan? What if the scan fails to notice IoC? What if the device was compromised after the last backup? What if the device was compromised via physical access? What if the device isn't compromised and the material was pushed maliciously or via drive-by download? It's dangerous to assume that all material on a network-connected device arrived with the consent of the user when it can accept incoming messages from strangers, trick people into downloading files, or be compromised without your knowledge. “That isn't mine” is going to be a tough defence if you can't even take measures to log where content came from. Client-side scanning seems to amplify this issue (which could still happen with cloud storage) because at least cloud storage doesn't generally ship with or integrate deeply with messaging apps, social media, a web browser, QR codes, App Clip Codes[1] etc. The impact might be fairly low right now with the current proposal (images would have to be uploaded to iCloud, so cached browser images don't get scanned as far as we know), but the existence of the non-consensual scan in the first place is worrying, because it means such attacks are only a policy change away. [1] : https://developer.apple.com/design/human-interface-guidelines/app-clips/overview/app-clip-codes/ https://developer.apple.com/design/human-interface-guideline...
- GeekyBear 5y ago
  > The executives acknowledged that a user could be implicated by malicious actors who win control of a device and remotely install known child abuse material. Since Google has been scanning your account for kiddie porn for the past decade, wouldn't this apply equally to Google accounts? >a man [was] arrested on child pornography charges, after Google tipped off authorities about illegal images found in the Houston suspect's Gmail account https://techcrunch.com/2014/08/06/why-the-gmail-scan-that-led-to-a-mans-arrest-for-child-porn-was-not-a-privacy-violation/ https://techcrunch.com/2014/08/06/why-the-gmail-scan-that-le... All people have to do is email you kiddie porn and Google will have you arrested?
quietbritishjim 5y ago
Something slightly different but very related happened to a senior police officer in the UK. She got sent a WhatsApp message by her sister containing a horrific CP act. It was captioned with a message asking people to circulate it to identify the adult in it, and probably those who sent it around (including the sister) were acting in good faith, but actually it was still illegal to send or even possess it. No doubt the originator of the caption was a deliberate troll. She was found guilty of "possessing an indecent image of a child". [1] She tried to argue that she hadn't noticed the message, but it's not surprising that wasn't believed given that she had immediately replied to her sister saying "please call". She was sentenced to 200 hours community service, and originally sacked from her job but recently reinstated after appealing. [2] It seems that she wasn't immediately in trouble when she received the message ... so long as she had immediately reported her own sister for distributing it, even though it's clear that she hadn't deliberately done anything wrong. (In fact the sister had contacted her to ask what she should do about it. Probably her answer was "don't have already sent it me!") [1] https://www.bbc.co.uk/news/uk-england-london-50476166 https://www.bbc.co.uk/news/uk-england-london-50476166 [2] https://www.bbc.co.uk/news/uk-england-london-57501764 https://www.bbc.co.uk/news/uk-england-london-57501764
- zimpenfish 5y ago
  > a senior police officer in the UK To be fair, this is partially because the laws in the UK are, I think, fairly bonkers strict about CSAM - mere possession, whether you've looked at it or not, whether you downloaded it or not, whether you even know it's there or not, etc., is counted as criminal.
  - zionic 5y ago
    The US is the same.
    - wutbrodo 5y ago
      I believe this is incorrect. > At the same time, because of the First Amendment, child pornography offenses are not "strict liability" crimes like statutory rape: in order to convict a defendant, the government must prove that the defendant knew the material involved the actual abuse of a child https://www.zmolaw.com/child-pornography-faqs# https://www.zmolaw.com/child-pornography-faqs# I've found similar claims on the websites of a few law offices. For some reason, the official DoJ materials are pretty cagey on the topic.
nicce 5y ago
> Who needs SWATing when you can send a CP pic (either real or with hash collision as per the thread few days ago) from a virtual overseas number/service and get FBI van to show up as well? You are talking like collisions are trivial to make. I bet they have had a deep conversations in this area. At first, you would need a real hash to even try (which are hidden). Secondly, to get real material it means that it must be in their database to trigger anything. This tells a lot from sender already, and is worth to tell for police. It is quite easy to prove that someone just send it to you. And one photo is not triggering anything. Besides, sender must know that those photos must go automatically into the cloud to mean anything. > What about injecting code into a public website to download same pic into local browser cache without user’s knowledge? At least US legistlation is precise that user must willingly obtain/download CSAM material, and it must be proved. So this is not harmful for the user in the end. A lot of speculation, but does not really lead for coencequences. Almost every system can be tried to be abused, but does it really mean something, is different story.
- vineyardmike 5y ago
  > At least US legistlation > does not really lead for coencequences Except that a trial, even with an innocent verdict will SUCK and have terrible news stories about you and poison any google search for you with CSAM stories
- themaninthedark 5y ago
  Step 1: Get copies of pictures of targets kid in bath from phone/SNS Step 2: Manipulate pictures so that hash collides with CSAM Step 3: Get pictures back on targets phone so they get scanned. I don't have the skills or understanding of how the hashes are created but would this be possible? >At first, you would need a real hash to even try (which are hidden). How are the hashes hidden? It looks like they are shared: https://www.thorn.org/reporting-child-sexual-abuse-content-shared-hash/ https://www.thorn.org/reporting-child-sexual-abuse-content-s...
  - nicce 5y ago
    > How are the hashes hidden? It looks like they are shared: https://www.thorn.org/reporting-child-sexual-abuse-content-s https://www.thorn.org/reporting-child-sexual-abuse-content-s.. These hashes are not generated by Apple and are not valid. (Must be generated by their new system) They are probably very strictly guarded. They will be stored on every iOS from 15 version, somehow securely. This must limit the support of older iPhones.
robertoandred 5y ago
Just because you assume attack vectors are simple doesn't mean they are. First of all, why would Apple forward a report about something that isn't CSAM to the NCMEC?

sschueller 5y ago

My public Wifi captivity portal...

bpoyner 5y ago

You could completely wreck somebody's life with this. SWATing will look trivial in comparison.

scbrg 5y ago
To be fair, SWATing kills people. Death is generally considered a non trivial and also life wrecking event.
- headmelted 5y ago
  Honestly I'd rather get shot dead by a SWAT team than implicated for something as atrocious as what this tool is looking for. I imagine many people with a family would feel the same way. It's an abomination that will destroy innocent people. The engineers behind this no doubt think it's fool-proof because they believe they're leagues smarter than any of those pesky naysayers ("hey, we're Apple"). If we've learned anything about Apple this year (as if we needed the reminder) is that their software is nowhere close to as flawless as they seem to think it is.
headmelted 5y ago
This. Combined with the unpatched remote-root-via-phone-number disclosed in the Pegasus leak this boils down to a single-click "destroy this person's life" tool.
donkeyd 5y ago
If you assume that cops will just arrest people without doing any further research... Then yeah.
- bpoyner 5y ago
  Oh, the police will get a search warrant, and find exactly what they were told would be on your device. The police aren't in the business of discovering your innocence. It's then up to you and your lawyer to prove you didn't put it on your device. Meanwhile your life will fall apart as you get fired, your wife divorces you, you lose all custody of your kids, etc.
- shakna 5y ago
  > If you assume that cops will just arrest people without doing any further research... Then yeah. Like when they arrested & charged someone for a poor facial recognition match that never had a hope of passing human review? [0] Just glancing at the original photo would have stopped that. Or checking his rock-solid alibi. Neither of those things happened. [0] https://www.wired.com/story/flawed-facial-recognition-system-sent-man-jail/ https://www.wired.com/story/flawed-facial-recognition-system...
dzhiurgis 5y ago
You can already do the same. Send message then call the cops. Just because it's auto-detected now doesn't mean it wasn't possible before.

hughrr 5y ago

This is why I just noped the fuck out of the Apple ecosystem. I won’t support anything which relies on opaque blacklisting to ruin lives. In this example as well on iCloud shared galleries you can upload to other people’s ones you have been invited to. What could possibly go wrong?

jstx1 5y ago
Maybe wait to see how it's implemented and how it works first? I really think that the HN crowd is having a giant knee-jerk reaction to all of this.
- bilekas 5y ago
  Its harder to take back policies like this than it is to object and get them stopped initially. Also people have a habit of 'forgetting' about it later. Until stories of how it is misused are found. And then it's another attack vector we need to be conscious of.
  - ElFitz 5y ago
    And that’s how France still has VAT & revenue taxes. Revenue tax? Have to pay for that expensive WWI war effort, you understand? For all the good it did. Same with the VAT. Have to rebuild after WWII, you understand. We also have an "Exceptional and Temporary Contribution" (CET), recently renamed to "Technical Equilibrium Contribution" (still CET. Smart one, that one). A funny one, for a change? When the Germans invaded in WWII, they changed France's timezone to theirs. After the war, we still called it "the German time". There were talks of going back for a few years… Guess who still has noon at 2pm in the summer, decades later? Change, no matter how ridiculously small or sensical, even when nobody benefits from the status quo (ie the damn timezone) is horrendously difficult. Thus one should always assume that once it’s here, whatever "it" is, it’s here to stay.
    - candiodari 5y ago
      There is still one constant: how the state system cares for victims of child abuse is still the same as in WW2. https://www.kansascity.com/news/special-reports/article238206754.html https://www.kansascity.com/news/special-reports/article23820... You would think money would go into the "backend": caring for kids where the state is responsible for everything BEFORE more money goes into the frontend: finding more kids to throw into the hellhole that is child services. Without the "backend" being in order and working well, raising well-educated, stable kids, the frontend is completely immoral. "Saving" kids from abuse, only to throw them into a slightly different kind of abuse ... if any person did that (e.g. a guy marrying a woman (or I guess vice-versa) with that resulting in that person abusing their new spouse's kids) would be considered a despicable crime. Somehow child services, who do the exact same thing (and they use violence to do it) is not a despicable crime. Somehow just because the state does it, makes such things all a-okay. But frankly this is merely the hole in the justification, all this should merely tell you one thing: any government that doesn't work hard to fix the child services backend does not have children's interests at heart when making these sorts of laws (and mostly they're making budget cuts in the backend, of course). Because fundamentally these laws throw children into the child services system. THAT is the real effect these efforts have on the actual children behind this. THAT is what is meant by "saving kids". And if that system is full of abuse, how is that any better than what paedophiles do? It's not. Which means the state is not attempting to help abused or disadvantaged children. In fact, they're doing the opposite.

jacquesm 5y ago

No code required. <img width=0 height=0> would do the job.

mimsee 5y ago
I wonder how long it takes until they add a feature to Safari to scan all the <img> <video> <canvas> elements for possibly illegal content. Would be very convenient considering Safari is the only browser engine on iOS.
oleganza 5y ago
That's fine™. You are just going to redirect blame on the original source, provided you got enough Apple Cash on balance to pay the lawyers and stay out of jail while sorting this out.
paulcole 5y ago
How is that going to get the image into your iCloud photo storage?
- jacquesm 5y ago
  It doesn't, but it does get the image into your browser cache and onto your machine.
  - paulcole 5y ago
    So what does that do in the context of this conversation about Apple and iCloud?
- schoolornot 5y ago
  Content-Disposition: attachment; hit the wrong button, done. It's in your iCloud/Downloads folder.
madmoose 5y ago
> <img width=0 height=0> would do the job. No, that's not how the Apple's system works.

mimsee 5y ago

Yes. This reminds me of when typing or receiving certain text would make an iPhone crash. But now having your account deleted makes it a feature. For example Whatsapp automatically downloads media to the camera roll which then get uploaded to iCloud. Of course that can be turned off prior, but this is like what happens with backing up. People want to backup, but don't invest the time in it. That's until it's too late, they lost their data and now want their stuff back.

laurent92 5y ago
Backup is a good point: - Apple: “Backup your phone to iCloud, it will be safe there.” - 5 minutes later: “We’ve wiped your account because of a photos of (porn actor here) which is not CP but technically minor at the time she filmed.” - “Also we’ve wiped your iPhone because we couldn’t knowingly let you keep that. Good luck contacting your parents, we’ve deleted your contacts. Good luck! PS: We’ve reported you to the police.” - Also you can’t connect to your iMac now.
- annamargot 5y ago
  Or photos of your own children. We have a Tumblr set up for family to view pics of the kids. Several photos and videos of our kids when they were under 2 were taken down either temporarily or permanently by their CP algo. These were a pic or video of kids in the bath or without a shirt. In none of them could you see bum or bits. Just a semi naked baby. Algorithms like this get things wrong all the time
  - tpush 5y ago
    This is not the kind of algorithm that Apple is be using. That one only scans for already known CSAM in NCMEC's database.
    - shakna 5y ago
      Which may contain the hashes of their photos, because they've been taken down in the past, which means they probably have been added to certain blacklists that may have been integrated into the blackbox of NCMEC's database.

simondotau 5y ago

None of those attacks would work against the system as described by Apple. The only photos scanned are items in your photo library prior to upload to iCloud. Your browser cache is not scanned. Hash collisions would fail human review. About the only consequence I can think of for hash collisions is that the person at Apple who performs the human review step has a slightly nicer day because they were about to look at an image... and then it wasn't CSAM.

soziawa 5y ago
> Hash collisions would not pass the human review. About the only consequence I can think of for hash collisions is that the person at Apple who performs the human review step has a slightly nicer day because they were about to look at an image... and then it wasn't CSAM. The whitepapers provided by Apple do not say what the human reviews consists of. They could just look at the hashes to make sure there isn‘t a bug in their system.
- simondotau 5y ago
  > The whitepapers provided by Apple do not say what the human reviews consists of. At minimum what we know is that each flagged image generates a "safety voucher" which consists of metadata, plus a low-resolution greyscale version of the image. The human review process involves viewing the metadata and thumbnail content enclosed in each safety voucher which cumulatively caused that account to be flagged.
  - foobar33333 5y ago
    A human at Apple likely doesn't get access to anything. I assume it would be part of the police group under strict restrictions checking these.
    - simondotau 5y ago
      The data is not sent to a "police group", it is sent to NCMEC. From Apple's FAQ: Will CSAM detection in iCloud Photos falsely flag innocent people to law enforcement? No. The system is designed to be very accurate, and the likelihood that the system would incorrectly flag any given account is less than one in one trillion per year. In addition, any time an account is flagged by the system, Apple conducts human review before making a report to NCMEC. As a result, system errors or attacks will not result in innocent people being reported to NCMEC.

2OEH8eoCRo0 5y ago

Don't worry. I'm sure the police will believe you and help you out. /s What you've described is pretty much the scariest thing I can imagine as far as computer crime goes.

ratww 5y ago

I remember WhatsApp used to save each received image to the iCloud Photo Album. I remember one day going to my album and seeing several memes and pics I had received but never saved. Having 3rd party apps that have access to the photo album being able to do that makes it a bit risky to have iCloud.

EtienneK 5y ago
WhatsApp was my first thought as well. Any app that automatically saves photos to iCloud without user interaction is a huge risk.

jbverschoor 5y ago

It's a good method of protecting important documents. Simply add some stamps on top of all documents in case someone steals them

sylens 5y ago

You don't even need to inject code into a public website. There have been no shortage of zero-click exploits for iMessage

modernerd 5y ago

The reported response from Apple offers little reassurance: > The executives acknowledged that a user could be implicated by malicious actors who win control of a device and remotely install known child abuse material. But they said they expected any such attacks to be very rare and that in any case a review would then look for other signs of criminal hacking. What triggers them to look for signs of criminal hacking? Does every manual review process involve such checks? Are they searching device backups for indicators of compromise [IoC]? What if there's no device backup or device image to scan? What if the scan fails to notice IoC? What if the device was compromised after the last backup? What if the device was compromised via physical access? What if the device isn't compromised and the material was pushed maliciously or via drive-by download? It's dangerous to assume that all material on a network-connected device arrived with the consent of the user when it can accept incoming messages from strangers, trick people into downloading files, or be compromised without your knowledge. “That isn't mine” is going to be a tough defence if you can't even take measures to log where content came from. Client-side scanning seems to amplify this issue (which could still happen with cloud storage) because at least cloud storage doesn't generally ship with or integrate deeply with messaging apps, social media, a web browser, QR codes, App Clip Codes[1] etc. The impact might be fairly low right now with the current proposal (images would have to be uploaded to iCloud, so cached browser images don't get scanned as far as we know), but the existence of the non-consensual scan in the first place is worrying, because it means such attacks are only a policy change away. [1] : https://developer.apple.com/design/human-interface-guidelines/app-clips/overview/app-clip-codes/ https://developer.apple.com/design/human-interface-guideline...

GeekyBear 5y ago
> The executives acknowledged that a user could be implicated by malicious actors who win control of a device and remotely install known child abuse material. Since Google has been scanning your account for kiddie porn for the past decade, wouldn't this apply equally to Google accounts? >a man [was] arrested on child pornography charges, after Google tipped off authorities about illegal images found in the Houston suspect's Gmail account https://techcrunch.com/2014/08/06/why-the-gmail-scan-that-led-to-a-mans-arrest-for-child-porn-was-not-a-privacy-violation/ https://techcrunch.com/2014/08/06/why-the-gmail-scan-that-le... All people have to do is email you kiddie porn and Google will have you arrested?

quietbritishjim 5y ago

Something slightly different but very related happened to a senior police officer in the UK. She got sent a WhatsApp message by her sister containing a horrific CP act. It was captioned with a message asking people to circulate it to identify the adult in it, and probably those who sent it around (including the sister) were acting in good faith, but actually it was still illegal to send or even possess it. No doubt the originator of the caption was a deliberate troll. She was found guilty of "possessing an indecent image of a child". [1] She tried to argue that she hadn't noticed the message, but it's not surprising that wasn't believed given that she had immediately replied to her sister saying "please call". She was sentenced to 200 hours community service, and originally sacked from her job but recently reinstated after appealing. [2] It seems that she wasn't immediately in trouble when she received the message ... so long as she had immediately reported her own sister for distributing it, even though it's clear that she hadn't deliberately done anything wrong. (In fact the sister had contacted her to ask what she should do about it. Probably her answer was "don't have already sent it me!") [1] https://www.bbc.co.uk/news/uk-england-london-50476166 https://www.bbc.co.uk/news/uk-england-london-50476166 [2] https://www.bbc.co.uk/news/uk-england-london-57501764 https://www.bbc.co.uk/news/uk-england-london-57501764

zimpenfish 5y ago
> a senior police officer in the UK To be fair, this is partially because the laws in the UK are, I think, fairly bonkers strict about CSAM - mere possession, whether you've looked at it or not, whether you downloaded it or not, whether you even know it's there or not, etc., is counted as criminal.
- zionic 5y ago
  The US is the same.
  - wutbrodo 5y ago
    I believe this is incorrect. > At the same time, because of the First Amendment, child pornography offenses are not "strict liability" crimes like statutory rape: in order to convict a defendant, the government must prove that the defendant knew the material involved the actual abuse of a child https://www.zmolaw.com/child-pornography-faqs# https://www.zmolaw.com/child-pornography-faqs# I've found similar claims on the websites of a few law offices. For some reason, the official DoJ materials are pretty cagey on the topic.

nicce 5y ago

> Who needs SWATing when you can send a CP pic (either real or with hash collision as per the thread few days ago) from a virtual overseas number/service and get FBI van to show up as well? You are talking like collisions are trivial to make. I bet they have had a deep conversations in this area. At first, you would need a real hash to even try (which are hidden). Secondly, to get real material it means that it must be in their database to trigger anything. This tells a lot from sender already, and is worth to tell for police. It is quite easy to prove that someone just send it to you. And one photo is not triggering anything. Besides, sender must know that those photos must go automatically into the cloud to mean anything. > What about injecting code into a public website to download same pic into local browser cache without user’s knowledge? At least US legistlation is precise that user must willingly obtain/download CSAM material, and it must be proved. So this is not harmful for the user in the end. A lot of speculation, but does not really lead for coencequences. Almost every system can be tried to be abused, but does it really mean something, is different story.

vineyardmike 5y ago
> At least US legistlation > does not really lead for coencequences Except that a trial, even with an innocent verdict will SUCK and have terrible news stories about you and poison any google search for you with CSAM stories
themaninthedark 5y ago
Step 1: Get copies of pictures of targets kid in bath from phone/SNS Step 2: Manipulate pictures so that hash collides with CSAM Step 3: Get pictures back on targets phone so they get scanned. I don't have the skills or understanding of how the hashes are created but would this be possible? >At first, you would need a real hash to even try (which are hidden). How are the hashes hidden? It looks like they are shared: https://www.thorn.org/reporting-child-sexual-abuse-content-shared-hash/ https://www.thorn.org/reporting-child-sexual-abuse-content-s...
- nicce 5y ago
  > How are the hashes hidden? It looks like they are shared: https://www.thorn.org/reporting-child-sexual-abuse-content-s https://www.thorn.org/reporting-child-sexual-abuse-content-s.. These hashes are not generated by Apple and are not valid. (Must be generated by their new system) They are probably very strictly guarded. They will be stored on every iOS from 15 version, somehow securely. This must limit the support of older iPhones.

robertoandred 5y ago

Just because you assume attack vectors are simple doesn't mean they are. First of all, why would Apple forward a report about something that isn't CSAM to the NCMEC?