Shannon's Take on Apple Child Protections - ThreatWire
Added 2021-08-10 18:25:09 +0000 UTCBy Shannon Morse, ThreatWire
Trigger warning for CSAM, which is an acronym referring to sensitive pictures of kids. On Thursday of last week, Apple announced a new feature they’re bringing to iOS, iPadOS, WatchOS, and MacOS in an effort to limit the spread of CSAM here in the United States. After this news broke, there have been a lot of hot takes on this controversial topic - one taking the approach of privacy should be a right, the other side of the coin arguing that “privacy isn’t a thing in 2021, you have a smartphone, so you have no privacy anyways” which is definitely an opinion.
On the surface, this doesn’t sound that controversial. Most companies do scan for CSAM, here in the US anyway, and they have to turn over information in legal warrants. Apple does already do this in iCloud, and other tech giants do as well for their cloud storage options. But Apple in particular is planning to do this on the client side as photos are uploaded to iCloud, as well as using on-device machine learning to vet iMessage content that is sent or received by minors under the age of 13, to warn parents of explicit photos.
On the not-so-controversial side, Apple is also going to introduce an update to both Siri and Search that displays an “intervention” if a user tries to search for CSAM related topics. This would alert the user that interest in this topic is harmful or problematic.
So first is the iMessage update. This machine learning analyzes image attachments. Apple says it doesn’t get access to the message, and that it’s going to be an Opt-In setting that parents have control of through the Family Sharing feature.
If a child receives a sensitive or explicit photo, it’ll be blurred and the child will be warned, along with given options to proceed with viewing the photo and prompted that parents will also get a notification. The same applies if a child wants to send a photo. The photo would be saved to the parental controls section for parents to review. Kids between 13-17 will also get a warning but no parental notification.
For the CSAM scanning, Apple is using image hashes provided by the National Center for Missing and Exploited Children along with other child safety orgs to scan photos with “NeuralHash” which works using the cryptographic protocol Private Set Intersection or PSI - that determines if there’s a match without revealing the result. Side note: Google also uses the PSI protocol for Password Checkup in Chrome. Scanning is automatic though it only works when iCloud photo sharing is enabled. The hash itself is stored locally on the users device. To quote Apple, “The device creates a cryptographic safety voucher that encodes the match result along with additional encrypted data about the image. This voucher is uploaded to iCloud Photos along with the image.”
To interpret the hashes, Apple is using another cryptographic technology, this one called Threshold Secret Sharing, to see if an iCloud Photo account crosses a threshold of known imagery. If too many images match, then the photos get manually reviewed for human confirmation, then the user's account is disabled and the material is reported to NCMEC and law enforcement. Again to quote Apple “the system ensures the contents of the safety vouchers cannot be interpreted by Apple unless the iCloud Photos account crosses a threshold of known CSAM content. The threshold is set to provide an extremely high level of accuracy and ensures less than a one in one trillion chance per year of incorrectly flagging a given account.“ If a user is incorrectly flagged, they can file an appeal.
As an auntie to three adorable little nieces, their protection is important to me and I’m sure others relate to that. But also, privacy advocates and researchers worry that this could be expanded to detect other kinds of content, or it could be used to frame innocent people.
While not widely discussed, the iMessage update will hopefully - not - be influenced by any gender bias from the machine learning, and will hopefully be able to distinguish between innocent photos or actual sensitive content. I could imagine parents getting tons of false flags after a birthday party at a pool in the summertime.
Many are concerned that this is basically Apple building a backdoor into their device encryption. While Apple themselves have advocated for end to end encryption, even battling government agencies on user’s privacy rights, this seems to muddy those waters. This is very well-intentioned. We all know this. But does this run afoul of their own encryption stances?
Should laws change, or Apple be required to do so, this machine learning could be expanded to look for additional types of photos or sensitive content. Now to that regard, Apple pushed back with an FAQ saying that they will “not accede to any government’s request to expand it”. Apple also mentioned that the hashes will not be expanded by Apple and the list of hashes is the same across all of their platforms. OR maybe Apple added the CSAM detection before upload because they’ll add end to end encryption in iCloud - who knows? At this time, any predictions are plausible, but not fact.
This will roll out in the US in iOS 15 and MacOS Monterey in the coming months. If you’re concerned about this new scanning technology and want to make your voice heard, you can sign an open letter at appleprivacyletter.com.
Apple Scans for Sensitive Imagery
https://www.apple.com/child-safety/
https://www.zdnet.com/article/apple-child-abuse-material-scanning-in-ios-15-draws-fire/
https://www.apple.com/child-safety/pdf/Expanded_Protections_for_Children_Technology_Summary.pdf
https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf
https://www.wired.com/story/apple-csam-detection-icloud-photos-encryption-privacy/
https://appleprivacyletter.com/
Support ThreatWire! https://www.patreon.com/threatwire
