It is ripe for abuse by governments, which Apple absolutely has a history of bowing to.
Saudi Arabia WILL use this to track dissidents and homosexuals. China WILL use this for whatever they need, on a daily basis. Hell even the US will use this, I'm sure.
The only way this would be OK was if the input data (the CSAM database) was cryptographically signed and no government, no single entity and not even Apple, could change the content, with the only signing key split in 10 parts to be held personally by Bruce Schneier, the Pope, Linus Torvalds, the Orthodox Patriarch, Keanu Reeves, a couple head Rabbis and a few of whatever their equivalent in Islam is, and they had to personally review the images one by one and certify that perceptual hash 0FB89C8A7DF6AA1945B is indeed CSAM content and agree to collectively sign it for addition.
This would only work if the full IOS source was fully published, and compiled as a reproducible build, so everyone could confirm the scanning code does what it's supposed to, and is not altered with any subsequent update.
P.S.
Don't nitpick on the names, it was a deliberately absurd list of people either with a good reputation or with a lot to lose in the respectively chosen afterlife.
It is ripe for abuse by governments, which Apple absolutely has a history of bowing to.
Saudi Arabia WILL use this to track dissidents and homosexuals. China WILL use this for whatever they need, on a daily basis. Hell even the US will use this, I'm sure.
The only way this would be OK was if the input data (the CSAM database) was cryptographically signed and no government, no single entity and not even Apple, could change the content, with the only signing key split in 10 parts to be held personally by Bruce Schneier, the Pope, Linus Torvalds, the Orthodox Patriarch, Keanu Reeves, a couple head Rabbis and a few of whatever their equivalent in Islam is, and they had to personally review the images one by one and certify that perceptual hash 0FB89C8A7DF6AA1945B is indeed CSAM content and agree to collectively sign it for addition.
This would only work if the full IOS source was fully published, and compiled as a reproducible build, so everyone could confirm the scanning code does what it's supposed to, and is not altered with any subsequent update.
P.S. Don't nitpick on the names, it was a deliberately absurd list of people either with a good reputation or with a lot to lose in the respectively chosen afterlife.