
To Fight Revenge Porn, Facebook Is Asking to See Your Nudes - kawera
https://motherboard.vice.com/en_us/article/7x478b/facebook-revenge-porn-nudes
======
CobrastanJorji
Hi, this is a legitimate email from Facebook. Nude photo blackmailing is a
growing problem. You may have heard that we, Facebook, are having users upload
nude photos to fight revenge porn. Please reply to this email, attaching any
nude photos that you want to block Facebook from ever publishing. Please also
include a rough idea of the amount of ransom money you would be willing to pay
to keep these photos out of public circulation.

------
y7
This will also be very useful for censorship and trolling, tagging images as
your nudes to prevent them from getting shared or having others get banned.
That might be why the images are transmitted to Facebook first, to check
whether it's actually a nude image of yourself.

~~~
RickS
It's especially tough if they're being truthful that they don't store the
image, only the hash. Very tough to dispute if the photo itself can't be
reviewed in a dispute to see if it was a meme. The other comment about ML img
categorization is probably the best bet.

------
netsharc
I really hope this doesn't become reality. Otherwise, you can look forward to
the phishing mail which will try to get your nudes. "DANGER! We found
potential nude pictures of you! Upload your pictures to verify your identity,
find who uploaded them without your consent, and block the pictures from being
published. If you do not act the pictures will be published." ...

------
cannonedhamster
...So are they deleting the original image afterward? Are you now giving
Facebook the right to advertise to you based on your nude images? What happens
if someone breaks into your account? Are your nude photos of yourself visible
to them? Also trusting a company expressly set up to data mine to help you
prevent abuse is pretty much the fox guarding the henhouse. Facebook is meant
to share. Let's hope they don't accidentally overshare.

Edit: I'm seeing the researcher saying they aren't storing an image but I'm
not seeing where they say they aren't.

Edit 2: They are apparently storing a link to the image on the internet, but
aren't storing the image itself, which again leads me to wonder how the hash
is being generated. The image has to be uploaded to the servers at some point.
Is it then deleted?

------
udp
Absolute clickbait. Facebook is allowing you to _flag_ pictures you have
_already_ shared in messages as something that you don't want to be shared any
further. They are not asking you to upload pictures they don't already have,
even if according to the article you can do this by starting a messenger
conversation with yourself. No idea why this is being upvoted.

~~~
hsitz
What is your source for your assertion that they're not asking you to upload
any pictures? That conflicts both with the article linked in this thread's
title and with their source: [http://www.abc.net.au/news/2017-11-02/facebook-
offers-reveng...](http://www.abc.net.au/news/2017-11-02/facebook-offers-
revenge-porn-solution/9112420)

The fact, as reported, is that you if you want to block a picture (which a
former partner may also have) and you don't have it on your facebook account
already, they will tell you that you have to upload the picture to facebook
(where they say they will retain only a hash) and then flag it. This is a test
program in Australia, as reported.

~~~
udp
Even the source on the original article you linked to isn't from Facebook. I
can't track down a single citation for anyone from Facebook saying "send
pictures to yourself and flag them".

------
mikecsh
Surely they could generate the hash on your own device..?

~~~
always_good
Since sha256 or whatever isn't going to cut it, I'd assume they were at least
using some sort of perceptual hash concept
([https://www.phash.org/](https://www.phash.org/)) and it's not implemented in
Javascript.

The system would be pretty useless otherwise if you had to report the same
image multiple times for each time someone recompressed it / took a screenshot
of it / a single bit was changed.

The other likelihood is that they aren't just generating a hash for the same
reason FaceID isn't just a hash but actually doing more interesting model-
building that they only implemented server-side.

Either way, it's not very great UX. But I can imagine that if you're at a
place where you need to preemptively block your nudes from imminently
transmitting through Facebook, you probably don't care.

~~~
jsjohnst
I’m sure they are using PhotoDNA if true, not a normal generic hashing algo.
They already use it for matching CP images. It will still match an image even
if it’s been edited (cropping/overlay/basic edits/etc).

------
cvwright
Is it possible that they're breaking the law here, by soliciting inappropriate
images from any users who are minors?

Isn't that what Anthony Weiner is going to jail for?

~~~
rubystallion
Is soliciting inappropriate images from minors still illegal, if it's made
impossible for employees to see them and the images are not stored?

------
peterwwillis
Wtf? They could easily hash the image on your device without transmitting the
image. And of course it doesn't apply to pictures taken by someone other than
you. For those it would need to use machine learning to detect your face +
nudity.

~~~
gushie
Although hashing on the device might make it easier for someone to work out
how to doctor the image so it doesn't create a match

~~~
jdavis703
You only need to make a one pixel change (or just use a lossy compression
format) before a hash such as SHA256 or MD5 returns a false negative. You need
to create a perception hash of the image. I still think that could be done on-
device, but who knows what kind of machine learning/object recognition stuff
they're doing on the server side.

~~~
kiriakasis
as many have said there perceptual hashing also exist (opposed to
cryptographic hashings) like phash of photoDNA

------
leroy_masochist
At the risk of asking a technically dumb question (I understand the basics of
hashing), would there be any technical impediment to creating an app that
hashes the file itself on the handset, then uploads to FB?

~~~
justinjlynn
A perceptual hash set for scale/rotation/crop/colour invariant search? no. But
that's presumably not all Facebook is doing and that is worrying. There isn't
really any reason for them to have direct access to the unblinded data - all
they need to be able to do is ask "does this image description sound familiar?
here app on phone, here's a more detailed extract of the questionable image
and here's the reference... are they the same?". There is no reason for
Facebook to request these images for unhashed upload unless they have an
ulterior motive or are lazy.

------
mLuby
If they're just hashing the image then blocking images with identical hashes,
sounds like a good way for "activist users" to block arbitrary images (perhaps
even ads—wouldn't that be fun!)

------
mcphage
> Marines United exemplified a part of Facebook that is difficult for it to
> moderate: secret groups.

Why are secret groups any more difficult than any other kind of group for FB
to moderate? FB has access to their own data.

~~~
Chaebixi
Because Facebook doesn't want to pay people to do the work of directly
policing them, and instead relies on community reports.

------
Asdfbla
I wonder in what form they store what the article calls a hash. Certainly no
simple file hash - I guess it's some kind of feature extraction that lets them
match the image against others?

Facebook probably is even just reusing the same systems that flag child porn
or other illegal images. One of their better and more user-friendly ideas for
sure (if you are locked into their ecosystem anyway, you might as well use
their surveillance expertise to control the content you spread there.)

~~~
tyingq
Probably PhotoDNA or similar.
[https://en.m.wikipedia.org/wiki/PhotoDNA](https://en.m.wikipedia.org/wiki/PhotoDNA)

That is, a hash that survives resizes, cropping, etc.

------
comatose
come on , this isn't complicated, don't over think it-- a woman (or man) meets
someone and sends a nude; regrets the next morning; fb im the image to
themselves and flags it their partner uploads it, a fb employee sees a nude
that matched a hash, the only reason they see it is the hash matched, you're
trusting fb not to do anything but hash the image and if your partner alters
it maybe the hash doesn't match, but typically the upload is the same and when
it's clear it's revenge porn the account is banned-- it'll probably catch a
boatload of bags of dicks if people trust fb, but maybe they shouldn't

------
taylodl
This doesn't make any sense. You shouldn't need to send Facebook explicit
photos of yourself. All they should need is personally identifiable
information - your face, tattoos, moles, etc. Women should be able to use
bikini shots and men should be able to use swim trunk shots. From that they
can scan photos and find your likeness in nude shots.

Nothing good can come from this approach Facebook is taking.

~~~
rubystallion
I assume machine learning doesn't work well enough when the photo gets shared
to friends of friends and facebook has to compare the bikini shots against
10,000 other users

------
tyingq
Sadly, "Not the Onion"

------
reilly3000
And how does a simple modification such as a watermark not break the hash?

~~~
jimktrains2
It's a lossy hash function. [https://www.phash.org](https://www.phash.org) or
photodna are both starting points.

------
meritt

        [message deleted]

~~~
username223
Probably not. I assume they pay thousands of Southeast Asian "independent
contractors" a few dollars a day to identify dick pics.

------
jondubois
It sounds stupid... Storing the hashes? How many permutations of that photo
will be protected? Surely the attacker can tweak the brightness or crop the
edges.

~~~
qilo
Microsoft has somewhat similar technology called PhotoDNA.

 _How does PhotoDNA work?_

 _PhotoDNA technology converts images into a common black-and-white format and
uniform size, then divides the image into squares and assigns a numerical
value that represents the unique shading found within each square. Together,
those numerical values represent the "PhotoDNA signature" or "hash" of an
image, which can then be compared against signatures of other images. While
the technology cannot be used to identify a person or object in an image, nor
can it be used to recreate an image, it can be used to find copies of a given
image with incredible accuracy and at scale across the 1.8 billion images
shared online every day, even when the images themselves have been altered._

[https://www.microsoft.com/en-
us/PhotoDNA/FAQ?3158ae95-fce7-4...](https://www.microsoft.com/en-
us/PhotoDNA/FAQ?3158ae95-fce7-480c-98aa-d3e75c620697=True)

~~~
ToFab123
Facebook are using (or was using PhotoDna.) dunno if they still do.
[https://m.huffpost.com/us/entry/864695](https://m.huffpost.com/us/entry/864695)

~~~
jsjohnst
They do, for policing known CP images.

