
Russian photographer matches random people with social network photos - davesailer
http://rbth.com/science_and_tech/2016/04/12/russian-photographer-matches-random-people-with-social-network-photos_584153
======
alexc05
The first thing that struck me (After I got over the whole,
creepy/horror/revulsion thing) was how much more fun people seemed in their
social media lives.

Rather than seeing this as an exposition of "people on social media are so
much more bland than they appear" I found myself feeling "people on the subway
are so much more beautiful and interesting in their 'real life'"

I feel like that might be backwards from how it gets presented elsewhere.

But, if you look for example at the blonde woman snarling a piece of cake
[https://birdinflight.com/ru/vdohnovenie/fotoproect/06042016-...](https://birdinflight.com/ru/vdohnovenie/fotoproect/06042016-face-
big-data.html) (second last) I thought, that was charming and funny... But
you'd never know that from under they grey fluorescent lights of the subway
car.

It's really interesting to be confronted with the realization that every
single person filling up space on the subway around you has a rich and
lustrous story that brought them to that place and time.

Still, super creepy.

~~~
sangnoir
> It's really interesting to be confronted with the realization that every
> single person filling up space on the subway around you has a rich and
> lustrous story that brought them to that place and time.

The dictionary of obscure sorrows[1] names this feeling "Sonder" _The
realization that each random passerby is living a life as vivid and complex as
your own—populated with their own ambitions, friends, routines, worries and
inherited craziness_

1\. it's a made-up word, not a formal English one.
[http://www.dictionaryofobscuresorrows.com/post/23536922667/s...](http://www.dictionaryofobscuresorrows.com/post/23536922667/sonder)

~~~
alexc05
Lovely! But quite the opposite of a sorrow, I felt it was a somewhat beautiful
realization.

I'm sure if I was having a low day it could take it as a sort of "we're
insignificant specks on a floating rock" but I was having a great day so tra-
la-la :)

------
justsaysmthng
I don't see why this can't be done in real time. Point your phone towards a
person in a cafe and immediately figure out her name, age, maybe relationship
status, social class, augmented reality style.

5 minutes later accidentally bump into her .. and start a conversation, armed
with lots of private info about the person.

Or even more basic - thieves picking their targets - is this woman's purse
likely to contain a lot of money and jewelry ?

I guess you could even write a tool which picks all the faces from a photo,
and detects who's probably richest, most popular, most followed, etc.

But this is unavoidable, right ? We just have to accept it as a fact of life
and adapt to it..

~~~
foltz
How long before anti-facial recognition makeup, hair, and accessories becomes
fashionable? [https://cvdazzle.com](https://cvdazzle.com)

~~~
cryptoz
That doesn't seem like it would work at all. The makeup and hair styles
suggested would make you stand out even more! Sure, 1990s-style facial
recognition won't work as reliably, but surely today's systems would flag you
right away!

~~~
foltz
I haven't tested it myself but according to CVdazzle it works. So the next
question is how long before facial recognition technology overcomes efforts to
confound it. Or how long until legislators get involved.

Yes the looks are ridiculous, but most fashion starts out that way. People
often look for ways to stand out and being "retro" could enhance the cool
factor.

What does someone care if a mugger's facial recognition software "flags" them?
And who exactly is being flagged? And if a security surveillance system flags
you what is the consequence? Who does that consequence apply to?

~~~
roywiggins
>And if a security surveillance system flags you what is the consequence? Who
does that consequence apply to?

[http://www.bbc.com/news/technology-35111363](http://www.bbc.com/news/technology-35111363)

> "What you can do now is link your face-recognition system to Facewatch and
> it will pull down the watch list that's relevant for your premises and your
> group," he says. "Then, if someone walks into your premises, it will send
> you an alert."

So if you have a criminal record, or your face somehow ends up in that
database, you could find yourself followed around shops and asked (or told) to
leave. These sorts of massive watch lists aren't feasible with merely human
recognizers.

------
larrik
The artist (or the article) suggests that the pictures he took on the metro
are the 'real' person, whereas the pictures on the social site are the 'ideal'
person.

I'm not so sure. I think its very possible that the person in the social
network photographs is actually far closer to the person they really are than
the photo of them riding alone on the subway.

~~~
sp332
I thought exactly the same - and the photos chosen for the article don't
really back up the point either. The pairs look like the same "person" to me,
not one real and one fake.

Anyway I don't know anybody who busts out their truest expression of self to
ride the subway.

~~~
tom_mellior
> The pairs look like the same "person" to me, not one real and one fake.

I seem to be alone in this, but... they don't even look like the same person
to me. Do we have any confirmation that the pairs show the same person? About
10 million Russian men look like the guys in the second pair of photos.

~~~
burkaman
I think they just picked the two most striking examples for that article. Look
at the rest here:
[https://birdinflight.com/ru/vdohnovenie/fotoproect/06042016-...](https://birdinflight.com/ru/vdohnovenie/fotoproect/06042016-face-
big-data.html)

Most of them are clearly the same person.

~~~
aab0
I dunno. Some of them clearly are because you can see they have moles in the
same place and stuff like that, but a bunch leave me dubious. For example,
that woman with the dyed white hair: the hair looks much frizzier/curlier in
one than the other. And some I wonder how the face algorithm could possibly
have matched them up - the guy in the parka vs the soldier don't even have the
same facial hair, never mind how the parka fuzz obscures most of his head.
(And you can't see more than a quarter of the next woman's face!) I wonder how
much of this matching was manual.

~~~
abrowne
Did it have to match those exact photos? I think he took a photo (or a few),
used that (or those) to find the social media account(s). Then he could use
whatever photos of that person he wanted to match up with a photo he had
teken.

------
blakesterz
I keep trying to figure out why this feels so wrong and creepy. You're in
public, you can't avoid some random person from taking your picture. You have
a public profile, you post pictures of yourself there, in public. But some how
this feels... wrong, why is that?

Something about the surprise of it? It feels like a surprise, it feels like
something that shouldn't be so easy.

~~~
mariodiana
I think it feels wrong because for all of human history a person could take
for granted a degree of anonymity while among strangers, and more and more
that seems to be no longer the case.

Forever, if someone recognized you, chances are you recognized them. People
were either within some extended social circle or they were not, and this
social circle was face-to-face. Now, we have something of an arms race going
on with respect to anonymity, or the lack thereof. Maybe someone will scan me
and identify me. What's my recourse? Either I go about in disguise, or I scan
everyone around me in order to equalize things.

How could all this not feel strange or wrong? Based on the norms of 10,000
years of human society, it's either a-social or downright anti-social.

~~~
SixSigma
> I think it feels wrong because for all of human history a person could take
> for granted a degree of anonymity

I think for most of human history, humans knew the majority of the other
humans within travelling distance and people were wary of strangers because
there was no reciprocal trust relationship.

------
xentronium
FindFace is a very interesting piece of technology. It's legit (the authors,
N-Tech.Lab, have successfully competed against google and other strong
opponents in MegaFace Benchmark). It's not real-time, but fast enough (0.5-1.0
minute time required to recognize faces and provide their profile links).
Their business model is freemium (first 10 or 30 searches per month are free),
and they're trying to make it a matching service, you can narrow search by age
and relationship status, and then you can like photos of people you find to
allow searching for "your type" of partners.

~~~
hayksaakian
their website was pretty unhelpful is there a signup page somewhere i want to
try it out

~~~
gk1
Here you go: [http://findface.ru/](http://findface.ru/)

Looks as though it's only available in Russian and only works for VKontakte
(the social network).

Warning: It highjacks your back button.

~~~
hayksaakian
thank you

------
nkw
This is one of the cool features people have discussed for the past 20 years
for wearable computers. I was hoping it would come to fruition with Google
glass until Google got all weird about facial recognition on glass (and
subsequently glass as a whole).

On one side it is creepy, on the other it would be nice to have your digital
assistant suitably remind you the name of the person who is walking towards
you and recognizes you but you cannot remember who they are. A quick summary
of who they are and perhaps a summary of your last interaction whispered in
your ear could be handy. Or even a couple of relevant recent headlines about
their company, etc.

Of course creepy side is pointing at random stranger on street and instantly
having their digital dossier overlaid on their face.

~~~
yolesaber
I personally would be pretty disgusted if someone used it on me. If someone
doesn't remember my name or their last interaction with me, maybe it's for the
best. The last thing I need is people faking nice and friendly after they just
scanned me without my consent.

~~~
type0
But this could be useful for someone with Alzheimers, or maybe a teacher that
needs to remember his pupils.

~~~
TeMPOraL
Or normal people who, like me, can't connect names to faces without at least
several interactions with a person in a short interval of time.

~~~
yolesaber
So instead of working on your memory or coming up with a mnemonic or just
asking their name again (everyone forgets a name), you think it'd be less
intrusive to scan someone without their permission?

~~~
TeMPOraL
Of course. Why would I need permission to analyze the same image that hits my
eyes and to consult a digital database I maintain (or public one I have access
to)? Going this way I'd end up rejecting bicycles and notepads because they're
not "natural" enough.

Personally, I believe that your tools are extensions of your body.

~~~
yolesaber
I'm sorry but your social faculties are way out of whack if you think it's a
better option to scan someone than to simply ask their name. Do you not
understand that I may feel uncomfortable if you've just scanned me and then
use that data to try and strike up a conversation? Talk about making me feel
like a product or mark. Bicycles and notepads do not actively analyze and
reduce people nor do they enable a surveillance state. Don't make such
specious extrapolations.

Tools are an extension of your body, yes, but you don't go using your hands to
feel up someone because you needed more texture data. You don't stare at
someone for a long time because that's rude and intrusive. Same goes for using
technology like this.

~~~
TeMPOraL
> _Do you not understand that I may feel uncomfortable if you 've just scanned
> me and then use that data to try and strike up a conversation? Talk about
> making me feel like a product or mark._

Do you feel uncomfortable about the fact that most of your friends don't even
know your birthday date or phone number, and have both noted down somewhere
(be it a phone or indirectly via profile info of your Facebook account)? Is it
weird for you that some may even note down things you like so that they can
check them out when looking for a good gift for you? Do you feel "like a
product or mark" when a conference badge you wear displays your occupation or
interests (sometimes used as an icebreaker)?

(Also, you shifted from just scanning to using the results to "try and strike
up a conversation"; those are two different things.)

> _Tools are an extension of your body, yes, but you don 't go using your
> hands to feel up someone because you needed more texture data._

By scanning I'm not touching you. That would be a violation, yes. But why
should I let you be a master of _my_ eyes, and the (hypothetical, at the
moment) tools I use to augment them?

People will get used to it, like they do with everything we've invented since
dawn of time. Social conventions will adjust. Just because my eye implant can
tell me what's your name doesn't mean I get to stop you on the street and
start talking to you, no more than it is an acceptable behaviour today.

~~~
yolesaber
The difference is that for my friends I have granted them explicit privileges
beforehand to have access to that kind of data. It's different when a stranger
is scanning me, mining me for data. And actually yes I do feel like a product
or mark at conferences when people come up to me and just chat me up because
of the company I work for (you've heard of it) - they almost always want quid-
pro-quo type interactions or otherwise don't care about _me_ per se, but just
the quantification of me that exists on the badge. It's one of the reasons I
don't enjoy industry conferences that much or when I do go, I prefer not to
wear a badge.

And honestly, scanning from a distance is still incredible invasive.

You can be the "master of your eyes" but don't get offended if I or others
think you're a creep. You may not be touching me but you're definitely
noticeably staring. The rejection Google Glass actually gives me some faith in
the decency of people, but you're probably right, society at large will adjust
because the profit motive and law enforcement capabilities for this sort of
technology are too great for them not to become entrenched by media and
government interests. Personally I find it inhumane and repulsive, but I'm old
fashioned like that. I prefer more organic human interactions, nuance, getting
to actually know someone beyond a readout on a screen. I'm a private person in
an increasingly public circus of society.

------
zmitri
Another corollary of this is that if you take a good image of someone else,
you can build a model of it (sometimes automatically), and you can map it on
to your face and appear as them digitally. I've built tools like this that
even let you do it live in the browser using this library
[https://github.com/auduno/clmtrackr/](https://github.com/auduno/clmtrackr/)

------
huuu
Reminds me of "Stranger Visions" from Heather Dewey-Hagborg. She created 3D
faces by using DNA found in public places:
[http://deweyhagborg.com/projects/stranger-
visions](http://deweyhagborg.com/projects/stranger-visions)

------
jpatokal
Direct link to the photographer's site:
[https://birdinflight.com/ru/vdohnovenie/fotoproect/06042016-...](https://birdinflight.com/ru/vdohnovenie/fotoproect/06042016-face-
big-data.html)

------
bcheung
I've seen this happen in real time with augmented reality googles. It had a
bunch of the Facebook profiles in advance though so it was a much smaller
search space.

------
imaginenore
I don't believe these claims.

These photos are matched too well, and I suspect he did it manually. Some of
the train photos are so bad, there's no way they got automatically recognized.

Not to mention the resources you would need just to get the data on 280
million users, let alone train a neural net on it, and make it work with a 70%
success rate.

I think it's a viral way to get app installs. The app reviews are really bad
btw, it doesn't work.

~~~
egjerlow
I agree that this seems very far-fetched. From what I know about facial
recognition (admittedly very little) for an algorithm to be able to recognize
these profiles in such different conditions as displayed in the article seems
very unlikely.

Incidentally, is there anyone who could ELI5 how picture data like this is
treated in order to run these algorithms on it?

------
gwern
I am not sure this works or that success rates like claimed are possible:

> In 70 percent of cases, he was quite easily able to identify the people he
> had photographed.

VKontakte has 280m+ accounts and is the second most popular website in Russia
([https://en.wikipedia.org/wiki/VK_%28social_networking%29);](https://en.wikipedia.org/wiki/VK_%28social_networking%29\);)
the Russian population is ~144m. I can't find any decent estimate of how many
Russians use VK, but it seems to be at least as dominate in Russia as Facebook
in the USA, which is to say, ~71% ([http://www.pewinternet.org/fact-
sheets/social-networking-fac...](http://www.pewinternet.org/fact-
sheets/social-networking-fact-sheet/)). So from non-VK usage alone, he should
be able to identify no more than 70-80 percent since you can't identify those
who aren't on VK in the first place.

Facial recognition is a notoriously difficult machine learning task, and doing
facial recognition on the VK population of St Petersburg (>249688 * 0.7 =
>174781) will be challenging even with extremely good facial recognition. I
don't know what this 'FindFace' application is, but I'll be charitable and
assume it matches the 2015 state of the art, "Facenet: A unified embedding for
face recognition and clustering"
[http://arxiv.org/abs/1503.03832](http://arxiv.org/abs/1503.03832) AFAIK, with
a best-case binary classification accuracy of 99.63%
([https://en.wikipedia.org/wiki/Accuracy_and_precision#In_bina...](https://en.wikipedia.org/wiki/Accuracy_and_precision#In_binary_classification)):

0.9963 = true positives + true negatives / true positives + true negatives +
false negatives + true negatives

If each person = 1 profile photo, then to classify a photo of someone there
would be 1 true positive, 174781-1 true negatives, and unknown number of false
negatives.

0.9963 = (1 + (174781-1)) / 1 + (174781-1) + x + (174781-1) 0.9963 = 174781 /
349561 + x 0.9963 _349561_ (1/174781) = x 1.9925943 = x

There would be ~2 false negatives for every true positive. So there would be a
<1/3 chance the photographed person is not on VK at all, and when one searched
for them when they are in fact on VK, they would only show up <1/3 of the time
(the other two-thirds their subway photograph would not 'match' when tested in
the dataset and it would be a false negative, and it gets much worse if the
face detection has one of the lower state of the art accuracies on the other
datasets). So on net you would expect to turn up a person <07 * 0.3 = <0.21 of
the time. (It gets worse if we take into account the false positives as well
but that particular paper doesn't report precision.)

There is no mention of any attempt to validate the findings, so I think for
many of his results, he may have merely found a similar-looking person
(possibly in some cases, look-alikes
[http://www.nytimes.com/2014/08/26/science/looking-at-twin-
pe...](http://www.nytimes.com/2014/08/26/science/looking-at-twin-personality-
through-look-alikes.html) [https://en.wikipedia.org/wiki/Look-
alike](https://en.wikipedia.org/wiki/Look-alike) ). Look-alikes are far from
identical twins, so it's not necessarily surprising that

> Often, there was a striking difference between a person’s real look and the
> image they projected on their social network profile: a shy and grim young
> man might appear to be the life and soul of the party and a lover of extreme
> sports on his VKontakte page.

This could be due to care in social presentation. Or it could just be that
they're not the same person in the first place. (The photos are not that
persuasive. I'm sure there's more than one grim young men with widow's peaks
in St Petersburg, or young women with curly black hair.)

~~~
gambler
I appreciate your skepticism. And I mean that in the most positive way
possible. Seems like most of HN visitors areskeptical about almost everything
except extraordinary claims regarding AI prowess.

But really, you're being _way_ too generous. 99.63% accuracy was reported on a
relatively constrained dataset[1] when being asked a yes-no question: "are
these two faces the same"? In our case, however, we're speaking about a search
algorithm that tries to match something in a huge database. The accuracy will
be way, way worse.

[http://vis-www.cs.umass.edu/lfw/](http://vis-www.cs.umass.edu/lfw/)

------
brokencog
Completely missing the point. "Social Media Site" aka, a place where we put
(mostly) public imagery. By definition this is curated, and as the prof says
is a self-selected view of ourselves.

But, the point being missed is that OF COURSE people don't look the same "in
day to day life" because those social media pics are just a moment of one day.
If the prof happened to meet the person moments after the social media picture
was taken then ... he would have concluded that "people present themselves
truly as they are in reality" ...

My point, the point ignored in the article, is that Social Media is a self-
reflective, self-curated snapshot. Not a continuous, objective representation.

------
dsfyu404ed
Maybe it's not a popular look over there but I couldn't help but notice there
were no photos of guys with a full beard, ball cap and aviator style
sunglasses...

------
prawn
We are not far off AR-type scenarios where someone in public is overlaid with
info about them and their interests. Keen on that girl? It instantly tells you
she likes a particular band and movie genre, giving you quick access to small
talk cues. Or that she never mentions a boyfriend online.

Or that the guy about to get off the subway earlier thanked his friend for
giving him that money he owed him.

------
flashman
Minority Report's personal advertising is getting closer and closer:
[https://www.youtube.com/watch?v=7bXJ_obaiYQ](https://www.youtube.com/watch?v=7bXJ_obaiYQ)

And they won't even need retinal scanners. Facial recognition will probably be
good enough.

------
jcoffland
I found it interesting that with out exception in the photos on the artist's
Website the people on the subway are activity using their smart phones and/or
have headphones on. The transformation of the human race into cyborgs, of
sorts, over the last two decades is uncanny.

------
prohor
So, you can no longer assume being anonymous when walking down the street. If
your random photo is matched with your profile, you can be basically fully
identified anywhere. With all those cameras everywhere, you could be fully
tracked.

~~~
Karlozkiller
Given that there are public images with your face in them and name connected
to them, that can be searched by a neural net... Yes.

~~~
brohee
No need for the images to be public. A police state could match images taken
by street CCTV to e.g. the drivers license database and track the citizenry in
real time.

You could easily reduce the search space to match a face taken by a CCTV
camera by using cell phone location data to limit it to people in the general
area...

~~~
Karlozkiller
A police state has always been able to do this. Of course it gets easier by
automatic face recognition and the possibility of sharing databases over long
distances etc.

The difference from before I guess, is that it's less in your face, and can
thereby happen without people noticing as easily.

~~~
chopin
>A police state has always been able to do this.

Not for everybody all the time, though...

------
bflesch
I have registered [http://wikiface.org/](http://wikiface.org/) some time ago
to create an open facial recognition database as a proof of concept project.

------
xerophyte12932
It's interesting how almost everyone he photographed completely ignored him by
either closing their eyes, diving into their cellphone or just pretending to
be interested in something else

------
eggy
And friends bash me for always using a caricature avatar of myself on my
social media accounts! I'll have to run it, and see if it matches a photo.
It's a greyscale line drawing sketch.

------
stefanix
This is very much a study of bad vs good lighting.

------
mamon
If this becomes more common then dating websites, like Badoo are doomed :)

------
known
If drones have Facial recognition software they'll become ultimate killing
machines;

------
naskwo
And this is exactly why, 11 years ago, I set up Famipix (www.famipix.com).

~~~
pbhjpbhj
Link is to a "private photo sharing" website.

I'm curious naskwo, it says in the tour "We don’t synchronise any content to
any external sites or social networks." \- this implies you don't have off-
site backups?

Second, for the birthdays calendar do you do anything like attempting to
recognise photos of birthdays and log them, or use image meta-data to do the
same?

