
Reverse-engineering facial recognition - GFK_of_xmaspast
http://sterlingcrispin.com/data-masks.html
======
coleca
Since the site seems to be down, I'll mention that there was an article on
Medium covering this artists' work a few days ago:
[https://medium.com/matter/this-is-what-your-face-looks-
like-...](https://medium.com/matter/this-is-what-your-face-looks-like-to-
facebook-f771b3e11ed)

Seems like what he did was take Umass' Labeled Faces in the Wild (LFW)
dataset, combine all the faces together and generate a composite rendering.
Hardly a representation of "What your face looks like to Facebook".

Articles like this do little to advance the discussion of the real concerns
around facial recognition other than scare folks into an Orwellian vision of
the future a la Hollywood and Minority Report.

In addition to the work that the NTIA is doing around setting privacy
standards for facial recognition applications ([http://www.ntia.doc.gov/other-
publication/2014/privacy-multi...](http://www.ntia.doc.gov/other-
publication/2014/privacy-multistakeholder-process-facial-recognition-
technology)) there are a lot of companies using the technology for good, but
it is very difficult to get as much press and attention for those types of
positive stories.

For example, my own company, ([http://www.kairos.com](http://www.kairos.com))
is working with some very brave and passionate guys at HelpingFaceless.com in
India who are using facial recognition to help combat the growing problem of
child trafficking and enslavement in India.

See more about their story here: [http://social.yourstory.com/2014/07/helping-
faceless/](http://social.yourstory.com/2014/07/helping-faceless/)

~~~
sterlingcrispin
That's not what I did.

I used a genetic algorithm with facial detection/recognition algorithms as
fitness functions

Recognition algorithms like eigenfaces, fischerfaces will reproduce one
individual.

More general detection algorithms like YEF realtime object detection as a
fitness function will result in more general faces which represent some of the
learned features from the LFW database. I'm not just combining images, there's
lots of machine learning involved.

Recheck my site in two weeks or so I'll be posting my MS thesis about the
subject with more technical details.

(Wrote this on my phone excuse the brevity)

~~~
michael_storm
You might want to summarize that at the top of your post, then, if you're
going to submit it to HN. When it contains such nuggets as:

 _These masks are shadows of human beings as seen by the minds-eye of the
machine-organism. These DATA-MASKS are animistic deities brought out of the
algorithmic-spirit-world of the machine and into our material world, ready to
tell us their secrets, or warn us of what’s to come._

It's not surprising that HN viewers are confused about what you actually did.

In any case, seems like a cool project, now that I know what it is :-).

~~~
sterlingcrispin
Thanks, but btw someone else submitted this to HN I hadn't planned on fully
publishing the project until a few weeks from now when my thesis is due but it
was posted to medium.com so I quickly put some documentation together

There are more technical details at the bottom of that page in the form of
diagrams

~~~
michael_storm
For some reason, I thought that green usernames (like yours) meant you were
the poster. Sorry! My mistake; you can't control when someone else posts your
work.

~~~
garrettgrimsley
Green usernames indicate new, or "green," accounts.

------
drpgq
As a researcher at a face recognition company, this is interesting, although
the fact that algorithms change pretty frequently makes me think the work is
quite ephemeral. Also I'm wondering if he's heard of liveness detection.

------
lbotos
Reminds me of this:

[http://www.zachblas.info/projects/facial-weaponization-
suite...](http://www.zachblas.info/projects/facial-weaponization-suite/)

Which is also pretty interesting.

~~~
sterlingcrispin
His project is pure poetics/politics and has no direct relationship to
computer vision

Its as much anti facial recognition as a paper bag

------
bane
This is really cool. This work, along with the previous work of reverse
engineering handwritting recognizers is important for us to better understand
what our tools are actually doing. A lot of what happens inside these ML
algorithms is not terribly well understood, even by the creators, this work
gives us a really good insight into what the mind of a machine is actually
thinking. When I was a teacher, I used to try to make sure students understood
things going both ways. This is very important work in this respect.

For people struggling to understand, it's a little like having a tool that
uses automatically generated regular expressions to find things in text,
except you don't know what regex the computer is actually using. For my name
it could be as simply as "bane" or something crazy that a human would never
think up, say b[a-z]+e. It matches my name, but it's interesting to explore
what else it matches: "barserphursnatche" is also perfectly acceptable.
Remember, you don't _know_ what the computer is using to recognize things, so
you have to simply probe: randomly generate strings, use some kind of genetic
algorithm etc.

In the end, you probably won't find b[a-z]+e, but you'll find some expression
with a sizable intersection like b[a-y][b-z]*e. Once you have that you can use
the expression as a generator and build a list of all the possible names it
will match with something like Microsoft's Rex [1]

What we find is that lots of the stuff the computer likely matches are not my
name at all, or even look like a name (e.g.
"baghuadsoadshasdguoasdughasdgaodsguafdghaufogafuhgafduoe") and it gives us
insight into what it's doing.

I think why I like this work is that it makes me think of a sci-fi short
story:

\- A boy encounters an alien civilization

\- The boy befriends a robot from the civilization

\- They go off on some adventures together

\- During a scene of doubt in their relationship, during a stressful final
adventure, the boy asks the robot what he thinks of him. The robot 3d-prints
off one of these heads and gives it to the boy.

1 - [http://research.microsoft.com/en-
us/downloads/7f1d87be-f6d9-...](http://research.microsoft.com/en-
us/downloads/7f1d87be-f6d9-495d-a699-f12599cea030/)

------
drcomputer
> Computer systems built to represent human identities have contained with
> them many ontological assumptions about what it is to be an individual and
> what personal identity is. These systems define the human as a “what” ie:
> that which can be measured, not as a “who” ie: our inner self.

> If the state of the art in computer science can produce a unique feature
> that describes an individual as such, what good does that do the individual
> if this knowledge is only leveraged against them?

> If private citizens personal information, social graphs, and communications
> are being analyzed then the results should be made available to said persons
> to empower rather than enslave them. This attitude has become popular in
> personal fitness but not in communications, biometric identity, or social
> networks.

> “We see the world, not as it is, but as we are” - Talmud

I love this. Really fantastic work.

~~~
sterlingcrispin
Thank you please recheck my site in a few weeks for my full thesis

------
MisterNegative
This does pose the question whether these mask work in real life. I bet their
software uses the fact that they can control the lightning and picture
quality. Creating a direct copy of a face seems easier for real life purposes
anyway.

Other than demonstrating the flaws of current facial recognition algorithms
there does not seem to be any use for this.

Anyway, pretty cool use of evolutionary algorithms and nice pictures.

------
userbinator
I was hoping that an article about "reverse engineering facial recognition,
facial detection, and image correlation techniques in order to reveal how they
represent human identity" would have at least one mention of eigenfaces:

[http://en.wikipedia.org/wiki/Eigenface](http://en.wikipedia.org/wiki/Eigenface)

~~~
sterlingcrispin
I know its not in plain sight but if you RTFA there's a diagram at the bottom
that mentions them.

[http://sterlingcrispin.com/data-
masks/map_of_attempts.png](http://sterlingcrispin.com/data-
masks/map_of_attempts.png)

------
mattxxx
Cool tangential topic:
[http://www.onformative.com/lab/googlefaces/](http://www.onformative.com/lab/googlefaces/)

...Reminds of being up at night as a kid, and seeing faces in shadows,
patterns on the wall, vvhatever

------
lucb1e
Cache:
[http://webcache.googleusercontent.com/search?q=cache:http://...](http://webcache.googleusercontent.com/search?q=cache:http://sterlingcrispin.com/data-
masks.html)

Edit: Original seems up again

------
Houshalter
Reminds me of this:
[https://www.youtube.com/watch?v=qS5HWBNvf9U](https://www.youtube.com/watch?v=qS5HWBNvf9U)

------
geekrax
Sounds interesting. But the link received "HN hug of death".

