

Facial Recognition Failed in Boston Bombing - jhonovich
http://ipvm.com/updates/1993

======
ck2
Not just facial recognition, the "terror watch list" failed (after a warning
from Russia about the older brother) lack of background checks on the AR15s
they owned failed, the secured perimeter failed (the teenager was outside of
it) the city lockdown failed (the guy smoking the cigarette outside when he
saw the blood was technically violating the lockdown).

In the end it was mostly dumb luck, dumb criminals, and the fact the poor
victim who lost their legs managed to survive. In fact _that_ was the success,
that more wounded people didn't die thanks to fantastic work by the hospital
and doctors.

Instead of patting law enforcement on their backs, how about thanking the
doctors, where is their applause?

~~~
kevincennis
It seems disingenuous to say that facial recognition failed. As far as I can
tell, all the images we had of these suspects were either very low-resolution,
taken from angles, or both. Seems plausible that law enforcement didn't have
any good candidate images from which to run a facial recognition search.

> In the end it was mostly dumb luck

What? Painstaking police work went in to getting images of these suspects, and
the release of those photos are almost certainly what led to Thursday's events
and, ultimately, the killing and capture of the suspects.

> the city lockdown failed (the guy smoking the cigarette outside when he saw
> the blood was technically violating the lockdown)

Firstly, the lockdown is what forced suspect #2 to hide out in a boat instead
of fleeing the area. Secondly, and less importantly, the guy smoking a
cigarette wasn't violating the lockdown. It had ended prior to him going
outside.

Was there luck involved? Of course. But there was also tremendous work done by
a number of law enforcement agencies - without which, the suspects may never
have been killed and captured.

~~~
ck2
The brothers didn't leave the area, didn't go into hiding and were able to
kill a campus officer and hijack a car, all without police finding them for a
week.

It was mostly dumb luck. It's not like they were hiding!

Oh, and [http://boston.cbslocal.com/2013/03/27/hanover-police-
trying-...](http://boston.cbslocal.com/2013/03/27/hanover-police-trying-to-
solve-home-homemade-bomb-mystery/)

~~~
kevincennis
They didn't go into hiding or leave because they didn't think they needed to.
It seems pretty clear they they didn't want to be caught.

From what I can tell, numerous people called in after the photos were
released. Even if Thursday's events hadn't taken place, the police would have
known the identities of both suspects shortly thereafter.

It just seems crazy to me not to acknowledge the role of investigators in all
of this and chock it up to luck.

As for the edit you just made about the bombs in Hanover: are you implying
there's a link to the Marathon bombers? Seems like conspiracy theory reaching
- unless there's some sort of evidence you'd like to share.

~~~
sliverstorm
_It just seems crazy to me not to acknowledge the role of investigators in all
of this and chock it up to luck_

People do crazy things when they have made up their minds before the story
even begins.

------
gcr
The problem is that any non-perfect automated system will wrongly convict so
many civilians as being terrorists that nobody will trust it anymore.

Let's say that you want to catch terrorists creeping around in your airport.
To do this, you have a magic camera that, when it takes a picture of a
civilian, compares it to a picture of a known terrorist. If they're in fact
the same person, it will flag security over. Let's say that it is indeed a
magic camera and makes the correct decision 99.99% of the time, which is far
better than the current state of the art[1].

Do you have any idea how much DHS or DoD would pay for such a magic camera?

But it won't be enough.

Consider a watch list of 100 terrorists that we're looking for in our airport.
If our camera compares each civilian's picture to all 100 terrorists in
succession, and if we model each comparison as a Bernoulli trial, then each
civilian has a 0.9999^100 = 99.004% chance of getting through security
unfazed, which means we flag innocent people as terrorists about 1% of the
time.

This doesn't work in an airport setting. Assuming 5,000 people pass through
your small airport per hour and you have a 14-hour day, that means your
security people have to go through 0.01 * 5000 * 14 = 700 false alarms each
day. That's about 1.2 false alarms per minute. Your security people will have
to go through all their checks each time this happens. Before long, they're
going to stop trusting your magic camera.

As you scale your watch list up, it gets worse. How well would it work for
1,000 people on the watch list? 10,000? Note that the current US watch list
has 50,000 people on it.

Even if it were a perfect magic camera, it coudln't catch terrorists on their
maiden crime who aren't on the watch list at all.

(Most of this post was adapted from:
<http://vast.uccs.edu/~tboult/tmp/IJCB11-tutorial-boult.ppt> , which is a
really interesting presentation if you're interested in these sorts of
things.)

1: <http://vis-www.cs.umass.edu/lfw/results.html>

~~~
stretchwithme
Do you know how unreliably people recognize strangers? Many people are already
wrongly convicted on such evidence.

Even perfect automated facial recognition won't convict people. It will merely
point out people that should be looked at. If a jury looking at the image and
the guy's face think the system is wrong, they can disregard it.

As far as prevention goes, what we need are systems that can detect deception
reliably, which will help if we ever decide to ask people meaningful
questions. Not to convict, but to focus attention on a possible threat.

~~~
gcr
Sure, I should have picked a better word than "Convict", especially since I
didn't mean it in a legal sense. Given the context, "flag" or "decide" or
"indicate" would have worked better. I only wanted to convey that the system
thinks the probe and gallery pictures are the same person

------
jhonovich
I am the author of the referenced article. The problem is matching
surveillance camera images (like from Lord & Taylor's) is impossible because
of the steep down tilt angle of the cameras plus their low resolution.
Smartphone images could be a little better as the angle is more head on and
the resolution is higher. However, even those images tended to be in a wide
field of view which means low pixel density, hurting matching performance.
Last, but not least, the caps and sunglasses hurt recognition as well.

~~~
melling
Why is surveillance video not HD? Shouldn't this quality be almost a commodity
by now? How different are these cameras than what you get in, say the latest
Samsung Galaxy S4, for example:

<http://en.wikipedia.org/wiki/Samsung_Galaxy_S4>

~~~
tripzilch
Probably because the cost of surveillance cameras is more than just the cost
of the resolution or storage. the ones that hang outside need to be weather
proof and vandalism proof. they were probably bought as part of a large and
expensive system, intended to be paid off over 10-15 years[0], to replace all
those cameras whenever more resolution becomes available, is probably quite
expensive, regardless of how cheap the actual camera is. It's a _lot_ of
cameras, in a lot of hard-to-reach places, with expensive reinforced casings,
and would the output format of the cameras be compatible with the old system?

OTOH, I _do_ actually see more and more HD (or _relatively_ high def)
surveillance footage. It all depends on how when the security system was
installed, I guess. Also smaller businesses can afford a shorter turnaround
for these things, as well as not having the handicap of a head-start[1].

[0] actually I'm guessing those lifetime spans, it seems a bit much? but I
also can't see anyone signing off on a security system that's expected to be
replaced in 5 years. good thing my job isn't facility management ... :P

[1]
[http://en.wikipedia.org/wiki/Law_of_the_handicap_of_a_head_s...](http://en.wikipedia.org/wiki/Law_of_the_handicap_of_a_head_start)
, how odd, is there no common English phrase for this law?

~~~
nekojima
In business terminology, a common English phrase would be "first-mover
disadvantage".

------
marknutter
How hard would it be to throw off facial recognition by simply wearing very
high quality makeup and prosthetics to alter your facial appearance (ala
hollywood movies)? If I were the boston bombers, I would have first learned
how to make myself look completely different. It's unbelievable to me that
these guys didn't realize they'd be captured on tons of cameras that day.

~~~
nwh
If you wore Mission Impossible style masks, there's no reason why the facial
recognition would be able to pick you up from underneath. You might alert the
suspicion of people around you more though, as they don't ever seem to be
perfect.

~~~
runarb
What if you wore something like this: <http://www.spfxmasks.com/ourmasks.php>
?

One should be able to blend in, and make video surveillance pretty useless...

Edit: Did work for this robbers:
[http://www.dailymail.co.uk/news/article-2192115/Robbers-
disg...](http://www.dailymail.co.uk/news/article-2192115/Robbers-disguised-
white-cops-caught--send-polite-thank-letter-company-unbelievable-latex-
masks.html)

------
aashaykumar92
When the FBI released the photos of the two suspects, I kept wondering, "Why
do they need the publics help when I'm sure there is facial recognition
software to help match the suspects' faces?" Well I guess this article
somewhat answers this, but I am still in disbelief that our government does
not have better software.

For how long it took to discover their names, the two suspects had TOO much
time to escape. They had about 48 hours to book flights out of the country,
flee to another part of the country, do more damage, etc. The lack of facial
recognition software could have ultimately led to this case being
unnecessarily longer. This is an issue that CAN and NEEDS to be fixed quickly
--in fact, I hope the government is working on a more sophisticated facial
recognition software right now. Ideally, the facial recognition software
should be able to crawl through the web to search for faces.

Now that I think about it, a good hacker could build a search engine that
allows people to post photos and the program will run an automated search
across the web (Social networks, Google, etc.) to see which pictures match it.
Hell, Facebook even has facial recognition to some extent. A more
sophisticated one cannot be that difficult for the government to build and put
to use.

~~~
sp332
Facebook only has to distinguish among friends of the uploader. And it's not
perfect. This man was wearing sunglasses and a baseball cap. He would not have
been identifiable out of the tens of thousands of people at the event, matched
across the (probably) millions of people in the database.

~~~
sliverstorm
Plus Facebook gets high resolution pictures _of people's faces_ , and asks for
confirmation. It doesn't get it right every time.

------
jere
I watched this lecture recently <http://youtu.be/-IUe3ce_29I> which talked
about how hard it is to do image recognition at low resolution. One solution
is a "fovea" system that scans an area for interesting items and zooms in on
them with a high resolution camera automatically to get better recognition.

I wonder if this would help in similar instances. A system that uses facial
detection to isolate faces and then zoom in on them very quickly seems
feasible. I'm assuming it would be overwhelmed in a scenario like a marathon
of course, but perhaps it would help in other situations.

------
bsenftner
My big questions is why were the suspects faces in the database? Why would a
19 year old, never done anything face be in the facial recognition database?

~~~
DannyBee
Best random guess: Something related to him being an alien and becoming a
naturalized citizen

~~~
DannyBee
Actually, figured it out:

Dzhokhar had a Massachusetts driver’s license, and those pictures go into a
facial recognition database.

Source: [http://www.washingtonpost.com/world/national-
security/inside...](http://www.washingtonpost.com/world/national-
security/inside-the-investigation-of-the-boston-marathon-
bombing/2013/04/20/19d8c322-a8ff-11e2-b029-8fb7e977ef71_print.html)

------
astar
Facial recognition is still treated like some magical technology, like the
"zoom in, enhance" feature much mocked about on TV crime shows. I bet
Facebook's technology is ahead of law enforcement's in terms of data,
algorithm, and refinement, and yet it still stumbles on pretty obvious faces
(if you have the default auto-tag feature left on)

~~~
anigbrowl
_"zoom in, enhance" feature much mocked about on TV crime shows_

Actually, those jokes refer to a scene in _Blade Runner_ :
<https://www.youtube.com/watch?v=qHepKd38pr0>

~~~
socillion
NTSF:SD:SUV Parody of the 'enhance' trope in cop drama setting:

[https://www.youtube.com/watch?v=xbCWYm7B_B4&t=58s](https://www.youtube.com/watch?v=xbCWYm7B_B4&t=58s)

------
jpalomaki
I can understand that recording live footage with very high quality is a
problem, but how about capturing quality stills to complement the video with
lower quality.

~~~
emiliobumachar
Someone with knowledge of cameras please either correct or endorse this, but I
think the largest price/quality tradeoff in any camera happens at the lens
choice, and given a cheap lens, the stills can't be much better than the
video.

~~~
brk
Not exactly.

A lot of it has to do with the fact that most security cameras are not setup
for capture of facial details (or license plates for that matter).

Most security cameras are setup for general overview shots, with around
10-20ppf (pixels per foot), whereas good detail starts at around 80ppf
(meaning that you'd need 4x as many cameras in most areas to start to get
something approaching an image quality that you could use for _reliable_
facial recognition.

On top of that there is the storage factor. A "common" camera in a new
deployment is a 1080p (2.1 Megapixel) camera. If this is set to record good
quality images 24/7, it consumes about 60GB of storage per day. Various tricks
like "record on motion", etc. can reduce this to maybe 10GB/day, at the risk
of potentially missing key evidence or having gaps in the recording.

Lenses _are_ important, but it's not that hard to find a lens with sufficient
resolution for these cameras. A good lens rated at 2+ megapixels is around
$70-$120 (depending on various factors), and a good outdoor camera (including
lens) can be had for $500-$1200. Big price ranges, yes, but not to the point
that the lenses are the limiting factor budget-wise.

------
mikecane
How to camouflage yourself from facial recognition technology
[http://venturebeat.com/2010/07/02/facial-recognition-
camoufl...](http://venturebeat.com/2010/07/02/facial-recognition-camouflage/)

~~~
anigbrowl
That's not conspicuous at all. And like zebras, there's no benefit unless
everyone else is doing it.

~~~
okamiueru
Dazzle camouflage isn't meant to hide you in your surroundings, but rather
make it more difficult to assess the geometry -- how large you are, where you
are heading, etc.

It's more for confusing the lion running after the zebra, than an attempt hide
from the lion.

------
eksith
Bottom line is that it's nearly impossible to exactly match facial features to
anyone beyond a set distance when there's too much ambiguity between distances
and rubbish quality CCTV video.

Matching gait may be a more effective means of identification at a distance
(and, in this and many other cases, at odd angles).

<http://www.am.sanken.osaka-u.ac.jp/~sagawa/pdf/accv06.pdf> (PDF)

------
mrkmcknz
Does anyone have detailed knowledge of startups in this space?

I know of Lambda labs but I personally haven't use their solution post
Face.com.

~~~
brk
There are a few, though less than you might think. Lots of general progress
has been made in terms of facial detection algorithms, but there are almost no
"general" security camera deployments that are setup to capture sufficient
detail to get a good face match.

There are not enough events like this in open places that drive requirements
for facial recognition use. Limited cases (airports, casinos, etc.) more
frequently have cameras setup to get good straight-on high-res facial shots,
but almost no typical outdoor area is covered that way.

------
WestCoastJustin
Why did it fail? That, to me, is the most interesting unanswered question.

~~~
new_test
Because facial recognition is a very difficult problem.

~~~
sliverstorm
Surely all you need to do is start an agile startup and throw some Ruby on
Rails at the problem?

