[I focus only on the performance of facial recognition system, not on whether it's good idea to have such systems in place.]
From the systems performance point of view, this is non-story. If you deploy detection system, you have to make trade-off between precision (how many false positives you will have) and recall (how many false negatives -- misses) you will have [1]. How you do this trade-off depends on cost of false positive, and cost of false negative.
For example, if false positive means, that someone will point camera to that part of stadium and someone will manually check if it's correct [2], and then arrest, then 2000 false positives might be good performance if recall is good, because there is no negative impact on false positive, only some wasted effort by police.
On the other hand, if this AI match would be used as evidence in court, then the it would be terrible, as it would cause much more harm than good. From the article it seems that police is using it more closely to the first case, than to the second case, so the performance of this system might be good, so really the article is non-story.
[1] I simplified the "precision" and "recall".
[2] Or send more security guards / police to that area, or whatever discrete measure.
“because there is no negative impact on false positive, only some wasted effort by police.”
This is incorrect. You don’t need to be “arrested” to be detained on a false positive hit, and being detained can ruin lives, missed work, missed family obligations, social stigma of having been a suspect. Also these systems do not have a way to correct false positives, so it’s likely to reoccur to the same individual multiple times which will have a real psychological impact. These systems have also shown to have higher false positive rate when looking at minority’s this creates a risk of further creating a cognitive bias among police.
Don’t forget being detained as a suspect could result in you being added to a list you can’t remove yourself from.
That was under condition "if false positive means, that someone will point camera to that part of stadium and someone will manually check if it's correct [2]". That means that false positive does not know that it's false positive.
> From the article it seems that police is using it more closely to the first case, than to the second case, so the performance of this system might be good, so really the article is non-story
I will quote Bruce Schneier on CCTV cameras
> But the question really isn't whether cameras reduce crime; the question is whether they're worth it. And given their cost (£500 m in the past 10 years), their limited effectiveness, the potential for abuse (spying on naked women in their own homes, sharing nude images, selling best-of videos, and even spying on national politicians) and their Orwellian effects on privacy and civil liberties, most of the time they're not. The funds spent on CCTV cameras would be far better spent on hiring experienced police officers.
I was going to state that I had read from a differently biased news source that nobody had been arrested, but the telegraph hasn't really reported it any differently to any other news outlet and contains this:
> It also said no-one had been arrested after an incorrect match.
Perhaps the real story here is that a system correctly identified 173 correct matches and that the police force clearly double-checked. That almost seems like an appropriate use of the technology.
Now, there is highly likely to be a desire to improve the precision. I think this may be where the problems arise. Let's say the system only had false-positive rate of 10%. An officer arresting people based on the information is likely going to become complacent and assume that the system is much more likely to be correct than wrong. It gets worse the better the false-positive rate gets.
Where my deep concerns are with such systems is that there is going to be a very strong temptation for the providers of such technology to store the false positives -- by definition not the people you thought they were and who will unlikely have given their consent -- in order to improve the precision and therefore their product. As there is effectively a wholesale opt-out of the GDPR for activities of the state at the cost of passing yet another ill-conceived law (to provide a lawful purpose or legal requirement) then there is no real reason why this won't be done.
Such behaviour will lead to companies which effectively become privatised portions of the state with no viable competition -- their data is their secret that keeps them the only viable choice, and when they have precision at 95% and as a competitor you can only dream of having precision of 25%, then it seems somewhat unlikely that anybody would choose to use the other system at any price. As a competitor you will not have the lawful purpose unless working with a police force, which is unlikely with precision at 25%.
As long as the data is used as one of many guideline sources which is scrutinised by human officers before the individuals are suspected and acted upon, I personally don't see a problem with the inaccuracy (which will get better over time).
> As long as the data is used as one of many guideline sources which is scrutinised by human officers
What makes you think that's going to happen? If anything, the trend is in the opposite direction - reducing the need for humans within automated systems as much as possible.
If a fast food store manager brings in robots to serve people food, do you think he'll still want to hire people around to make sure the robots are serving the right food to people? No, if he brought in robots, then it's because he wants to get rid of as many human employees as possible.
Maybe the humans will be there for like the first month or two while the robots are being tested, but that's it - or at least that would be the goal. Look how fast Musk wanted to get rid of human employees at Tesla, too. Complete 100% automation will be the goal for both companies and governments.
Even the Pentagon is rushing to allow its AI drones to kill "targets" on their own (thanks, Google). You don't think governments will rush to get humans out of the equation for more "trivial" stuff like arresting people?
If 20% of the people are wrongfully accused, but the system catches 99.9% of the actual criminals, then they will be very happy with that system. Just like Facebook is happy to censor 10%, 20%, 40% other innocent content, as long as it can brag to Congress that it "censored 99% of the terrorist content" (which probably isn't true, either). They don't care about the collateral damage.
Some programs are designed to replace humans, others are designed as tools. Automation does not always equal the replacement of humans.
I appreciate your analogies but there is not evidence in the article to suggests that police and investigators are being replaced by automated systems.
Data such as this can be used to eliminate a lot of the investigative work which usually goes into law enforcement.
Are you familiar with a police lineup? Police detain a selection of suspects based on various characteristics and ask witnesses to select the perpetrator, who may or may not be present at the lineup.
Neither a police lineup nor facial recognition are accurate or reliable when used alone, but when used in combination with other facts, tools, and resources an arrest can be made with increased speed and confidence.
> If 20% of the people are wrongfully accused, but the system catches 99.9% of the actual criminals, then they will be very happy with that system.
The opposite (presumption of innocence, preferring having some criminals leave free than a single innocent person wrongly imprisonned, etc) has been the norm in all occidental democracies for the last two centuries or so. What makes you think it could change?
While the article does go into the detail of the accuracy of the deployed systems, it's both sad and worrying that the counter arguments to mass facial recognition are reduced to a single positioning quote.
The algorithm isn’t arresting people. It’s cutting down the suspect pool from 17000 to 2500, the fact that 2000 of those are innoscent while 500 are criminals isn’t a damning issue, it’s just a point where it could improve. Without the system the 500 people would not have been identified at all.
That assumes that both numbers are for the same period and that each arrest correspond to 1 match, which most likely will not be be true. For example if system identified only subset of people, but when you make arrest you arrest all. That means that now you have more arrest attributed to system, than number of matches.
Since article does not do any attempt to answer these discrepancies, there is high chance the devil is in the details of definitions of those numbers.
Why do you consider 170000 law abiding citizens as potential suspects? That's an extremely paranoid approach to law enforcement, especially when according to basic criminal law, these citizens are supposed to be considered innocent by default.
Mass automatic surveillance of this reverses the idea of innocent until proven guilty without actually aknowleding it. That's scary
From the systems performance point of view, this is non-story. If you deploy detection system, you have to make trade-off between precision (how many false positives you will have) and recall (how many false negatives -- misses) you will have [1]. How you do this trade-off depends on cost of false positive, and cost of false negative.
For example, if false positive means, that someone will point camera to that part of stadium and someone will manually check if it's correct [2], and then arrest, then 2000 false positives might be good performance if recall is good, because there is no negative impact on false positive, only some wasted effort by police.
On the other hand, if this AI match would be used as evidence in court, then the it would be terrible, as it would cause much more harm than good. From the article it seems that police is using it more closely to the first case, than to the second case, so the performance of this system might be good, so really the article is non-story.
[1] I simplified the "precision" and "recall". [2] Or send more security guards / police to that area, or whatever discrete measure.