
A new way police are surveilling: Calculating threat ‘score’ - danso
https://www.washingtonpost.com/local/public-safety/the-new-way-police-are-surveilling-you-calculating-your-threat-score/2016/01/10/e42bccac-8e15-11e5-baf4-bdf37355da0c_story.html
======
intopieces
>Fresno Police Chief Jerry Dyer said officers are often working on scant or
even inaccurate information when they respond to calls, so Beware and the Real
Time Crime Center give them a sense of what may be behind the next door.

This assumes that the information present on the Internet is accurate, a scary
thought. How difficult would it be for someone to make a profile with your
first and last name on any or all of the major social networks, painting you
as a gun nut, a recovering drug addict, or worse?

Allowing this system to inform police work without the least bit of public
oversight is irresponsible. But that won't become a concern, I suspect, until
something unfortunate happens. For now, the only stories we have about this
technology's use point to the 'bad guys' being caught.

~~~
wfo
I think the idea of someone fabricating legal-but-indicitave online
information about you so that you turn red in the police's system and then
setting you up to be considered a threat in an incident where the police
assume you are dangerous but you are not and are goaded into shooting you is
kind of a stretch. Maybe a good plot for a movie.

But the program itself is somewhat troubling. But right now we have a
profiling system the police use. It's 'dark skin -> almost certainly has a
gun' instead of 'BEWARE says green BEWARE says red'. Is looking at public info
with an explicit review-able algorithm better or worse than letting officers
profile naturally?

I'd agree using data like this for police purposes can be fraught with issues
and is troubling but even as a privacy advocate this system seems at the very
least not worse than what we currently do. Not because it is good, but because
what we currently do is so bad.

~~~
Lawtonfogle
The notion of people seeing a comment they thought was sexist and hounding
your employer to ensure you were fired seems kinda far fetch as well. The
issue is that there are enough people out there that even far fetch scenarios
will happen from time to time.

~~~
newman314
And it only seems far fetched until it happens to you. Then it's front and
center.

------
cgearhart
The scariest part is that this kind of program drops all pretense and
explicitly characterizes all interactions with the authorities in the context
of the "threat" each of us poses. It anchors the discussion in terms
inherently antagonistic relationship, and normalizes aggressive policing
behavior. _Why aren 't more people scared by this?_

Although it is inconvenient for law enforcement, the government should not
conduct persistent, universal surveillance on the citizenry - even if it only
consists of monitoring & compiling publicly available information. The value
of these tools is dubious, and the potential for abuse is high; there is no
accountability for the costs in comparison to the utility, and they are
routinely established in the absence of oversight _by design_.

Mechanisms already exist for targeted surveillance when authorities have
probable cause. Normalizing casual surveillance with no specific intent
marginalizes the discussion of more invasive surveillance techniques, and
emboldens authorities who dismiss privacy as a trivial impediment to security.

~~~
dmix
Thought experiment: is this very different from police targeting high-risk
neighbourhoods?

Police have been over-policing black and poor neighbourhoods for decades but
no one is protesting that. There will certainly be a correlation between
"interactions with the authorities" <-> where police spend their time
patrolling, skewing the data.

The only time I've ever been stopped and interrogated randomly, simply for
walking down the sidewalk with a hoodie on (in winter) in a bad neighbourhood,
was visiting a friend in a housing project. Asking where I was going and who
am I meeting.

All of his family and friends get harassed by police living there, despite
having zero background of criminality. And in Toronto they record your
name/location each time you get stopped and carded in an "anti-gang" database.

~~~
pjc50
_Police have been over-policing black and poor neighbourhoods for decades but
no one is protesting that._

Er, apart from all those Black Lives Matter protesters that keep getting in
the news?

~~~
dmix
Apologies I wasn't aware over-policing black neighbourhoods was one of their
issues.

It seems to be rare that this issue specifically gets raised. Usually the
media focuses on the conduct on officers upon an altercation rather than how
often police are engaging with black people needlessly.

As the parent comment pointed out, the use of counting the amount of
altercations as crime data is a worrying point. Amplifying the side-effects of
over-policing certain areas.

------
hackuser
A few considerations:

1) If these are good ideas and there is nothing wrong with them, than why does
law enforcement keep them secret? Why don't they advertise to citizens: 'We
track everything you do, including who you call, where you go, what you say
online, and then we predict if you are a threat or not and respond
appropriately. This makes you and the community safer!'

2) Does being black or Muslim, having far-left or far-right political views,
increase or decrease your threat score?

3) Law enforcement has a very well-established history of using their power
not for public safety, but to abuse and oppress vulnerable classes of people
and people politically unpopular with the state (e.g., peace protesters). Just
think of all the shootings, arrests, COINTELPRO, etc. etc. Why would this
power be used any differently?

For example, many blacks already experience harassmet from law enforcement,
including such things as pulling them over for 'driving while black',
suspecting them of criminality for being in a predominently white shopping
area, arresting them for no cause, etc. What will life be like for blacks on a
police threat list? Will their life be constant harassment? Will the predicted
threat be self-fulfilling because they will be arrested inevitably? Will
corrupt police just plant something on citizens on the threat list and arrest
them to preemptively get them off the street (and then tout the accuracy and
value of the threat list)?

4) Police already are given the authority to shoot people based on suspicion
or a feeling of threat. For example, someone putting their hand in their
pocket or reaching for something in their car could be reaching for a gun.
Will being on the threat list be justification for violence against citizens?

5) Aren't we presumed innocent until proven guilty? Aren't we to be judged by
our actions and not what someone else thinks we will do? What happened to the
land of opportunity?

~~~
kragen
> Does being black or Muslim, having far-left or far-right political views,
> increase or decrease your threat score?

One of the things they explicitly mentioned was "gang associations". In my US
experience, groups of black or Hispanic* youths who sometimes do illegal
things (shoplifting, trespassing, underage drinking, pot smoking) are often
called "gangs" in cases where groups of white or Asian youths wouldn't, even
if they were doing the same things. So I think it's inevitable that this kind
of Citizenship Score will reproduce and fortify existing structures of racist
discrimination.

* This primarily applies in parts of the US where the local system of racism classifies Hispanic people as nonwhite.

~~~
sliverstorm
Since when has "gang" excluded "white"? The Chicago Outfit, Hell's Angels?

~~~
herbertldon
He didn't say 'gang' excluded 'white', did he?

He said that a group of black youths that sometimes do shady stuff will be
labelled a 'gang', while a group of white youths will not.

~~~
sliverstorm
I guess, anecdote vs. anecdote, I've just never seen that happen.

------
jMyles
Turning the profiles and prejudices of police into an explicit metric, whose
code can be examined and debated, seems like an idea that might have not have
legs in a free society.

However, surprisingly unmentioned here is the inevitability, so long as the
client is the state, that these metrics include political dissent as a threat
model. Has anyone asked Beware for comment on this?

~~~
sliverstorm
If political dissent is correlated to violence, why _shouldn 't_ it be a
parameter in the threat model?

Everyone, including police, form threat models in their own minds all the
time. Ever avoided the hoodied skinhead hiding his hands in his pockets in a
dark alley? Threat model.

The model isn't grounds for prosecution, but it is reasonable grounds to
adjust your response.

~~~
pjc50
_If political dissent is correlated to violence, why shouldn 't it be a
parameter in the threat model?_

Because we know this will be implemented in a partisan way. "Dissent" implies
that there is a _single_ politics from which everything different must be
considered suspect. Simply using the word implies totalitarianism.

You can see this in the US where the white people talking about the violent
overthrow of the government somehow aren't considered terrorists and are
relatively unmolested even if they actually occupy Federal land and buildings,
but a nonwhite kid holding a toy gun gets shot in a playground.

Edit: just to drive the point home, does the threat model include a database
of gun owners? Do we think gun ownership might be correlated with crime? Or is
it illegal to investigate that as it is for the CDC?

~~~
sliverstorm
How does "dissent" imply totalitarianism? If someone was to believe that women
should not be allowed to vote today in the USA, they would be dissenters. Does
that mean the USA is totalitarian?

As far as I can tell, they're relatively unmolested because:

[http://imgur.com/AYNRxlo](http://imgur.com/AYNRxlo)

------
habitue
This is interesting since, really, cops' brains are already creating "threat
scores", it's just that they aren't available to look at or tinker with how
they're calculated. It's likely these computer models are inferior to the
mental models, but making them explicit is a big step.

~~~
mordocai
It would be a big step if those models were under public scrutiny and not a
"trade secret" of the for profit company that makes it.

~~~
xiaoma
As soon as the model were public it would be useless. The first thing an
aspiring criminal or terrorist would do would be study the model and then do
whatever necessary to make his or her own threat score low.

~~~
r00fus
[https://xkcd.com/810/](https://xkcd.com/810/)

~~~
mercer
Heh. I was about to write "lower their threat score by not being gun nuts and
not committing felonies?" but it appears XKCD got there first...

------
Nadya
Ah yes, more propaganda supporting a surveillance state. What makes these
sorts of things difficult to argue about is all a matter of how they get
framed.

"It's good for a more peaceful society. You aren't violent? Right? You have
nothing to worry about then! This technology will help keep you safe. You want
to be safe? Right?"

I'm glad to see many citizens are at least sharing their voice against this.
But it reads all to much like Fresno took "big strides too fast" and if they
had taken baby-steps there would have been less outrage. You have to whittle
down people's liberties - not try and take them all at once! If you do it
slowly enough, they don't even notice.

As an aside... typo + how do they know their score? I thought only operators
knew it? How does she know her threat level was elevated because she tweeted
about "Rage"?

 _> Once council member referred to a local media report saying that a woman’s
threat level was elevated because she was tweeting about a card game titled
“Rage,” which could be a keyword in Beware’s assessment of social media._

------
jcromartie
Why shouldn't we use statistical inference to deliver information that enables
responders to be better prepared to deal with a situation? I would hate to be
a false positive, but I'd rather have the chance of being flagged as "green"
when police are headed my way, rather than always being presumed to be a "red"
as seems to be the case now. This seems like it would be the opposite of being
presumed guilty until proven innocent, for most people.

And this isn't surveillance: they aren't collecting new information, just
putting available data sources together. Unless deducing things from existing
data is somehow surveillance?

~~~
lukeschlather
The anecdote from the article strikes me as a really good example of why this
sort of statistical profiling is unnecessary. Guy threatens his girlfriend,
algorithm flags him as red, so they... call in a negotiator and successfully
de-escalate the situation. I'm struggling to see how the system flagging him
as red had anything to do with their actions. It was just a case of cool heads
doing good police work.

~~~
VLM
Cops already have a lot of contact with parole inmates who have dramatically
reduced rights. So they'll just lump "red" in with the parole inmates. You'll
hear cops saying things to each other like "he wouldn't be red unless he
deserved it". Generally, having the cops prejudiced against someone before
they even meet them is not going to turn out well.

------
VLM
Note that there's not just shoot to kill response.

They specifically mention "property records". For me, that works out pretty
well, I have enough money to afford enough property to indicate I can easily
afford legal representation and will be treated as such by the police, yes mr
vlm, no mr vlm, you are free to go mr vlm, have a nice day mr vlm. On the
other hand, people verified to be dirt poor can expect dramatically different
treatment knowing they cannot afford legal representation. On the downside I
can expect no mercy WRT ticket fees, because they'll know I can afford it. You
can envision traffic cameras lighting up when I drive thru a speed trap
village, between my age, clean criminal record, and property ownership they'll
know they can collect quite a toll off me at zero personal risk.

Aside from property, anyone who's ever posted anything vaguely drug or alcohol
related on the internet, ever, now has perpetual probable cause for a
breathalyzer test or a drug test down at the hospital. This is perfect for
police harassment.

------
jrcii
They must have read this article [https://www.aclu.org/blog/free-
future/chinas-nightmarish-cit...](https://www.aclu.org/blog/free-
future/chinas-nightmarish-citizen-scores-are-warning-americans) and misread
"warning" as "inspiration"

------
roflchoppa
It appears that they are only using it as a caution to ensure that the
responding officers have the resources so that the public, and the offices are
not in harm. Now as far as the aggregation of publicly available posts to the
internet, as well as corresponding that to private-state information (previous
records / offences). Its nice that SAAS today can be updated on the fly to
ensure that the threat scores can become more refined, so in the example of
the Yellow House score for the council member, might have been avoided....
unless hes actually a yellow based on his web traffic.....

I wonder if they apply this to pulling people over as well. Think, is the car
reported stolen (alert: RED), if not then the highest color for the house
hold.

But lets also keep in mind that Fresno Ca, is not Palo Alto, Ca.

------
fapjacks
So... Here's my thoughts on it. At first blush _how dare they_! But as I let
this stew for a while and turned it over a few times, I'm arriving at a
different conclusion, helped along by some of the insightful comments here.

Since the police _really are_ doing this kind of thing in their heads anyways,
what I see here is an opportunity for accountability. Did the man in body
armor with the gun consult the magic number before he shot the kid on the
playground with a toy gun? Or did he just get excited? Like body cams, I think
this is actually something that could be used in favor of a more responsible
police force than not.

------
ErikVandeWater
Technology moves too fast for the law. _Even if_ police believe using a
particular technology is unconstitutional, they are likely to use it anyway,
because it will likely take years for a challenge to be brought to court, and
definitely years to get to the highest (influential) courts. Even after all
this, the police will only have to stop the unconstitutional activity, and
immunity will handle the rest.

States need to enact laws that will prevent police departments from using any
new software without prior approval of their constituents (e.g. city council
approval).

------
iamleppert
You know I hate to admit it, but as techies we are responsible for most of
this power that the police have. We have created all this technology, and then
some business has gone into local police departments and pitched the idea.

Kind of sad to realize there are people in our industry who don't mind doing
bad things with their talents or are at best indifferent to what their company
is doing. It's the "I just work here" attitude so you don't have to care if
the company has an actual business model, helps people, creates a good
product, or is even doing something ethical or not.

------
philfrasty
Reminds me exactly of the intro from „Person of interest“
[https://www.youtube.com/watch?v=WOnQ8CD3v4g](https://www.youtube.com/watch?v=WOnQ8CD3v4g)

~~~
wj
It also is the central plot point to the anime Psycho Pass.

(off-topic but the watch in Psycho Pass was what I hoped the Apple Watch would
be)

------
newman314
I'll also point out that this is inherently unfair fron the standpoint using a
person's name as the primary key for lookup. Given the roaring success that is
the no fly list (/s), people's names are not evenly distributed, there are a
lot more John Smiths than say Larry Ellison. It should not mean one person is
easier to identify because of their name. If that's the case, there should be
an even name distribution to give everyone a fair shake. Let's call everyone
Bob!

------
hackuser
Also, police in at least some cities use systems that predict whether someone
will become a criminal and give that person extra attention.

Here's one story on it that I found with a quick search:

[http://www.nytimes.com/2015/09/25/us/police-program-aims-
to-...](http://www.nytimes.com/2015/09/25/us/police-program-aims-to-pinpoint-
those-most-likely-to-commit-crimes.html)

~~~
ryandrake
Wow, they're even calling it "Predictive Policing". Here's my prediction: By
2020, we will have fully automated arrest warrants being served and executed
based solely on big data analysis, and the legality of that practice will be
working its way through the courts.

~~~
oxryly1
Precog Unit.

------
kevinSuttle
The scary thing is, this is pretty much the plot for Captain America: The
Winter Soldier.
[http://www.imdb.com/title/tt1843866/synopsis?ref_=tt_stry_pl](http://www.imdb.com/title/tt1843866/synopsis?ref_=tt_stry_pl)

------
ck2
Oh wow, they probably just gave equifax a truly horrible idea.

I wonder if this is even legal since it creates prejudice, probably is through
some loophole.

Did you ever protest a war? Ah you have a terrible threat score.

Next they open it up to employers.

------
joshbuddy
Imagine that this is just employing machine learning, where the training set
is police reports filled out after the fact. Is this just big data being
applied to threat prediction?

------
SN76477
Reminds me of the Sesame Score that was being talked about so much a few weeks
ago.

The Mark of the Beast looks more and more real every day it seems.

------
leke
It's like a high-tech version of profiling.

~~~
griffinmb
It reminds me of Psycho-Pass.

"Psycho-Pass is set in 2113. The Sibyl System (シビュラシステム Shibyura Shisutemu?)
is actively measuring the populace's mental states, personalities, and the
probability that individuals will commit crimes, using a "cymatic scan" of the
brain. The resulting assessment is called a Psycho-Pass (サイコパス Saikopasu?).
When the probability of a person engaging in crimes measured by the Crime
Coefficient (犯罪係数 Hanzaikeisū?) index exceeds a certain level in an
individual, he or she is pursued, apprehended, and killed if necessary."

Taken from: [https://en.wikipedia.org/wiki/Psycho-
Pass](https://en.wikipedia.org/wiki/Psycho-Pass)

~~~
Paul_S
As plausible as Minority Report. Even if possible it would not be implemented
because of how the majority of people think of human free will.

------
elevation
"The program scoured billions of data points, including arrest reports,
property records, commercial databases, deep Web searches and the man’s
social- media postings."

Does "deep Web searches" mean the program has access to your tor activity? How
is that information acquired?

~~~
kragen
No, the "deep Web" is things like friends-of-friends Facebook posts, Twitter
posts to locked accounts, and friends-locked LiveJournal posts — things that
are in some sense "on the web" but that a standard search engine can't find.
You're thinking of the "dark Web", which includes things like onion sites.
Your confusion is not unusual;
[https://en.wikipedia.org/wiki/Dark_web](https://en.wikipedia.org/wiki/Dark_web)
says, "The dark web has often been confused with the deep web, the parts of
the web not searched by search engines. This confusion dates to at least
2009."

