
China using big data to detain people before crime is committed - walterbell
https://www.theglobeandmail.com/news/world/china-using-big-data-to-detain-people-in-re-education-before-crime-committed-report/article38126551/
======
anonytrary
For those of you who find this interesting and also enjoy anime, Psycho-Pass
explores this premise.

[https://en.wikipedia.org/wiki/Psycho-
Pass](https://en.wikipedia.org/wiki/Psycho-Pass)

TLDR:

> The story takes place in an authoritarian future dystopia, where omnipresent
> public sensors continuously scan the mental states of every passing citizen.
> Collected data on both present mentality and aggregated personality data is
> used to gauge the probability of that citizen committing a crime, the rating
> referred to as that citizen's Psycho-Pass.

~~~
Noos
Psycho-Pass is excellent. They also released a visual novel in the states,
Psycho-Pass: Mandatory Happiness. I also found when watching it it felt like
it was decently inspired from the older anime Cyber City Oedo 808, which had
the plot of the criminal enforcers hunting others.

~~~
krapp
Shame they never made a second season.

~~~
coldacid
:^)

------
ejlangev
It's a shame we moved a lot of the US industrial base to an authoritarian and
potentially hostile country like China. A bit scary to think of the generation
or two of people who won't have the knowledge to manufacture things should
they ever need to because of conflict in some form. Seems like it reduces any
leverage the US has for holding China accountable for much that it does.

~~~
anthonyleecook
We need some investors to band up and recreate the hardware environment in
Shenzhen. There would be a lot of money made that way too so not sure why
there wouldn’t be one in Nevada or Arizona or something.

~~~
korethr
While I like the idea, I think there are presently cultural barriers in the
way of that succeeding, mainly the Western view of IP vs the Eastern view of
IP.

Bunnie talks about it on his blog here:
[https://www.bunniestudios.com/blog/?p=4297](https://www.bunniestudios.com/blog/?p=4297)

~~~
telmnstr
Let's be fair, they steal a ton of IP from the USA.

Isn't there large issues in USA colleges with students from China cheating
left and right?

It's much easier to copy.

------
polskibus
As described by Martin Kleppmann at the end of his great book, big data
analytics, when used without proper regulation, creates a downward spiral.
Western world enforced rules against discrimination based on sex, race, etc.,
yet everyone seems fine when it's the algorithm spotting and executing on the
biases, which statistically are fine, but nevertheless, may be wrong in many
individual cases.

~~~
smogcutter
Maciej Ceglowski on this: "machine learning is like money laundering for
bias."

[http://idlewords.com/talks/sase_panel.htm](http://idlewords.com/talks/sase_panel.htm)

~~~
ghostcluster
the problem is 'bias' is often based on a model that is statistically and
operationally accurate, but unfair from a higher order social moral framework.

a sufficiently layered net will pick up second and third order consequential
effects that are not formally protected by anti-bias laws, but end up
effectively mapping onto the sorts of people in categories the laws were
intended to protect.

that's the fundamental problem as I see it, and it's necessarily complicated
and nuanced.

------
cdepman
We’re on our way there too in the US:

[https://www.theverge.com/2018/2/27/17054740/palantir-
predict...](https://www.theverge.com/2018/2/27/17054740/palantir-predictive-
policing-tool-new-orleans-nopd)

~~~
srcmap
If someone is taking fly training of commercial jet without wanting to learn
how to land the plane, I absolutely want the government to find out why.

Same for someone who want to purchase large quality of fertilizer for no real
needs.

~~~
drb91
This is crazy. The government is not a wall between you and what scares you;
that is what politicians want you to think. The wall is never high enough; it
is never deep enough; there are never enough cameras; we need better people
deciding what is OK to do on camera and what is not OK.

This is all very fine to want, but it's an entirely different conception of
government than the one described in the constitution. It's a government much
closer to the one in Russia, or China, or Turkey, or North Korea. While it may
be used to, in part, fight crime, the line item that justifies the
surveillance state is the ability to invest in a guarantee of power.

Trying to turn the US into this state is just going to lead to abuse of power,
abuse of people without voices, and a much more wasteful budget:
infrastructure spending by investment in paranoia, with a return of more power
by fear. Literally government by terrorism.

~~~
forapurpose
> The government is not a wall between you and what scares you

While I'm sympathetic to your point, it can be taken too far. Government does
provide security to its citizens, and it does that partly by gaining prior
intelligence about suspicious activities. Some things should be investigated -
if someone posted online about committing murder, in detail and with clear
plans, I'd hope law enforcement would check it out.

------
Paul_S
Once the algorithms realise that everyone commits at least a few crimes a day
they will become 100% infallible by issuing arrest warrants for anyone with a
pulse.

~~~
corysama
I never read _Brave New World_ , but I've heard that was a theme. Everyone
used illegal drugs, but there was no enforcement. Instead, if they wanted to
bring you in, they just charged you with illegal drug use --because everyone
knows that everyone uses illegal drugs.

~~~
rotskoff
Not quite... In Brave New World there is a drug called "soma" that is an opium
for the masses; it's rationed by the government. The subplot around soma
focuses more on trying to get people to not take it so that their minds can be
freed.

~~~
sli
That's also a major (and non-spoiler, if you're familiar with films at all)
plot point of Equilibrium.

------
JumpCrisscross
Does it actually work? Or are the "criminals" just social activists?

~~~
Dangeranger
They are not "criminals" if they never committed a crime, they are people
being detained based on probabilities.

~~~
JumpCrisscross
Has it had an effect on crime rates? If so, this is interesting. It means
crime is predictable based on the inputs the system is measuring. It also
means the Chinese AI effort is more fruitful than I expected it to be. If this
is reducing crime, then we have a discussion of trade-offs. That is
interesting, and potentially nuanced.

If it's the latter, however, this is not interesting and there is no
reasonable discussion to be had.

~~~
SerLava
>if this is reducing crimes

Every apprehension under this program is a crime by any definition other than
the arbitrary declaration of an illegitimate regime.

It would be absolutely astonishing if the people apprehended were likely to
illegally kidnap and hold at least 1 citizen in their basement for at least
the length of time that these people are detained.

Thus it can pretty much only increase crime.

~~~
mc32
Not necessarily. If they are capturing these people because they will have
committed a crime and that in and of itself is crime then its not reducing
crime.

It's possible that once enough people who on balance might commit a crime are
arrested or impeded, then crime will indeed decrease.

~~~
rayalez
I think you've missed the point.

Even if this system works, people who have not committed a crime (yet) are
innocent.

Taking an innocent person, and improsoning him against his will is a crime,
pretty extreme one.

Even if this system works really well, it's hard to imagine that the crimes it
prevents can be more extreme than the crimes it commits.

~~~
JumpCrisscross
> _Taking an innocent person, and improsoning him against his will is a crime_

Every criminal justice system has to balance (1) the risk of convicting the
innocent, (2) the risk of letting the true crime-committer commit additional
crimes (due to not being caught or caught later), and (3) the risk of being
abused. We don’t like pre-crime convictions because we believe risks 1 and 3
outweigh any gains in 2. A culture with a different balance of risks, however,
could come to a different but consistent conclusion. I don’t like that
conclusion. But it’s at least debatable. Putting political enemies in jail is
not.

------
elmar
"Minority Report"

~~~
anujdeshpande
For the uninitiated, it's an amazing movie starring Tom Cruise which talks
about "precrime" \- although no big data involved. Highly recommend that one
watch it
[https://en.wikipedia.org/wiki/Minority_Report_(film)](https://en.wikipedia.org/wiki/Minority_Report_\(film\))

~~~
Noted
Based on a short story by Philip K. Dick
[https://en.wikipedia.org/wiki/The_Minority_Report](https://en.wikipedia.org/wiki/The_Minority_Report)

~~~
miketery
For those into Black Mirror or PKD, new show called "Electric Dreams" is based
on PKD short stories - I recommend it, a few excellent episodes.

------
arnaudsm
The NSA is already doing the same thing [https://arstechnica.com/information-
technology/2016/02/the-n...](https://arstechnica.com/information-
technology/2016/02/the-nsas-skynet-program-may-be-killing-thousands-of-
innocent-people/)

Machine learning is still a black box, and politicians don't seem to
understand this.

~~~
Cacti
There is a massive difference: the NSA/USG isn't using that against it's own
citizens.

~~~
mamon
Of course it is, you are just unaware of it, because, unlike its Chinese
counterpart, NSA chooses to maintain illusion of freedom and democracy.

------
smsm42
More correct headline should be "China using big data to detain people who did
not commit any crime". At that point you're realizing this has already
routinely been done by many countries, including China, without using any big
data at all. In fact, detaining people who didn't commit any crime has been
done before people knew what "data" is.

------
liberte82
It appears that police are still building profiles themselves by sifting the
big data for terrorist-matching patterns.

The next step is to let AI do it all. We don't even have to know what is being
matched on what, just that Citizen A's "profile" is sufficiently similar to
Terrorist B's "profile".

This is how AI takes over the world without requiring sentience.

------
FridgeSeal
This is not the world we want.

~~~
liberte82
But is it the world we need?

~~~
Dangeranger
No, but it's the world we deserve by allowing mass surveillance and the police
state to exist.

~~~
duncan_bayne
Allowing? People will literally pay for it to happen. Witness smartphones,
social media, smart homes, ...

~~~
yanaccount
The blame should not be on people who choose not to forgo the comfort of
modern technology, it should be on a state which abuses that technology for
dubious purposes.

~~~
duncan_bayne
No-one (perhaps except folks like Stallman?) is blameless here.

One of the responsibilities of citizenship is to preserve your liberty against
encroachment by State and private actors. Neglecting to consider that is a
moral failure, and one I've been guilty of in the past.

I'm not saying that it's necessarily immoral to, say, have a FB account. But
it is to have one without considering the implications and acting accordingly.

~~~
z2
Is it also a failure (moral or otherwise) for the State to not tell citizens
of this particular duty? Speaking anecdotally, citizens' duties are vaguely
described as "participate, stay informed, obey and support laws." It took me
decades to notice the subtle and complex intentions of the State and private
actors that counter freedoms in said laws.

Not being Stallman or a full-time civil liberties defender, what's a good way
for the basic citizen to be more informed, and better evaluate FB-like
services?

~~~
duncan_bayne
> Is it also a failure (moral or otherwise) for the State to not tell citizens
> of this particular duty?

Yes - but realistically, what do you think State education is _for_?

Primarily, it's to ensure a citizenry that obeys and supports laws. Staying
informed is a very distant second, and participation is actively quashed from
the get-go. (What proportion of your local school board consists of children?)

As for how to stay informed - learn the basics of the technologies involved,
and follow the news feeds of folks with a deep interest in the field. In this
case, the EFF is a good start.

But realistically, there's too much for one person to cover. Tech? Sure,
perhaps. But then there's medicine, finance, real estate, construction ...

So really, what you're left with is electing representatives who will provide
responsible oversight. But realistically, almost all of them are in the
pockets of lobbyists at this point.

TBH I'm not sure what the solution is, here.

So perhaps it _isn't_ a moral failure any more, because the scope has grown to
the point where most people say "fuck it" and give up.

------
MikeGale
Many countries are using these pre-crime estimators. What we need to see are
the false positive and false negative rates.

Anybody got measures of those?

~~~
3pt14159
Well it's pretty easy to generate the ROC curves for this one. We're talking
discrete events over a defined timespan, right? Just create a control group
and monitor the difference.

------
Dangeranger
“The existence of a majority logically implies a corresponding minority.”

------
arialeks
Pysho pass anyone?

~~~
zitterbewegung
That’s probably the closest to the article because the system justifies itself
to be right. Other commenters have said that it’s a case of parallel
reconstruction . (Which would rule out minority report).

------
45h34jh53k4j
This will be rolled out further and further, first the rest of China, then
other parts of Asia, eventually Europe and the Americas. Strongly resist the
rollout of this technology, along with the social credit rating systems.

This is like the last chapter of humanity before we stagnate for hundreds of
years.

~~~
IntronExon
In the US at least, there are probably enough people with a significant number
of firearms that it would take a war to make this work. I’m not usually fond
of anti-government fanatics, but I have to admit that in extremes they can be
useful.

~~~
jstarfish
This is not a situation that can (or should) be resolved through violent
rebellion.

A violent response would only serve to justify the application of the system,
to cut the head off of any leadership structure before it materializes (or
just pick off the lone wackos before they cause problems).

~~~
IntronExon
If it came to violence then disaster would already have struck, but there is a
deterrent effect when a statistically significant portion of your population
would fight to prevent it. The cost moves from, “pass some laws, enforce as
usual,” to “commit to atrocity, and massive civilian losses.”

------
rw
"Your scientists were so preoccupied with whether or not they could, that they
didn't stop to think if they should."

\- Jeff Goldblum as Dr. Ian Malcolm in Jurassic Park

------
kwhitefoot
Tony Blair apparently wanted to do the same by psychiatric examination of pre-
school children:
[http://news.bbc.co.uk/2/hi/uk_news/politics/5301824.stm](http://news.bbc.co.uk/2/hi/uk_news/politics/5301824.stm).

------
zitterbewegung
So as many of the comments echoed they are using big data to perform parallel
reconstruction to detain people and justify it using some kind of
computational technique ?

------
shmerl
Almost Watchbird:
[http://www.gutenberg.org/ebooks/29579](http://www.gutenberg.org/ebooks/29579)

------
breitling
Don't we all already do this to some extent?

Our intelligence agencies (at least try to) predict terrorist activity and put
a stop to it be before anything happens.

~~~
Thriptic
Flagging people for human review and monitoring =\= detaining them

------
liberte82
If you want a vision of the future, imagine a boot stamping on a human face -
forever.

------
CyberDildonics
I would like the see analysis of the politicians that allowed this.

~~~
duncan_bayne
Would it surprise you to learn that they resist being exploited by the very
systems they promote?

[http://www.zdnet.com/article/fcc-chairman-browsing-
history-f...](http://www.zdnet.com/article/fcc-chairman-browsing-history-
freedom-of-information/)

------
luizfzs
Person of Interest

------
freecodyx
reminds me of pychopass

