
Safe Crime Prediction: Encrypted Deep Learning for Less Intrusive Surveillance - williamtrask
https://iamtrask.github.io/2017/06/05/homomorphic-surveillance/?hn=1
======
TTPrograms
The number of people killed by terrorism in developed countries per year is,
what, a hundredth of those killed in car accidents? A tenth of those shot in
Chicago alone?

[http://www.businessinsider.com/death-risk-statistics-
terrori...](http://www.businessinsider.com/death-risk-statistics-terrorism-
disease-accidents-2017-1)

Terrorists have to kill literally hundreds of times more people per year for
it to justify even the meagerest response. And one may argue about the
economic impacts, but that's just a reflection of this underlying
irrationality. If we had media focusing on "Keep Calm and Carry On" like
Israel does now or Britain did in WWII it would have a much smaller economic
impact - and terrorists would stop doing it, since it would be less effective
at achieving the subgoal of "terror".

So this post is really begging the question IMO.

~~~
petre
If the media wouldn't cover terrorism it would probably go away by itself.
Israel is probably the only country dealing with it effectively.

~~~
__jal
Not just the media. A lot of current U.S. politicians apparently believe their
best path to power is endless fear mongering. And it works pretty well.

~~~
mmirate
That's interesting. While studying the Reichstag Fire in grade-school history
class, I ran across a quote from someone in the Nazi propaganda machine, who
asserted something to the effect that exerting control over a democracy was
simply a matter of manufacturing a large enough exterior threat for them to
fear. Sadly, I cannot remember the quote nor the author.

~~~
__jal
It was a well-discussed idea at the time, so I'm not sure who you're referring
to. But Hermann Goering famously discussed it with Gustave Gilbert:

"[...] the people can always be brought to the bidding of the leaders. That is
easy. All you have to do is tell them they are being attacked and denounce the
pacifists for lack of patriotism and exposing the country to danger. It works
the same way in any country."

~~~
mmirate
That was the one! Clearly I didn't remember the details of the wording well
enough to use the right search term.

------
CiPHPerCoder
There are multiple elements to unpack here. The obvious pre-crime problem
other commenters raise. The fact that terrorism is such a rare risk it isn't
worthy of our attention.

But I'd like to raise another objection: Homomorphic encryption does not
provide integrity over the ciphertext, which could open the door to active
attacks against the systems that undermine its privacy goals.

[https://news.ycombinator.com/item?id=14443191](https://news.ycombinator.com/item?id=14443191)

[https://paragonie.com/blog/2016/08/crypto-misnomers-zero-
kno...](https://paragonie.com/blog/2016/08/crypto-misnomers-zero-knowledge-
considered-self-descriptive)

If you really need to build such a dangerous and needless system, would you
want it to be built with such an error-prone cryptographic design? I'd say
"No".

------
Asdfbla
Interesting, I didn't know that homomorphic encryption was already advanced
enough to be feasible for actual computation. Years ago I just read about the
proof-of-concept systems that worked but were too slow.

I'm still doubtful if the specificity of such pre-crime systems will be high
enough that only a negligible number people will be wrongfully investigated.
After all, the prevalence of terrorism is extremely low in any population. I
guess if you trust law enforcement enough to escalate the investigation slowly
and carefully (instead of putting a suspect on a no-fly list immediately) it
can work.

Also, with regard to the audits by NGOs or government watchdogs: I suppose you
would also need the auditors to cryptographically sign the version of the
software they audited so that users can check that a trusted surveillance
system was deployed.

~~~
williamtrask
Re: cryptographically signing the software version. I think that's a brilliant
idea. I'll look into it.

------
rocqua
Ok, I'm probably missing something.

How is public-key homomorphic encryption possible? Suppose I know the pub-key,
and have some cipher text X. Can't I simply encrypt 0 and try, for all
plausible values V, whether encrypt(V) - X = encrypt(0). Or is the encryption
function not reversible, i.e. there are multiple 0s?

If not, it seems like you'd need quite a bit of true entropy in the plausible
values. I don't see how you add something like a Nonce to artificially add
entropy.

Heck, if you have nice integer under/over flow on division, that would give a
crude implementation of comparison, bringing the search down by a logarithm.

It seems like the only reasonable situation is one where indeed multiple
different values decrypt to 0.

~~~
murbard2
> Or is the encryption function not reversible, i.e. there are multiple 0s?

Yep, there's padding.

~~~
rocqua
Whats to prevent me from finding 'almost all' zeros by trying enough (say 10
000 000) versions of enc(v) - enc(v).

That, combined with comparison based on integer division and underflow would
still make decryption quite easy.

~~~
murbard2
replace 10 000 000 with 2^256

------
sambull
"A positive prediction should launch an investigation, not put someone behind
bars directly."

That is the scariest thing I've read. All we need is a black box to
investigate people at anytime.

~~~
williamtrask
There are already thousands of black boxes. Consider satellite photography,
malware detectors, fire alarms, sniffing dogs, credit fraud detectors. All of
these are tools ("black boxes") used to launch investigations. The future of
crime fighting is impossible without tools like these as criminals become more
sophisticated.

The real question is who owns them and are they audit-able? This blogpost is
about making neural networks used for these purposes auditable by a third
party without making them vulnerable to evasion by criminals.

~~~
rrggrr
Agreed. Stop, assess and arrest is now audit-able thanks to body cameras and
we're already seeing the results in greater accountability. However, with
greater visual accountability has come de-policing, a serious problem in some
cities.

------
bilotrace
Several problems with your points \- Data

For the SPAM example you provided, you used a Data Set available publicly with
no consequences. But where will the training data for prediction of homicides
will come from? Will it be accurate? Will it unfairly target minorities?

The predictor uses features in the data to predict the future. What features
will the machine learning algorithm learn to predict homicide? location? age?
gender? ethnicity? mannerism? income? I could see big problems with any
features used to detect and potentially send law enforcement for
"Investigations"

\- Detectors

Capturing and sending users data to a warehouse is a privacy/security risk.
But what is worse is installing a detector in their house, or in their
computer. What happens when a citizen removes the SPAM detector from their
computer? Well, the next logical step is to pass a law requiring all citizens
to run a specific process on their computer and are not allowed to reverse
engineer it or else...

For me, this is even worse than having my privacy violated. It would mean
users would not have root access to their own computer. It would mean, many
applications will be illegal and developers cannot write certain applications
that enable people to send emails. If this is not impossible, it would be the
worst outcome for everyone.

\- Globalization

The SPAM detector, the fire detector and the sniffer dog are great localized
examples. But todays problems are globalized. Attacks might be planned and
coordinated from different country with different set of rules. And not all
countries are considerate when it comes to privacy. How will global security
surveillance deployment work when we can't even agree on matters of climate
change?

------
dbranes
For those unfamiliar with homomorphic encryption, I found this to be a good
algebraic treatment.
[https://web.wpi.edu/Pubs/E-project/Available/E-project-04261...](https://web.wpi.edu/Pubs/E-project/Available/E-project-042612-132350/unrestricted/Meissen_MQP2.pdf)

------
liamcardenas
I like this, but I would argue that predicting crime isn't the core problem
that law enforcement has today. The real issue is the incentive structure that
disincentivizes crime prevention.

That sad truth is that police bear no cost when they fail to prevent crime
and, in fact, get more funding and power if crime goes up.

~~~
zappo2938
>> The sad truth is that teachers bear no cost when they fail to teach a
student.

~~~
liamcardenas
In public schools, this is often the case. In private schools, bad education
will lead to a loss of profit. If you think this doesn't apply to teachers,
you must've never been in a bad public school

------
jsonderulio
I think you mean "Pre-crime"

~~~
williamtrask
i think you're right

~~~
jsonderulio
The dog argument was cute. But the dog also doesn't make a searchable,
indexable list of all your personal information.

Someone will inevitably make this though, and it will inevitably be abused.

Plus what if I start switching search parameters from say, 'planning a
terrorist attack' to, likely to to vote one way, believe on thing, or be of a
certain religion.

We will trade all our privacy and the nefarious people will switch to a new
method of comms...like they always do.

~~~
williamtrask
So part of what makes this work is that it's _not_ an index. As opposed to
doing general storage of people's data, this restricts a surveillance
operation to only be able to identify specific concepts.

~~~
williamtrask
that's strange, i would have assumed that they would be averse to a tool that
doesn't give them un-restricted access to the data. right now, the dialogue is
"privacy vs security... take your pick" but this breaks that rhetoric...
showing that it's a false choice.

~~~
williamtrask
Far from it, this tool prevents the need for us to give up our privacy. (not
sure why HN isn't giving me a "reply" button to your posts)

------
AceOfSpace
I'm concerned with a tool such as this that can be directed to target anything
of interest to those administering the system. We've already seen mass
surveillance shift from being used "exclusively" for terrorism to also
assisting in the war on drugs. Where else would the lens be trained? As the
author suggests, murder. What about organized crime? Gangs? Illegal downloads
... the sharing of Netflix passwords?

------
grandalf
Does anyone know what minimum level of "homomorphism" is necessary to insulate
the entity doing surveillance or analytics from legal action?

I ask this about the following contexts:

\- government surveillance: does using a homomorphism remove the need for a
search warrant?

\- in-app analytics: does using homomorphisms allow a firm to consider data
not disclaimed by the firm's privacy policy?

\- research: when does a homomorphism eliminate the need for IRB approval?

~~~
williamtrask
Great question. I sincerely doubt there is legal precedent in this area
though. I'd love to be wrong here though :)

------
aub3bhat
There was a great talk at USENIX on how Whatsapp reduced spam while
maintaining end to end encryption

[https://youtu.be/LBTOKlrhKXk](https://youtu.be/LBTOKlrhKXk)

------
aomix
I have nothing useful to add to this discussion. I just like that this idea is
a core part of the TV show Person of Interest.

