
UK Cops Want to Use AI to Spot Child Abuse But It Mistakes Desert Pics for Nudes - zeristor
https://gizmodo.com/british-cops-want-to-use-ai-to-spot-porn-but-it-keeps-m-1821384511
======
jjp
Gizmodo rewrote
[http://www.telegraph.co.uk/technology/2017/12/18/artificial-...](http://www.telegraph.co.uk/technology/2017/12/18/artificial-
intelligence-will-detect-child-abuse-images-save/) and didn't include some
interesting details.

~~~
collyw
Was it Gizmodo that posted the doctored version of James Damores memo where
all references were removed? Clearly not a reputable place to get information
from.

------
gaius
The UK police are dealing with a scandal right now in that they tend to
suppress electronic evidence proving a suspect is innocent and try to
prosecute anyway. The last thing we should give them is another electronic
capability.

[http://www.telegraph.co.uk/news/2017/12/19/met-orders-
review...](http://www.telegraph.co.uk/news/2017/12/19/met-orders-review-rape-
cases-second-trial-collapses/)

------
YeGoblynQueenne
So, the Met wants to train image recognition AI to find child abuse images?
I'd be surprised if there aren't some interesting ethical questions around
this.

To begin with, the way image recognition algorithms work -they learn from
_examples_ \- to train an image recognition AI to identify images of child
abuse you'd have to provide it with a substantial database of such images. I'm
pretty sure that's illegal, although of course the Met probably has some sort
of exception for the purpose of fighting crime. But, they still have to keep
around a big database of child abuse images - and update it regularly, to do
this job. It's kind of... icky.

Then again, what about victim protection? Besides the fact that keeping such a
database is risky because it can always fall into the wrong hands (it only
takes one misconfigured database server) it's also a characteristic of the
most advanced image recognition algorithms (conv-nets) that their models can
be used to _generate_ new images of the kind they've learned to identify. So
presumably, if the _model_ itself fell into the wrong hands, someone could
start generating new child-abuse images from it (low-fidelity and not really
very useful, I imagine, but still).

So I wonder if the Met has addressed such ethical issues, anywhere.

~~~
ravenstine
Who wants to be the developer to train a neural network on child porn? You
couldn't pay me all the money in the world twice over to take that job.

------
emmelaich
Not unexpected. It's surprisingly difficult to tell your arse from your elbow:

[http://stupidstuff.org/ass_elbow/](http://stupidstuff.org/ass_elbow/)

~~~
tritium
But... I... I just want to know the answers. That reset quiz button... It’s...
Why?

~~~
henpa
lol! The missing "submit" and only a "reset" button also drove me nuts!!! That
page actually does some javascript code, which is not currently working. (you
can see this when you click on one of the options while in console/devtools).

~~~
tritium
Oh, ha! Awesome! I was killing time on my phone, without full debug options,
and from the looks of the site, I figured maybe they were being _extra_ cute.

------
thecopy
A bigger question is why does the police spend time looking for pornography?
What type of society is the UK developing into?

~~~
ry_ry
Isn't this specifically intended to identify images of child abuse though,
rather than rifle through the average users pornhub cache?

~~~
beaconstudios
Yes, but we are in fact getting a nation wide porn filter in the near future.

~~~
shocks
Are we? Source?

~~~
have_faith
I believe they are referring to the opt-out default filter isp's have been
enabling. Not sure on the laws surrounding it.

~~~
edjw
No laws requiring ISPs to apply filters. It's just a voluntary agreement
between ISPs and the Government. The Digital Economy Act 2017 has provisions
in it to allow ISPs to apply default filters if subscribers agree to them in
terms of service.

There are requirements in the DEA 2017 for 'adult' websites to verify the age
of people who visit the site with the threat of their website being blocked if
they don't comply. Check openrightsgroup.org for more on this

~~~
nly
You're missing one important detail: the reason the agreement is 'voluntary'
is that the Government threatened to force ISPs to do it through law if they
didn't volunteer.

~~~
lostlogin
Voluntold.

------
nitwit005
> With the help of Silicon Valley providers, AI could be trained to detect
> abusive images "within two-three years", Mr Stokes said.

If the goal is to get something so high quality that the police won't ever
have to look at it, I doubt it. You have Google and Facebook employing people
to deal with user reports about this stuff, as well as ad companies manually
checking websites.

~~~
IanCal
There would be a significant benefit to just being able to reliably get rough
numbers plus some examples of the worst pictures for an expert to grade rather
than having the task of searching 10000 horrific pictures to see if any fall
into certain categories.

~~~
Brotkrumen
That's what their story is, but unlikely to be true. All "horrific" pictures
will have to be graded by a human, because you have the right to face your
accuser, the service providers will not let you near their codebase and AI
won't be 100% accurate in the next few years.

What they will be able to do with that software is trawl through orders of
magnitudes more pictures, filter by probable hits and show those to staff.
Right now someone has to be suspected to warrant the expensive attention of an
expert. With that software you can fish in everyone's data.

~~~
IanCal
Cases reported in the media usually talk about people having thousands of
pictures each, I wasn't talking about thousands of different people.

~~~
Brotkrumen
Thats the point. None of the 90% normal pictures will do any psychological
damage, the 10% illegal pictures will but those will have to be looked at
anyway.

The advantage of an algo detecting illegal content isn't in protecting civil
servants, it's scanning everyone without them.

------
ajmurmann
I don't get it. Who thinks it's a good idea to fully automate anything like
this? Regardless of how accurate the algorithm is a human should look at this
before any action is taken. As long as the aren't false negative this is
probably a huge win. Even if we have 5% false positives that hugely cuts down
at what people need to look at. So desert pictures looking like child porn
doesn't matter much. Child porn looking like something else is much worse.

Edit: awful typo "lol" -> "look"

------
wiredfool
well, they can kiss my shiny metal:
[https://www.flickr.com/photos/wiredfool/901581193/](https://www.flickr.com/photos/wiredfool/901581193/)

And then there's the classic Weston photograph:
[https://en.m.wikipedia.org/wiki/Pepper_No._30](https://en.m.wikipedia.org/wiki/Pepper_No._30)

(Edit, IOW, not a new problem, photographers have been pushing at this sort of
issue for a long time)

------
demosthenes111
I suspect part of the problem with this approach relates to data set
construction. Even though models these days should theoretically be able to
handle this task, there are clear ethical concerns and practical issues with
making datasets of illegal imagery large enough for training. It really raises
the question - is creating a dataset like that ethical, assuming the
intentions are to stop further abuse and dissemination?

There might be work arounds (like training one model for nudity and another
for age) but such approaches are almost certain to have "suboptimal"
performance as compared to a single model trained on a relevant dataset. Maybe
something like that is the cause for the performance issues discussed in the
article.

~~~
soundwave106
Honestly, if they stored evidence collected from previous cases, the data
probably already exists in some form. As long as the data was closely guarded
internally, I'm not sure if there would be an ethics problem using this for
training.

The biggest problem with obscenity detection, though, is getting the context
right. The AI might be able to get to the point where it can detect "naked
human" at a good percentage level. At the moment, however, I doubt it could
detect whether the naked human was considered obscene in current culture, eg:
the difference between "child pornography" and "famous Vietnam War photo" as
alluded to in the Gizmodo article. So no matter how good their model gets,
without further refinements in AI it would only be good for a "first pass", I
would think.

------
avh02
Having grown up in Saudi, I had heard that they had (literally) the same issue
when MMS was originally introduced many years ago - but they have a lot more
desert and are simply looking for porn, not even anything abusive.

~~~
lostlogin
Given that children are forced into marriage in Saudi, it would seem strange
for them to care about images of the abuse they condone. A small bit of
consistency at least.

------
tomcam
> Stokes told The Telegraph that the department is working with “Silicon
> Valley providers” to help train the AI to successfully scan for images of
> child abuse

AI requires large image sets to do the training. These images are illegal to
possess. I wonder how such training gets done without violating the law.
However, I do not wonder enough to do web searches about this matter. I am
probably on enough government lists already

------
peter303
The Silicon Valley television show had the best parody: a deep learning
program trained to find hot dogs for a foodie program repurposed for a phallus
porn detection program.

------
raverbashing
There are some training techniques that rely on negative examples to improve
classification, though I'm not sure they exist for images

~~~
microcolonel
Well, this can somewhat be accomplished by classifying exactly what an image
_isn 't_, but as you can imagine, that is (in the general case) a combinatoric
nightmare.

~~~
yorwba
You can just do binary classification. That makes the decision boundary a bit
more complicated, but if you're using neural networks, that is solved by
adding another layer at the end.

The "problem" here is probably not the lack of negative training data, but
just that they want as few false negatives as possible, so they use a very
sensitive model. That naturally increases the number of false positives. It's
still a win if you only have to look at some desert pictures instead of
literally every picture on someone's computer.

------
phillywiggins
Anyone else have an Arrested Development flashback? :)

~~~
dpiers
Close up, they always look like landscape...

------
edjw
That headline. FFS. It's either child abuse or pornography. It can't be both.

~~~
efdee
Sure it can. One person's abuse can be another person's pornography.
Disheartening but no less true.

~~~
bionoid
> One person's abuse can be another person's pornography.

That is accurate for virtually all pornography. If you research how the
"industry" works, you will quickly discover that it's built on advanced
psychology, specifically hypnosis, torture, culture of fear and other
brainwashing techniques.

The porn cult is every bit as vicious, evil and destructive to women as any
cult you can name.

~~~
zimablue
I'm not familiar with the wider thesis you seem to be advocating, (could we
get a link?) but calling porn a "cult" seems like a stretch, it's not
something that has been created by a group and spread through brainwashing,
porn is throughout history and across cultures.

~~~
bionoid
I don't have a link that explains the situation well, which is why I am
[slowly] working on a documentary project to bring it to the public.

> calling porn a "cult" seems like a stretch

That is the general perception, but as I said initially, girls are
indoctrinated and trained using advanced psychological manipulation,
indistinguishable from what most people think a "cult" is.

Everyone is free to believe that massive numbers of young girls just
absolutely love the taste of cum, rimming, and so on. Nothing fishy is going
on...

~~~
M2Ys4U
>Everyone is free to believe that massive numbers of young girls just
absolutely love the taste of cum, rimming, and so on. Nothing fishy is going
on...

Or they could be, you know, _acting_.

~~~
bionoid
> Or they could be, you know, acting.

Yes, precisely like Tom Cruise is just an actor. Let's forget about the
scientology thing.

------
pvaldes
perl -e 'print "yep, is porn\n" if $filename =~ /(Porn|Sex|XXX)/;'

Should spot the 99% of cases.

~~~
guitarbill
perl -e 'print "yep, is porn\n" if $filename =~ /(.avi|.mp4|.wmv)$/;

