
UK unveils extremism blocking tool - dberhane
http://www.bbc.co.uk/news/technology-43037899
======
sequence7
So it looks like one of two things has happened:

1) The British government have built AI tech capable of identifying extremist
views in any video at the incredibly low price of £600000, doing what no other
technology company in the world is capable of.

2) A smart consulting company has convinced the British government to waste
£600000 on building a model that can flag certain videos from a very specific
training set but in real world use is horribly innacurate and completely
useless.

~~~
BenedictEggs
The politician pushing this is the Home Secretary Amber Rudd, who is notorious
even among UK politicians for her shaky grasp of technical matters. Anyone
unfamiliar with her previous work is advised to Google "Amber Rudd
encryption".

My guess: someone enterprising has seen her on TV and spotted a business
opportunity. We should be grateful that they only took us for £600k.

------
slivym
What's really worrying about this to me, is how deep this goes. Hacker News is
a relatiely well informed audience about this topic. We understand:

* that the false positive rate is going to result in more legal material being taken down than ISIS propaganda by an order of magnitude even if what they claim is true.

* That the success rate on the data set you've been training on is highly likely to completely mis-represent your true success rate.

* That the processing power required for this is likely prohibitive. Not capable of being run real-time, and any actual implementation will have much worse results due to performance optimization.

That's laying aside the deep and obvious problems with the government forcing
private companies to censor legal speech with no over-sight or even a human in
the loop. Let alone law enforcement in the loop.

What this press release is, is a calculated attack on free speech.
Deliberately misleading the public about the capabilities of technology to
attack the technology companies they claim to want to work with. To apply
public pressure to private companies to do police enforcement jobs.

The only response to this is to state the obvious: If the government wants
something censored they can apply to a court injunction as is due process, and
in the mean time, let's get rid of this abhorrent stream of Home Secretaries.

~~~
npr11
* False positive rate probably depends on the cut-off for when something is flagged up for a humnan to review, but yeah, for all we know could be high

* No one quotes metrics based on the training set, so I'm sure these guys haven't done that.

* Most trained ML models don't require much computing at all to make predictions.

* The idea seems to be to flag content for review by a human, so there is a human in the loop

* No one is forcing anyone to use this... so there's no straight forward censorship issue

* maybe this does embarass bigger companies into explaining why they don't develop there own system to do this (removing IS content seems uncontroversial) - I presume this is why the Home Sec is really interested

~~~
Sionyn
Not so. They specifically stated they'd force this out technology companies if
they didn't volunteer. Typical right-wing fascism.

------
rlpb
"ASI Data Science said the software is capable of detecting 94% of IS's online
activity, with an accuracy of 99.995%."

I'm not sure what this means. It may be a 0.005% false positive rate. If
they're scanning, say, Youtube videos, then the resulting false positive
number would be huge.

~~~
m0nty
I would think it unlikely that they have a comprehensive corpus of "offensive
content", meaning they can't train/test their system on all of it. So really
they're claiming "99.995% accuracy against what we already have". The real
world experience will surely be different. You're right that the number is
essentially meaningless.

------
toomanybeersies
Why does the UK seem to take 1984 as a guide book, rather than a cautionary
tale?

As much as I applaud efforts to stop extremism, censorship in this form is
concerning. Who is the arbiter of what counts as extremism? Obviously any
website urging people to join Daesh should be blocked by their standard, but
what about websites promoting the PLO, or the PKK? How about websites about
the Rohingya? The Burmese government certainly seems to think they are
terrorists.

~~~
erikrothoff
Isn't that the difference between conservative and liberal politics? With the
conservative Tories in power this is right up their alley.

Disclaimer: Am Swedish so have absolutely no idea

~~~
gaius
You don’t remember Labour’s ID cards and biometric database? Or their
expansion of the surveillance state?

~~~
pjc50
Labour were never "liberal", though; this is why trying to map everything onto
the "conservative/liberal" dichotomy is a mistake.

Old Labour were characterised by a weak state and strong unions; New Labour
were characterised by a strong redistributive state that was safe enough for
middle-class voters; it remains to be seen what Momentum Labour actually do,
if anything.

~~~
toomanybeersies
The problem is that people keep trying to map non-American political systems
onto the American political spectrum. Tory vs Labour is a completely different
horse race to Republican vs Democrat in the USA. Same goes for most other
country's political systems.

------
pjc50
Hands up those who remember the previous incarnation of this, the broadcast
ban on Sinn Fein:
[https://www.youtube.com/watch?v=Xr8bsOgmGhI](https://www.youtube.com/watch?v=Xr8bsOgmGhI)
/
[http://news.bbc.co.uk/1/hi/4409447.stm](http://news.bbc.co.uk/1/hi/4409447.stm)

------
mosselman
Censorship is never something to get excited about. It is a slippery slope
towards the event in Spain last year when the government misused censorship
allowances to silence political opposition[1].

[1]: [https://www.eff.org/deeplinks/2017/10/no-justification-
spani...](https://www.eff.org/deeplinks/2017/10/no-justification-spanish-
internet-censorship-during-catalonian-referendum)

~~~
anvandare
"As the Americans learned so painfully in Earth's final century, free flow of
information is the only safeguard against tyranny. The once-chained people
whose leaders at last lose their grip on information flow will soon burst with
freedom and vitality, but the free nation gradually constricting its grip on
public discourse has begun its rapid slide into despotism. Beware of he who
would deny you access to information, for in his heart he deems himself your
master."

— Sid Meier's Alpha Centauri

~~~
baud147258
I've always found that SMAC is an example of an horror game, with just the
tech descriptions.

~~~
pvdebbe
SMAC is just wonderful for all lovers of technoporn

------
white-flame
As always this whole thing is pointless and stupid. As things get blocked, the
posters of this material will tweak it until it passes the AI-driven censors.

It'd all be a laughable trainwreck... if it didn't set further precedent of
censorship and potentially destroy people who fall into false positives.

------
SideburnsOfDoom
Will it block comments like "If you believe you’re a citizen of the world,
you’re a citizen of nowhere." ?

[http://www.independent.co.uk/news/uk/politics/theresa-may-
me...](http://www.independent.co.uk/news/uk/politics/theresa-may-mein-kampf-
adolf-hitler-nazi-vince-cable-liberal-democrat-conservatives-a7825381.html)

------
mavdi
We all ultimately follow the Chinese when it comes to tech. Just after much
drama and a couple terror attacks.

------
anonytrary
What a sad state. It starts with "protecting" people from terrorism, but it
will quickly devolve into humanity-stunting censorship. Instead, educate
people about terrorism. We cannot defer education in the hopes of protecting
people. This never, ever ends the way it is intended.

One day I fear I will open my eyes to find parts of the world blurry, because
of state-mandated image filtering device, embedded in my eye.

------
huffmsa
Blocking porn => blocking "Xtreme" Content => Oceania has always been at war
with Eastasia, never at war with Eurasia

------
tankenmate
And what about academics that study these phenomenon? It will encourage
research to go underground.

~~~
realusername
And journalists as well... And that's implying the video detection works
perfectly 100% of the time (which of course it won't).

------
moomin
If it doesn’t spot The Telegraph, I think it needs better training data.

------
crdoconnor
If the government were actually serious about cracking down on extremism, they
wouldn't block the content, they'd intercept it and subtly change it to make
it ridiculous and humiliating for ISIS. This really wouldn't be that hard.

Turning the notion of ISIS recruits into a joke (which, sadly, they actually
kind of are) would potentially stem their flow. Censorship, however, will be
interpreted by potential recruits as "we are afraid of the truth", will harden
their resolve and they'll figure out a way to get around it anyway.

Amber Rudd and Theresa May are simply trying to create the infrastructure for
a police state, though, using ISIS as a pretext. They're authoritarians at
heart and always will be.

------
finknotal
Does anyone believe this will work as well as they imply it will?

~~~
EnderMB
The key point for me is that Amber Rudd wouldn't rule out forcing tech
companies to use it by law.

If the UK government are going to throw their weight behind a tool, they'd
better hope that it's of sufficient quality to not be torn to shreds by some
of the most technically competent people in the world.

The UK government has always had a hostile approach to technology. Now they
want to give a tech company the chance to be hostile back, and if this tool
doesn't work I can see a very public response to the legitimacy of this tool.

~~~
pjc50
Whether it works or not has nothing to do with whether people will be forced
to use it. Tech companies who fight it will get "ENEMY OF THE PEOPLE"
headlines against them in the Mail.

~~~
EnderMB
On that subject, I'm surprised that we haven't seen a clash between
journalists and the "tech elite" yet.

Sure, there's a lot of crap written about Google, Facebook, Twitter, and the
like on printed media, but given how reliant UK broadcasters are on social
media I'm surprised a negative reaction hasn't led to Twitter banning
journalists, or Google de-listing a publication for hate-speech.

~~~
pjc50
We've had a few rows around what counts as "abuse" and "citizen journalism" in
Scotland, including but not limited to:

[http://www.pressgazette.co.uk/scottish-news-websites-
twitter...](http://www.pressgazette.co.uk/scottish-news-websites-twitter-
account-briefly-suspended/)

------
jlebrech
as long as they acknowledge a communist can also be extremist.

~~~
klez
But that's exactly the point, isn't it? To me, people trying to censor media
are extremists, because of how _I_ view the world. In the same way, they may
flag me as an extremist because of this.

In the end, the problem with censorship is that it gets defined by someone who
decides what is extremist and what's not.

