
The Rise of the Weaponized AI Propaganda Machine - prostoalex
https://scout.ai/story/the-rise-of-the-weaponized-ai-propaganda-machine
======
lurchpop
Ok, so people living outside major cities in UK voting against centralization
of unaccountable power in the EU and unemployed, politically abandoned
industrial workers in the US Rust Belt were just a bunch of idiot rubes
tricked by click bait headlines. Does it occur o anyone they may have actually
had real grievances against globalization? Maybe they were tired of being
belittled by metropolitan and coastal elites ignoring those grievances and
dismissing them as racists to be ignored.

~~~
badsock
I agree 100% that the working class people in America have been screwed over.
But just because they've been wronged doesn't mean that their solutions are
right.

Dumb ideas deserve to be attacked. Ideas that are an affront to practically
every positive development that dragged us out from under the rule of the
oligarchs of the 19th century and the fear-mongering ideologues of the 20th,
deserve to be attacked.

If that hurts the feelings of those who are promoting them, that's
unfortunate, but it's the price of admission if you're trying to exert
influence in the public sphere.

~~~
flukus
> I agree 100% that the working class people in America have been screwed
> over. But just because they've been wronged doesn't mean that their
> solutions are right.

The problem is that no one is putting forward better solutions. "The left" is
now run by educated elites, not the working class, "the right" might not
represent the working class, but at least they don't ignore them.

~~~
Retra
Plenty of better solutions are being put forward. Nobody listens to them
because they come from "educated elites" AKA those people with unique
experiences and depth of knowledge necessary to come up with new ideas.

If you don't see better solutions, it's because you're actively avoiding them.

~~~
flukus
> If you don't see better solutions, it's because you're actively avoiding
> them.

Not sure exactly what you're talking about but I've long been a proponent of
UBI for instance. But new ideas like this aren't being adopting by the
mainstream left and probably won't for some time. You never get any progress
without a firebrand to sell the message. The mainstream left seems more
interested in playing identity politics as a wedge than coming out with a
positive action plan.

~~~
Retra
That has nothing to do with the "mainstream left", and everything to do with
the volatile nature of holding political office in a sound-byte culture. Stop
listening to professional campaigners and start listening to professional
thinkers.

The fact that you even invoked the term "mainstream left" here makes it
perfectly clear that you're more than happy playing identity politics. You
speak in terms of political labels, not in terms of mechanisms for solving
problems.

~~~
flukus
> That has nothing to do with the "mainstream left", and everything to do with
> the volatile nature of holding political office in a sound-byte culture.
> Stop listening to professional campaigners and start listening to
> professional thinkers.

Why? The greatest mind in the world might solve poverty tomorrow but unless
they can convince the public they may as well keep it to themselves. Let's
also not forget that professional thinkers are often incorrect and
disconnected from reality. Support for communism was quite high amongst
professional thinkers. Not to mention that professional thinkers these days
includes the like of feminist studies professors that are about as regressive
as they come.

~~~
Retra
I'm not sure why you feel inclined to dismiss the analytical value of
historians, doctors, economists, and scientists because they might take
seriously the opinions of feminists and communists.

So, yeah. _Let 's_ forget that. Because it's juvenile and petty.

PS: You're talking to a feminist and communist. I'm not sure why you'd imagine
I'd object as you do to the points you raise. I doubt you've ever read any
communist or feminist literature, or that you'd care enough to understand it
if you did.

~~~
flukus
> PS: You're talking to a feminist and communist.

Your the intellectual elitists that Trump voters (and myself) hate. You're
influence on the left is why they are losing elections and why there are very
few left wing parties I would vote for these days.

~~~
Retra
You know nothing about me, and I certainly have no influence on the left. You
do realize that you're talking to some random person on the internet, right?
And I certainly don't give a shit who you'd vote for. I'm not surprised that I
am hated. US Republicans have been sucking on a hate-spewing propaganda teat
for almost a decade. I'm only asking that you understand the consequences of
that. You'll suffer them either way.

But whatever. You're petty and can't avoid identity politics. And I would
rather talk about actually relevant ideas, so have a good life.

------
cs702
The article mentions a "groundbreaking study" from 2013 that a psychologist
"spent years developing"...

It sounds impressive, until I looked at the actual study[0], which uses simple
linear or logistic regression on 100 features obtained by low-rank
approximation of an m users × n Facebook-likes matrix via plain-old singular
value decomposition (SVD)[1], as shown here:
[http://www.pnas.org/content/110/15/5802/F1.large.jpg](http://www.pnas.org/content/110/15/5802/F1.large.jpg)

I don't know how anyone could call this "groundbreaking," considering that
lots of people in Silicon Valley have been using matrix factorization to
predict consumer behavior and make recommendations for a couple of decades,
give or take.

I stopped reading at that point.

[0] [http://www.pnas.org/content/110/15/5802.figures-
only](http://www.pnas.org/content/110/15/5802.figures-only)

[1] [http://www.ams.org/samplings/feature-column/fcarc-
svd](http://www.ams.org/samplings/feature-column/fcarc-svd)

~~~
monk_e_boy
I don't know much about US elections in regard to facebook and fake news, but
I did pay attention during Brexit. Most 'news' was fed to us through
newspapers (online and offline) the TV and radio. I didn't see a single fake
news site, I saw some crap forwarded on facebook most of which had lots of
comments calling it out as crap.

~~~
dredmorbius
Resurrecting an inexplicably dead comment:

 _Ithere were a ton of brexit bots on twitter and in comment sections, they
later started working for trump, and are now working for le pen. some stuff
about it here -_ [https://heatst.com/world/how-russias-twitter-bots-and-
trolls...](https://heatst.com/world/how-russias-twitter-bots-and-trolls-work-
with-donald-trump-campaign-accounts/)

~~~
leereeves
Perhaps it's dead because it's dismissing people as bots and trolls because
the author doesn't agree with them.

> Broadly speaking, Trump has two categories of support on Twitter. Alt-right
> trolls, and Russian bot accounts

------
zeteo
> By leveraging automated emotional manipulation alongside swarms of bots,
> Facebook dark posts, A/B testing, and fake news networks, a company called
> Cambridge Analytica has activated an invisible machine that preys on the
> personalities of individual voters to create large shifts in public opinion

And somehow managed to overcome the constant "fact checking" by major cable
networks, the Post, NYT and generally a large majority of the established
media. Let's be serious. The narrative is ridiculous, but even if we were to
take it at face value the cure is worse than the disease. Basically voters can
no longer be trusted with making up their own mind in a democracy and they
need to be automatically manipulated for the greater good:

> From now on, the distinguishing factor between those who win elections and
> those who lose them will be how a candidate uses that data to refine their
> machine learning algorithms and automated engagement tactics. Elections in
> 2018 and 2020 won’t be a contest of ideas, but a battle of automated
> behavior change.

I think at this point the losing side needs to accept that they didn't make
their case properly ("the other guy is worse" only gets you so far) and stop
chasing ridiculous scapegoats. American democracy has endured through
secession, yellow journalism and Joseph McCarthy. It will survive Facebook and
sentiment analysis bots.

~~~
pgodzin
> And somehow managed to overcome the constant "fact checking" by major cable
> networks, the Post, NYT and generally a large majority of the established
> media.

Well a big part of the campaign was delegitimizing the establishment as "fake
news" that is in bed with Hillary. Combined with a bombardment of headlines
that confirm their biases, targeted by personality, it could have a big
impact.

~~~
flukus
Could it be that the tactic only worked because the establishment was in bed
with Hillary? She is about as establishment as the come after all.

~~~
pgodzin
Except the delegitimizing was done to the point where fact checking actual
lies and falsehoods were simply ignored. This targeted approach didn't have to
"overcome" that, as the parent claimed, just make people ignore it entirely.

------
SomeStupidPoint
This is our fault, you know.

Many of us, myself included, have worked on technologies that were meant to
profile and manipulate users. Many more of us worked for companies who did
this or did similar things through traditional means (eg, psychologically
manipulative ads). We've not only allowed, but encouraged technology to have a
manipulative and addictive relationship with users. We took the paycheck and
played red team against humanity. (This is common -- red team is sexier than
blue team, and has an easier time of it, pretty much in any field.)

So, personally, I think it's time we spent some of our effort undoing that
damage and playing blue team.

I don't know what that looks like in full -- though I have some ideas -- but I
do know that allowing the weaponization of psychology against humanity to
continue unabated will end in disaster.

In some ways, that's the struggle of the 21st century: will technology serve
the public or enslave it.

~~~
Natsu
It would seem more honest to help people avoid manipulation in general, rather
than simply helping to manipulate people in a different way.

~~~
SomeStupidPoint
...which is why that's basically what I suggested.

"Red team" and "blue team" are referencing offensive and defensive teams using
standard security lingo. In this case, red team is trying to manipulate and
control the public. As another commentator pointed out, it's a little unclear
what blue team even _should_ be doing.

I avoided just saying "help people avoid manipulation", because the
implementation of a defensive strategy might be self-directed manipulation
overriding the other manipulation you're exposed to, as it's not necessarily
possible to avoid manipulation entirely. (Or even a deairable goal, as you
might want to adjust your existing habits.)

My main point was that we need to start developing defensive tactics and
strategies, because the offense dominated present situation will end in
catastrophe.

~~~
zardo
>As another commentator pointed out, it's a little unclear what blue team even
should be doing.

If the primary tools of offence are making use of addictive technology and
manipulation, the defencive move is making technology available to help people
overcome addiction and give direction to their own lives.

As you say, self directed manipulation, and I would add introspection. Maybe
FOSS tools for cognitive behavioral therapy.

~~~
SomeStupidPoint
I generally think there are a few aspects we can do:

* Allowing people to intentionally program themselves, as a means of overwriting and weakening offensive programming. In some ways, this cements the status quo, but in others, it allows your interest in "human values" (eg, wanting to be able to converse with family who disagree) to effectively fight back against centralized radicalization.

* Collating and rewriting news/information with intentional bias added, but things like dog-whistle phrases removed. Think averaging across a wide set of news to get a "fact sheet", then using something like style transfer to rewrite it in several tones/styles/biases. We can't (and shouldn't) eliminate the spectrum of perspectives, but we can probably clean some of the crap the channels are carrying. The organizational structure to manage that process, though, is a very open question. (My vote is a panel of enemies -- can we get WaPo/NYT and RT and China's people to agree on a system?) Also, people subscribing "across the spectrum" probably are harder to program.

* Education and organization. It turns out people are able to do some of these things on their own, if they know they should. I think there's really a range here -- from basic awareness campaigns to things like organizing trade groups and lobbying for ethical guidelines. Arguably, many of the professional psychologists at places like Facebook are engaged in mass-scale, unregulated psychiatry on non-consenting persons. We should call them out on that.

* Sort of a subpoint of the previous one, but theoretically you can use red team tactics to forcibly inject awareness of red team tactics. The ethics of this are questionable, so I put it last, and mostly to highlight the need to have a real conversation about ethics, guidelines, and organization.

Honestly, I know how to do much of that myself (and would really like to work
on it), but "just do it alone" isn't a good solution here because that's just
projecting my view on the situation. So my post was partly reaching out to see
if other people felt the same and might want to organize. (And particularly
people who don't share my views on most issues!)

------
myowncrapulence

      Silicon Valley spent the last ten years building platforms whose natural end state is digital addiction. In 2016, Trump and his allies hijacked them.
    

I wouldn't call it "hijacking" when that was already the intended purpose of
those sites. Just because neo-liberals lost their perceived power (I'm neither
liberal nor conservative) in said industries from years of abuse of said
systems doesn't make it somehow more nefarious now.

------
otaviokz
So yet another culprit for the DNP defeat in the last election?

I'm definitely not a Trump supporter (I'm not even American), but if I
remember correctly the DNP also had a gargantuan apparatus of traditional
media and social networking tools working 24/7 to help Hillary win the
election.

I've seen "Fake news" in both sides of every election I followed (Brazil and
US since ever and UK in the last 4 years). The big difference now is that,
aside from the fact Trump is a scary individual, this time the "big media"
side lost. Not that it is any advantage in the case at hand.

~~~
3131s
I believe you mean the DNC.

DNP is an organic compound used by bodybuilders to make cellular respiration
less efficient and therefore increase metabolism. It's quite dangerous :)

------
dredmorbius
It's just another form of advertising. The AdTech world should be delighted

And if you think advertisers won't exploit this (and aren't already), well ...
that's just precious. Bless your heart.

Advertising (and "public relations") are rooted in propaganda. Which happen to
be the titles of Edward Bernay's two classic works on the subject:

[https://archive.org/stream/BernaysEdwardPropagandaEN1928153P...](https://archive.org/stream/BernaysEdwardPropagandaEN1928153P./Bernays%2C%20Edward%20-%20Propaganda%20%28EN%2C%201928%2C%20153%20p.%29#page/n0/mode/2up)

[http://www.worldcat.org/title/public-
relations/oclc/546945](http://www.worldcat.org/title/public-
relations/oclc/546945)

And it's always been inherently political. Something the advertising
industrial support complex, which includes Google, may not wish to admit.

[https://archive.org/details/BernaysEdwardL.CrystalizingPubli...](https://archive.org/details/BernaysEdwardL.CrystalizingPublicOpinion1923noOCR)

Sleep well.﻿

~~~
davidivadavid
That kind of begs the question, though.

If you read Bernays, he doesn't have a negative opinion of public relations
and advertising, and clearly lays down why they are in fact a service rendered
to the public.

It makes as much sense to say: propaganda is rooted in advertising and public
relations.

~~~
dredmorbius
NB: raises the question, not begs.

Bernays unabashadly used PR and advertising to vile, vile ends, in his
professional heyday.

He seems to have had something of a change of heart late in life, to the point
that he advocated licensing of PR officials, in marked contrast to those he
held earlier (also mentioned in this obit):

 _[I]n an interview in 1991, when he turned 100, he said: "Public relations
today is horrible. Any dope, any nitwit, any idiot can call him or herself a
public relations practitioner." He said he was still consulting with clients
and regarded public relations loftily as a "social science."..._

 _Around his 100th birthday, he campaigned unsuccessfully to get legislation
passed in Massachusetts and other states that would have required the
licensing of public-relations practitioners._

[http://www.nytimes.com/books/98/08/16/specials/bernays-
obit....](http://www.nytimes.com/books/98/08/16/specials/bernays-obit.html)

Views can change over a lifetime.

~~~
davidivadavid
I think "beg the question" is correct.

> 2\. assume the truth of an argument or proposition to be proved, without
> arguing it.

~~~
dredmorbius
What question, specifically, do you see being begged?

I remain unpersuaded.

[http://www.quickanddirtytips.com/education/grammar/begs-
the-...](http://www.quickanddirtytips.com/education/grammar/begs-the-question-
update)

 _Begs the question is a term that comes from formal logic. It’s a translation
of the Latin phrase petitio principii, and it 's used to mean that someone has
made a conclusion based on a premise that lacks support. (1, 2) It can be a
premise that's independent from the conclusion (3) or in a simpler form, a
premise that’s just a restatement of the conclusion itself. (4,5)_

...

 _Begs the question is used wrong a lot. It took me about two seconds to find
examples of bad usage in the news. Many people mistakenly believe it 's OK to
use the phrase to introduce a clever or obvious question._

~~~
davidivadavid
I'm aware of how the expression is misused.

The question being begged is whether "propaganda" is a bad thing in itself.
The way I read your argument, you were summoning Bernays to show that since PR
and advertising come from propaganda, they're bad by association.

The problem is that it's a mischaracterisation of the concept of propaganda as
understood by Bernays, who saw it as what we see _today_ as PR, which is much
less controversial than what "propaganda" has come to mean through accidents
of history.

Essentially, that argument is akin to saying "PR (today) is bad, because PR
(back then) is bad." Ergo, I see that as begging the question, due to
equivocation on the term "propaganda."

~~~
dredmorbius
No, that's still _raising_ the question of whether or not propaganda is bad.
Though that's a fair question.

I'll disagree with your assertions and premises, though detailing why is going
to take more time and space than I can devote here. I've been researching the
question heavily for the past few months and should, the gods willing, be
posting something to
[https://reddit.com/r/dredmorbius](https://reddit.com/r/dredmorbius) ...
eventually.

------
dkarapetyan
So. Much. Clickbait. Isn't the article itself an example of such "weaponized"
propaganda?

On a slightly meta point I'm starting to notice that a much larger portion of
the population is prone to conspiracy theories than I previously suspected. It
might be another one of those broken design things. The propensity to see
patterns in everything naturally gives rise to conspiracy theory galore.

------
laxatives
This seemed inevitable. People lamented the growth of fake news and niche
circles reinforcing extreme attitudes on the internet. Ads exist because they
effectively target increasingly small and specific groups of people. Its
concerning enough when the motive is profit, but the same techniques can be
applied to much greater effect as propaganda. The real concern is when those
reinforcing themes are applied to individuals, instead of large swaths of
people.

Using Facebook, Google, LinkedIn, and other data aggregators exposes you to
manipulation. It can be convenient when it shows you relevant ads and the
motivation is clear (money). But by the time you realize you're being
manipulated for greater causes, like electing officials to power, its far too
late to revert those changes.

------
jawarner
An election is a war of opinion, and whoever wields the strongest memes wins.
Ten times as many people subscribe to Trump's subreddit as Clinton's, and
Trump memes circulate other anonymous Internet culture hubs like 4chan.

At stake is the position of the most powerful man in the world. It has been
shown to be possible that a group of skilled programmers / hackers could
influence the election. Why wouldn't, say, an expert team be hired to program
ML that optimizes a candidate's size of web presence? Cheap price for a lot of
political power.

A "weaponized propoganda machine"? Maybe not. But if it's really that easy to
game the system, you can't ignore the possibility.

------
chatmasta
When Obama hires "data crunchers" [0] to "rally individual voters" [1], it's
"innovative microtargeting."

When Trump does the same, with four years of technology advancement, it's
"weaponized AI propaganda."

C'mon...

[0] [http://swampland.time.com/2012/11/07/inside-the-secret-
world...](http://swampland.time.com/2012/11/07/inside-the-secret-world-of-
quants-and-data-crunchers-who-helped-obama-win/)

[1] [https://www.technologyreview.com/s/509026/how-obamas-team-
us...](https://www.technologyreview.com/s/509026/how-obamas-team-used-big-
data-to-rally-voters/)

~~~
uniclaude
Those two wordings aren't even coming from the same source, what exactly are
you trying to prove here?

~~~
chatmasta
I'm comparing interpretations of what amounts to the same action by Trump and
Obama. Admittedly I'm generalizing, but the analogy is more valid than a
comparison of apples to oranges. Both Trump and Obama outsourced campaign
analytics to professionals, and in both cases there were articles explaining
the innovation. In Trump's case, this article (and others I have seen), has a
distinctly negative tone. For Obama, the articles praised his data scientists
and boasted of their high post-campaign salaries.

I challenge you to find an article describing Obama's campaign analytics with
similarly negative tone to this article about "weaponized AI propaganda."

The differing interpretations of two similar scenarios are worth observing,
since the variability represents evidence of bias on the part of the
journalists writing the articles.

~~~
webmaven
There is a difference in quality in using analytics (whether demographic,
psychological, or what have you) to target a state, county, block, or
individual with a campaign message (clearly marked as such) in order to try
and sway voters to some action, and using the same or similar techniques to
create targeted "news" articles not labelled as campaign materials.

If even a fraction of what this article is alleging is true, the FEC may have
quite a bit to say about it:
[http://www.fec.gov/pages/brochures/notices.shtml](http://www.fec.gov/pages/brochures/notices.shtml)

------
PretzelFisch
"Scout combines science fiction and journalism to bring you weekly online
dispatches on the future of technology."

------
EGreg
People will keep explaining it with regular explanations.

But the rise of AI threatens all our systems that rely on the inefficiency of
an attacker.

The first one to go is trust in the media.

Look at facebook and how it uses big data to manipulate people into spending
more and clicking more and posting more and checking notifications more. Even
if it means echo chambers. Think about your own experience. Are you more
addicted today than, say, 5 years ago?

People think that they make their own decisions but when the best ideas are
aggregated then it's you vs an entire big data server farm crunching numbers.
Soon your willpower won't be enough, because peer pressure, media stories and
even laws will bring you back to increasing their metrics. They can
micromanage more and more interactions, get dossiers on everyone, and they
will outmaneuver old media more easily every year.

------
navs
I find it ironic that this article includes the following trackers:

\- segmentio

\- intercomio

\- google analytics

\- doubleclick.net

\- facebook

\- mixpanel

~~~
mirimir

        Privacy Badger detected 0 potential trackers on this page. These sliders let you
        control how Privacy Badger handles each one. You shouldn't need to adjust them
        unless something is broken.
        The domains below don't appear to be tracking you
        scout-cdn.s3.amazonaws.com
        cdn.embedly.com
        fonts.googleapis.com
        www.google.com
        fonts.gstatic.com

~~~
ENTP
I can confirm navs is correct. Brave Browser indicates blocking of 9 trackers:

    
    
      https://googleads.g.doubleclick.net/pagead/id
      https://static.doubleclick.net/instream/ad_status.js
      https://api.segment.io/v1/p
      https://www.google-analytics.com/analytics.js
      https://widget.intercom.io/widget/
      https://api.mixpanel.com/decide/
      https://api.mixpanel.com/track/
      https://api.segment.io/v1/t
      https://api.mixpanel.com/track/

~~~
mirimir
OK, so why the difference?

I'm using Firefox 51.0.1 with Privacy Badger 2017.1.26, NoScript 2.9.5.3 and
Adblock Plus 2.8.2.

Edit: OK, so I had doubleclick.net, facebook.com, google-analytics.com and
mxpnl.com untrusted in NoScript. After trusting them all, Privacy Badger still
reports no tracking, but lists these trackers:

    
    
        cdn.embedly.com
        connect.facebook.net
        www.google-analytics.com
        fonts.googleapis.com
        www.google.com
    

Strange. So is Privacy Badger failing to detect tracking?

~~~
bamboozled
I'm going to take a wild guess here, It's probably NoScript preventing the
tracking code form being loaded before ABP can perform detections?

~~~
mirimir
Well, I did trust/allow domains in NoScript. But I gather that NoScript does
other stuff by default. And there's also the fact that I'm blocking WebRTC.

Mostly I was just worried that Privacy Badger was failing.

------
mirimir
> From now on, the distinguishing factor between those who win elections and
> those who lose them will be how a candidate uses that data to refine their
> machine learning algorithms and automated engagement tactics. Elections in
> 2018 and 2020 won’t be a contest of ideas, but a battle of automated
> behavior change.

------
malloryerik
CambridgeAnalytica's site is interesting.
[https://cambridgeanalytica.org](https://cambridgeanalytica.org)

Overall this kind of thing very much jibes with Yuval Harari's ideas about
liberalism's approaching demise, as forecast in his book Homo Deus.

Edit: A Bloomberg article of note here
[https://www.bloomberg.com/view/articles/2016-12-08/no-big-
da...](https://www.bloomberg.com/view/articles/2016-12-08/no-big-data-didn-t-
win-the-u-s-election)

------
spitfire
If anyone has a copy of the main paper Cambridge Analytica were using please
link here.

I know that's vague, but to narrow it down on a previous article there was a
picture of an individual reviewing said paper. Someone noted it and linked to
the original paper.

I made a copy, but deleted it while cleaning things up.

------
SubiculumCode
Privacy is important for democracy.

------
lend000
This is painfully naïve, as though the careful selection and omission of
stories to craft narratives on the more popular "real news" sources is somehow
less concerning.

