
The coming automation of propaganda - donara
https://warontherocks.com/2019/08/the-coming-automation-of-propaganda/
======
im3w1l
Imagine you go to an online forum and that there is a 99.99% chance that a
given message was written by a bot. Imagine further they are sufficiently well
written you can't tell from the post itself whether it was made by a bot or
not How would people respond to that?

Well the obvious consequence is that you have to stop paying attention to
random posts. You have to depend on something outside of the forum itself to
find a list of people to pay attention to. That will be your friends and your
family. And then it will be whatever media tells you to pay attention to.

Getting out of your filter bubble will be impossible because as soon as you
step out you are faced by an onslaught of bots.

Maybe you could travel around IRL and acquire a diverse list of voices that
way. But realistically only the elite could afford that.

Actually lets be even more cynical. Say you go traveling, meet someone. You
add them on a social network. But it's not them it's a bot. And they get a few
bucks for selling your eyeballs.

But then we can even further imagine that you follow them with a fake account
too, and so the eyeballs were actually worthless.

~~~
throwaway94857
I believe the age of the anonymous internet is over. It's time for real world
identity linked accounts for participating in online conversations.

This is necessary for 2 reasons. First, it gives us confidence the people we
are speaking to are (probably) not bots.

Second, it reduces the need for online censorship beyond what is required by
law. So much of the hateful garbage posted online is only posted in the first
place because of anonymity. Remove that, we probably don't get that posting in
the first place so we can avoid the messy issue of enforced top down
censorship.

~~~
corndoge
It is a slippery slope from regulating "hateful garbage" (to be explicit, I
interpret this to include speech that is not necessarily hate speech) to
censorship of whatever the state determines to be wrongthink.

I'll take my anonymity where I can get it, thanks. And as an aside, it's
somewhat ironic you post this on a throwaway.

~~~
throwaway94857
I agree completely. I'm saying if we remove anonymity we won't need top down
censorship.

------
jointpdf
Recommended reading on this subject from RAND:
[https://www.rand.org/pubs/perspectives/PE198.html](https://www.rand.org/pubs/perspectives/PE198.html)

It covers the underlying strategy of Russian disinformation operations and
why/how it works. Mind you this is from 2016, so prior to the Mueller
investigation. If nothing else, read the blurbs in the text boxes.

~~~
hackermailman
The right wing has their own version of this, a KGB defector Yuri Bezmenov,
who has powerful (meme) videos about the coming destruction of America and how
it would take 3 decades to fulfill, all recorded in the early 80s. 'To change
the perception of reality of every American, despite their abundence of
information, nobody is able to come to sensible conclusions,... it's a great
brainwashing process that takes decades'
[https://www.youtube.com/watch?v=bX3EZCVj2XA](https://www.youtube.com/watch?v=bX3EZCVj2XA)
Yuri also claims the Soviet Union never fell, it just reinvented itself. This
is pretty standard 'alt-right' materials that people should pay attention to
if they wish to understand that reactionary wing of politics

------
joe_the_user
The idea is that something like the (formerly) OpenAI text generator could be
used to enhance "boiler room" propaganda operations.

The idea is one person could decide on a message and hundred of articles
pushing the message could be generated.

Maybe that would work. Maybe you'd see so much of this people would start to
get more selective. Maybe it wouldn't matter because people believe what suits
them rather than just being conditioned.

~~~
zxcmx
IMO the end state is way worse.

Imagine campaigns that:

Have a general idea of your life stage, interests, career, personality etc,
based on advertising profile and commercial surveillance.

Can tailor each message to the recipient.

Can a/b test the crap out of every message for target groups, optimising for
best response.

Can deliver fully customised messages in a natural voice (aka google duplex).
Conversational ai not required, but can understand some stock phrases and
responses.

The future is robots calling you, messaging you, infiltrating social media
groups, probing your personality and beliefs, finding YOUR buttons and pushing
them, repeatedly.

~~~
trqx
That sounds a lot like how current web ads works to me, only with some further
optimizations.

Am I wrong ? I hope so.

------
olivermarks
'As U.S. policymakers remain indecisive over how to prevent a repeat of the
2016 election interference' Was this ever proven? all I ever heard about was
conspiracy theories around 100k of Facebook ads and Cambridge Analytica being
accused of various things. Nothing concrete was ever seriously discussed and
the Mueller team findings were inconclusive.

~~~
hackermailman
The interference really was only the leaked Clinton emails, the facebook
propaganda is just typical propaganda nobody really pays attention to unless
it's to confirm their own bias, nobody anywhere changes their political
opinion because of something they read on facebook. I doubt Russians had any
real influence, the question is those leaked emails, which everybody wants to
know the provenance. Personally I think it was just some DNC pro-Bernie
activist staffer, who burned from the primary shenanigans decided to leak to
Assange and not Russians, (because we all know those radical DNC tech staffers
are bernie bros (now Yang shills) from SV) despite the nonsense indicators of
compromise that the FBI never saw but was only provided by the DNC themselves.
Even if you believe that was a major influencer I don't think so, as a neutral
foreign observer of US politics I saw a wild card that the Americans were
ready to put into office to 'disrupt' as a protest vote because if anybody
remembers, they were only given two choices before the disruptor, Clinton 2.0
or Bush 3.0. They responded accordingly by pushing the maniac into power just
to implode the system from what my foreign analysis indicates.

I think the most dangerous propaganda are the news propagandists from MSNBC
and other networks who pretend to be from your specific subculture/groups, and
then feed you lies and propaganda. That to me is the most dangerous, the
Russian bots? nope everybody sees through cruise control shilling but when
it's somebody from their own group in a news org and they're spouting lies
it's a lot more difficult. I'm old so back in the 1990s news propaganda was
just absolutely blatant, any kid could tell Norman General
Swartzkopf(spelling) was making up bs about 'precision bombs' we all knew they
were just landing wherever and blowing up Mother Theresa's Iraq orphanage by
accident, but now it's different. The news corps discovered they need to
recruit people from specific identities and use them as a weapon to feed lies
to their subculture.

~~~
shadowbanme
Honestly I’m still prettt unconvinced by the emails too, for the simple reason
that I’d expect Russian intelligence to have a load of information _much_ more
embarrassing than those DNC emails. If they’d shared the 33,000 “deleted”
emails off Clinton’s private server, then that would have been something.

Also I’m interested in the dog that does not bark — Chinese interference in US
politics. Does China prefer one side to the other? Which one? What are they
doing about it?

~~~
jsnider3
> Chinese interference in US politics

Check out
[https://en.wikipedia.org/wiki/Confucius_Institute](https://en.wikipedia.org/wiki/Confucius_Institute)
and
[https://www.theatlantic.com/international/archive/2011/02/of...](https://www.theatlantic.com/international/archive/2011/02/official-
chinese-propaganda-now-online-from-the-wapo/70690/)

------
patientplatypus
This is mildly related, but it's an interesting data point.

I was at a bar tonight and I chatted with a guy who works for a major cable
provider. Apparently they generate 10 petabytes of data _an hour_ on what you
watch, when you watch it etc. That's a lot when you take into account that its
all text columns in a MSQL database somewhere. I think they have enough room
to have it stashed in S3 for a week before it goes into cold storage because
otherwise it's just too bloody expensive.

What are they storing you ask? There's a whole tree on what you're
demographics are, a tree on what your device is, a tree on what you are
watching etc. And they're all cross referenced with what ad you watch, for how
long, when in the show etc.

All of this whizbang tech to find out how to make sure you spend that extra
dollar on whatever crap they want you to. I'm sure that the coming automation
is pretty horrifying, but what we have now is pretty terrible as well.

------
SubiculumCode
I read a recent article on foreignpolicy.com about a Chinese Bonds/etc rating
agency which systematically gave Democratic nations lower ratings than
authoritarian ratings. This was taken as a form of Chinese propaganda and
ignored as such...but this article makes me wonder, as much as I hate to say
it, if they are right...

------
inflatableDodo
Great. Plato's Cave is tricky enough to navigate as it is, without it being
covered in graffiti.

------
rrggrr
This is old news BUT hasn't reached it's Apex by a long shot. Candidates that
are aggressively transparent, speak with brutal truth, and accept it may cost
them an election... They will ironically be hard to beat long haul in this
environment.

~~~
i_am_nomad
In some ways, that’s what Trump was - he didn’t stick to the predictable
talking points and empty platitudes that form the language most other
politicians speak in. He spoke much more bluntly and aggressively than we were
used to. It seemed to have helped him enormously, even though the things he
actually said turned out to be pure falsehoods.

On the other hand, we have had actual honest and realistic candidates, like
Larry Lessig. And they were so honest they got squashed by their party
apparatus before they ever had a chance.

------
SideburnsOfDoom
> While the 2016 election mythologized the power of these influence-actors,
> such work is slow, costly, and labor-intensive. Humans must manually create
> and manage accounts, hand-write posts and comments, and spend countless
> hours reading content online to signal-boost particular narratives. However,
> recent advances in artificial intelligence (AI) may soon enable the
> automation of much of this work, massively amplifying the disruptive
> potential of online influence operations.

All I disagree with is that it's already semi-automated, has been for some
time.

A long time ago, I saw a tech support person with a spreadsheet full of
formulaic responses for common issues. "We're sorry you're having trouble with
_x_ , dear _y_, have you tried _z_. Please accept a $10 discount voucher for
your trouble. Thanks, Bobby Techops" kind of thing.

It has progressed, I'm sure. Why wouldn't it? And will progress more.

And the large actors have no incentive to tell everyone how far they have got.
The opposite, in fact, they will try to hide it, e.g. Cambridge Analytica
being dragged kicking and screaming into the light.

And a small army of clever educated young people isn't that expensive to a
large corporate or state actor. Call Centers depend on it.

------
roywiggins
At this point you could probably generate qanon posts with a Markov chain, no
deep learning required.

------
geargrinder
Pretty soon the internet will be the place for bots to meet and manipulate one
another. I'm sure this is happening to some extent already.

The Democrat bots are triggering the Republican bots, and the Russian bots are
fighting the Chinese bots for mindshare. Googlebot cannot save us.

------
throwaway13337
This could actually encourage filtering for better quality content.

I could totally see bots writing most of the YouTube, reddit, and Instagram
me-too-style content.

What about your average HN post?

If people want to not look like spam, they're going to have to start being
more insightful. This is good.

~~~
techntoke
I don't think automation will work or has worked. Look at emails and
robocalls. I still get plenty of spam, even if it gets filtered into a spam
folder, there is no reason I should have to peruse my spam folder but still
some important emails get labeled as spam. I believe the issue is that
corporations as a whole simply can't be trusted, and that the only solution is
decentralization with open source technology and making it easy for users to
create their own filters and control their own data.

Also, the bots I see on reddit, YouTube and Instagram generally aren't pushing
liberal agendas but right-wing. The automation I see are networks of
influencers working together to push out the same narratives.

------
IIAOPSW
I never got around to finishing the Muller Report, but from what I recall and
what was made public, Russia's influence campaign was largely traditional and
human powered.

Here's a bitter pill for everyone. If it turns out that a large fraction of
people really can have their opinion changed by bots that wouldn't even pass a
Turing test, maybe its time to rethink the idea that everyone gets to vote.

~~~
jlawson
The thing about voting is, the purpose is not to increase the quality of the
decision. The practical purpose of voting is simply to avoid civil war.
Without voting, a large group of people who are being acted against have no
choice but to use force to get what they want. Democracy is a pressure release
valve (same with free speech).

If I can't use the soapbox I use the ballot box. If I can't use the ballot
box, I use the ammo box.

So it doesn't matter how dumb the voters are. What matters if is if they can
generate violence.

~~~
IIAOPSW
>The thing about voting is, the purpose is not to increase the quality of the
decision. The practical purpose of voting is simply to avoid civil war.

Actually its both. Voting is a legitimacy mechanism (aka avoiding civil war)
and a competence mechanism (aka making quality decisions). It is much better
at the former, and typically better than historical systems (re: monarchy) at
the latter.

It is perfectly reasonable to consider a trade-off that buys us higher
competence at a loss of 10%-15% legitimacy. That's a small enough fraction of
the population that I wouldn't worry too much about them starting a civil war.
Especially if homogeneously distributed.

------
pgcj_poster
I can wait until Reddit makes me give it my social security number before I
can log in.

------
FakeComments
I’d argue that we saw automated propaganda at least three years ago — and
there’s a case Facebook et al have been engaging for at least a decade.

------
verisimilitudes
> _As U.S. policymakers remain indecisive over how to prevent a repeat of the
> 2016 election interference, the threat is looming ever more ominous on the
> horizon._

That interference is specious, I think. In any case, the US interferes in
every election it can, so is it unreasonable to believe every other country
that can interfere would?

> _You probably know enough to avoid following small, suspicious accounts on
> Twitter or browsing links to sites of which you’ve never heard._

I suppose ''surfing the web'' is truly dead, then. This seems to me like an
article written for sheep.

> _The second vulnerability is a legal system with numerous blind spots that
> lawmakers should close. Reddit is the third-most-popular social media site
> on the Internet, surpassing Facebook among American Internet users. It is
> also shockingly vulnerable, requiring only an e-mail address to register an
> account. Consequently, anyone could theoretically register an unlimited
> number of accounts and, being careful not to stand out to system
> administrators, effectively control conversations on whatever topics they
> want._

So, here's the meat of this garbage, then, in which it reveals its ulterior
motive. Yes, Reddit is the Wild West, only requiring an email instead of the
socially acceptable email, phone, government ID, etc.

> _The third vulnerability is online anonymity. While we absolutely do not
> advocate for de-anonymizing the Internet, it is now so influential over
> American society that legislators should not leave regulation to social
> media platforms alone. Congress should put more pressure on social media
> companies to ensure their users are, at a minimum, who they say they are._

Well, what I was going to write about this issue, after reading just the first
few paragraphs, should then be a response to this nonsense. As an aside, there
is no ''Who they say they are'' in a truly anonymous system.

I was going to suggest anonymity, true anonymity, as a solution to this issue.
I prefer those venues where there are no names attached and one can't tell
even if multiple messages are sent by the same person. When you have a
discussion in this way, you only judge each message on its own merit. There is
no ''Gee, this can't be a bot or government agent, since this guy has a
history.'' or any such thing. So, if it seems a great many messages have been
made in support of something that isn't usually in the venue, such as a
political candidate, say, it seems fair to regard it as astroturfing. When no
one or few have a face, it's harder to astroturf, if only because everyone
will constantly be on edge and are less likely to be swayed by a few messages.
Not only that, but since none or few of the real users have identities, you
can't target them with specialized messages or track them or other such
things.

Anyway, what a terrible article, ironically submitted by someone with an
account less than a day old. I almost wish I could make a living writing such
cheap garbage, but I figure it would eventually get to me.

~~~
_jn
Emails are optional on reddit.

------
darepublic
Another attack by the old guard trying to use the AI bogeyman to transform the
internet into a place they can control.

------
scarejunba
This is the best thing to happen to human information processing since the
printing press. The democratization of propaganda will mean that no longer
will people be slave to the voices that are loudest. Now, all ideas will fight
equally.

No longer will Judith Miller be powerful and lead us into war because we will
not trust her. Because when everyone is a liar, we will not choose one liar to
trust.

~~~
dredmorbius
Amplification technologies tend to favour the empowered and invulnerable
blessed with immunity or impunity. The power dynamic is not neutral.

~~~
scarejunba
Ah but it's not _amplification_ , it's _maximalization_. Anonymous ubiquitous
propaganda!

