
The spread of low-credibility content by social bots - infodocket
https://www.nature.com/articles/s41467-018-06930-7
======
hlecuanda
I have been feeling depressed for a few days pondering the general credibility
crisis that prevales over human communications as of late.

The spread of low quality content or plain disinformation is just an aspect of
the problem. We have those who stand to gain from, politically or financially
from influencing the global conversation to their advantage, and we've seen an
endless parade of examples, that have elevated Astroturfing and manipulative
deceit from a curiosity into an exact science and art form: Cambridge
analytica, Facebook and their HR firm, Bannon and Breitbart, the Russian troll
farms, twitterbots and fake profiles, the inexplicable fact that FB knows and
tells advertisers exactly how many times their ad was shown and bills them
accordingly, but they need to develop new technology to figure out exactly how
this very same features were used to needle in an election, sow discord and
amplify animosity.

OTOH I can read and hear all sorts of opinions from all kinds of experts on
every field who I no longer trust, since I don't know if they are using their
audience for profit and ever so subtly change the perception of those who
listen: it puzzles me to no end that nowdays Microsoft is considered open
source's best new friend, the same Microsoft that no more than 15 years ago
was covertly funding SCO's Lawsuits vs Linux.

I feel like advertising has seemed to the very fabric of human communications,
not to better humanity but for the short term gain and selfish goals of a few
who can afford such services.

The internet used to be an Electronic Frontier where everybody could be who
they really were and speak their minds. Now it's a poisoned cesspit where
everybody lies about everything and those who do it better get to sell you
stuff. <//3

~~~
scottlocklin
I've made this point before; there are 5 PR people for every reporter in the
US.

[https://muckrack.com/blog/2016/04/14/america-now-has-
nearly-...](https://muckrack.com/blog/2016/04/14/america-now-has-nearly-5-pr-
people-for-every-reporter-double-the-rate-from-a-decade-ago)

That and the fact that various parts of the 'expert class' has largely
discredited itself; from the replication crisis to the death of physics as a
field .... we don't need to blame bots; the trust level is low.

~~~
bilbo0s
> _there are 5 PR people for every reporter in the US..._

And the PR people are, in general, far better paid.

------
sjeohp
The assumptions are:

1) third party fact-checkers (eg. snopes) are credible and reliable.

2) the third party "Botometer" (machine-learning model) is valid and was
trained on an accurately classified corpus.

This type of study always relies on the elevation of someone's subjective
assessment (eg. "this account is a bot/human", "this fact-checker is honest")
to the status of objective truth.

It is a serious weakness when studying subjects loaded with political
consequence, wherein well-resourced stakeholders are constantly attempting to
shape competing narratives by any means available to them.

~~~
ImaCake
EDIT: just to be clear, I think it's important to note that Nature and these
authors likely have a bias towards what they consider more truthful news and
that will be reflected in their choices of material and tools for this study.

>2) the third party "Botometer" (machine-learning model) is valid and was
trained on an accurately classified corpus.

For anyone curious enough to follow up on how effective their "botometer" tool
was, the methods section [0] details what they used and links to said tools.
The language used is simple enough for most people to follow. I am not great
at following their statistics, but the botometer seems to be at least decently
accurate. Certainly, figure 2d seems pretty convincing to me for validating
the botometer.

>1) third party fact-checkers (eg. snopes) are credible and reliable.

The methods section talks about how their model works with Onion articles. The
great thing about The Onion is that it is a perfect positive control i.e it is
always false. The authors state the following in the methods section:

>Many low-credibility sources label their content as satirical, and viral
satire is sometimes mistaken for real news. For these reasons, satire sites
are not excluded from the list of low-credibility sources. However, our
findings are robust with respect to this choice. The Onion is the satirical
source with the highest total volume of shares. We repeated our analyses of
most viral articles (e.g., Fig. 3a) with articles from theonion.com excluded
and the results were not affected.

0\.
[https://www.nature.com/articles/s41467-018-06930-7#Sec8](https://www.nature.com/articles/s41467-018-06930-7#Sec8)

------
olivermarks
I think the massively successful clickbait junk/fake news purveyors like
Taboola and Outbrain are equally damaging because they are openly publishing
garbage on the sites people visit to read 'news' and form opinions

------
thrower123
Wiping out the bots would absolutely destroy the user counts of most social
networks, so it's just not going to happen.

On the other end, there are a lot of formerly credible sources of information
that have stepped in it badly and repeatedly, and tarnished their aura of
authority. The fourth estate is looking more and more bought and paid for, and
the academy has their own issues there, not to mention a mother of a
reproducibility problem.

~~~
thoughtexplorer
What % are bots?

~~~
thrower123
With Twitter and Instagram, it feels like there are more bots than real
people. Facebook, I'm less sure of, but with as many random friend requests as
I get from unknown "people" with half-baked profiles and no connections, it's
happening there too.

------
webwanderings
> curbing social bots may be an effective strategy for mitigating the spread
> of online misinformation.

Why not turn them off entirely? The least of self regulation these popular
social media companies can do for humans.

~~~
asaph
Not all social media bots are malicious. And not all social media bots are
posting content from low credibility sources.

~~~
eiaoa
> Not all social media bots are malicious. And not all social media bots are
> posting content from low credibility sources.

The collateral damage from banning them all may be worth it, if they're a
major means for pushing low credibility content and disinformation.

~~~
robotresearcher
Eliminating all the internet bots would be the mundane real life first step of
the Butlerian Jihad of the Dune universe. In the stories, they eliminate all
machine computers in order to prevent AIs destroying society, and go back to
human computers.

The idea that AIs (broadly defined to include simple bots) are net more
trouble than they are worth has been kicked around in literature forever. It’s
fun to live in the age where we get to wrestle with it in practice.

------
elvinyung
Shameless self-promotion: I wrote down some thoughts a while ago about this:
[https://www.notion.so/The-Hyperreality-of-Big-Data-
ae0aebb35...](https://www.notion.so/The-Hyperreality-of-Big-Data-
ae0aebb358c34d8a9b01eb61d3b77b48)

It almost feels like the only solution (for one aspect of the problem) is for
the "good guys" to outmeme the "bad guys". But of course, that's much, much,
much easier said than done.

~~~
ghthor
This is a very good short term option, that I hope others have been working
on. My personal actions have been to encourage my family member to stop using
social media, point them to top quality content producers like Joe Rogan &
redlettermedia. And most importantly, I help them to calm down, the
information available is exhausting, but the world around us isn't changing
that fast, I remind them to use that as a reference, talk to people I person,
get involved in local politics, take a nature walk, life is short.

~~~
WhompingWindows
Is Joe Rogan truly the highest quality content producer? I've seen/heard some
of his content, it's quite diffuse wrt information density. Do I really want
to listen to 2.5 hour podcast to get what I can in 30 minutes from other
podcasters? I also heard him asking Neil deGrasse Tyson about whether the moon
landing was faked with a lot of skepticism in his voice...

~~~
ghthor
I think hes a quality content source at the very least. You can usually find
an abridged or best of for the most interesting podcasts. And you can reliably
fall down to a complete unedited interview to get more balanced observation 9f
his or guests opinions. He sticks out to me as one of my most interesting and
he shares similar levels of openness and drive for the truth compared to the
other independent producers I am aware of it frequent. There are other quality
producers that do better jobs at interviewing different niches.

The truth is always changing, entertaining controversial hypotheses is
reinforcing or exposing the truth. As my awareness, research, observational
and social skills have expanded, so too has my worldview. If it we represent a
worldview as a polygon, where each time you accept a new story into your
worldview you add that point as a new vertex to your shape. What happens as
you validate and accept more points outside your ahapn? Your total perimeter
likely increases is length, ever increasing the total number of potentially
mysterious stories available to you. The larger your worldview, the larger set
of "what ifs" become available to you, "what ifs" produce larger impacts
requiring larger and larger changes to your worldview. If you dont entertain
the what ifs and discuss potentially impactful what ipfs, the more cataclysmic
the potential change you'll have to make.

He likely asked In jest, but I feel it was probably healthily for Niel to
retell the facts for himself as it was for Joe to entertain the unlikely idea,
but incredibly impactful the story is if it's not the whole truth.

------
706f6f70
It strikes me as somewhat ironic that Nature would public this given that they
recently tweeted:

> Editorial: The US Department of Health and Human Services proposes to
> establish a legal definition of whether someone is male or female based on
> the genitals they are born with. This proposal has no foundation in science
> and should be abandoned. [1]

That last sentence just absolutely boggles the mind and I'm not entirely sure
how this journal retains any semblance of reputation when its editorial board
is so clearly willing to put ideology over science.

\---

[1]
[https://twitter.com/nature/status/1064694083090812928](https://twitter.com/nature/status/1064694083090812928)

~~~
yesenadam
It doesn't 'boggle' my mind. It would be helpful, I suppose, if you could
explain how the last sentence shows they are 'so clearly willing to put
ideology over science' \- nothing could be less clear to me, although I'm far
from expert. Otherwise it will seem like that's precisely what you're doing
yourself, which I suspect is why you've been heavily downvoted.

~~~
tree_of_item
The last sentence shows that they, Nature, are seriously suggesting that there
is something wrong with defining sex based on observable sexual
characteristics. This shows they are willing to put ideology over science: the
only reason to suggest such a thing is ideology.

~~~
tzs
> The last sentence shows that they, Nature, are seriously suggesting that
> there is something wrong with defining sex based on observable sexual
> characteristics

No. They are suggesting that there is something wrong with defining it based
on just one particular observable characteristic.

There are numerous characteristics in humans that come in two forms or
varieties, one generally associated with males and one generally with females.
Many people end up with some characteristics of the form generally associated
with males and with some characteristics generally associated with females.

For example, there is a response in the hypothalamus to the pheromone
androstadienone that is different in males and females, and can be recognized
on an MRI scan.

Another example is how "male" and "female" brains perform visual and spatial
memory tasks, such as imagining how a shape would look when rotated. Males
generally are better at this than females, and brain scans show that when
doing this task males are using different parts of the brain than females use
--male brains approach this task differently than do female brains.

Some people with male genitals have female androstadienone response and female
visual and spatial processing, and some people with female genitals have male
androstadienone response and male visual and spatial processing. In
particular, studies have found that transgender people are often this way,
with their brain having the responses of the gender they identify as rather
than the gender their genitals suggest.

------
dstroot
“Successful low-credibility sources are heavily supported by social bots.
These results suggest that curbing social bots may be an effective strategy
for mitigating the spread of online misinformation.”

No, really?

Sorry - go ahead and vote me down. I couldn’t resist. Sometimes science
uncovers facts that seem obvious.

------
thoughtexplorer
Stuff like this is scary. Check out this video about the caravan and migrants
climbing the border wall. Strangely edited. Presented as breaking news. Music
as though it's a suspenseful movie. Yet looking at the comments everyone seems
to be buying into it.

[https://www.youtube.com/watch?v=2GuuNzR_AZg](https://www.youtube.com/watch?v=2GuuNzR_AZg)

It has 2M+ views in just a few days.

~~~
jimrandomh
Neither the comments nor the view count are necessarily legit; both can be
created by bot traffic.

