

When Science Becomes News, The Facts Can Go Up In Smoke - jboynyc
http://www.npr.org/blogs/13.7/2014/05/02/308926616/when-science-becomes-news-the-facts-can-go-up-in-smoke?ft=1&f=114424647

======
garenp
Humor aside, it really concerns me that so much of the information in
mainstream science news articles can mislead readers.

More worrying is that it's often not possible to persuade someone who is
swayed in the wrong direction, because they just don't have the base level of
knowledge to allow it.

On this point I almost want to say that every person who graduates from High
School ought to have gone through a rigorous class in logic and another in
statistics. It's all well and good to say everyone should have "critical
thinking" skills, but you can't get there without some pretty solid
intellectual tools.

~~~
spikels
Once someone becomes convinced of something it is often very difficult to
change their mind despite logic and even strong evidence. I have noticed this
even in my own thinking despite being aware of it.

You become invested in an idea by your own thoughts and actions as well as
statements to others. Then to change your mind you have to admit to yourself
that you are wrong as well as to others who may still hold the mistaken
belief. This can be unpleasant so many people just avoid it.

Keep in mind that very few things are black and white so there is always some
"reasoning" that can defend the incorrect belief even if it is very unlikely.
And there are often people who benefit from the incorrect beliefs (you may
disagree but I think the $5 billion US vitamin/supplement industry is a good
example).

While if everyone had excellent "critical thinking" tools it should help I'm
skeptical this would overcome natural human biases such as pride. And perhaps
it is a good thing that at least a few people disagree with any consensus no
matter how strongly supported by logic and evidence.

~~~
epochwolf
> You become invested in an idea by your own thoughts and actions as well as
> statements to others.

While I've observed this is true for most people, it isn't true for everyone.

I'd rather be correct than be right. It's one of the characteristics of people
with a Myers-Briggs Type Indicator of INTJ. (And perhaps a few others.) I've
also heard it called "strong opinions, weakly held" before.

I'm pretty sure both those with this characteristic and those without it spent
a fair amount of time being greatly annoyed with eachother.

~~~
bashinator
Related to your topic, there's a large body of criticism against MBTI. My
understanding is that it's basically not used by anyone doing professional
psychological work.

[https://en.wikipedia.org/wiki/Myers-
Briggs_Type_Indicator#Cr...](https://en.wikipedia.org/wiki/Myers-
Briggs_Type_Indicator#Criticism)

------
hyperion2010
In cases like this I think it would be helpful if the reporter (probably with
the help of the scientist) laid out all the possible explanations for this
finding. In this case it would include 'people with a smaller nucleus
accumbens are more likely to smoke pot.' Showing people all the possible
interpretations of a scientific result is hard, even for scientists. We often
miss interpretations that in retrospect seem obvious given the data. Science
news needs to report about the science first and about any human elements
second.

~~~
SoftwareMaven
I don't know; I bet that happens pretty well now: Scientist says, "well, foo
could be because A, B, or C; we just don't know." Journalist asks, "what would
happen if B?" Scientist says, "well, it's unlikely, but D could happen".

Newspaper reports, "Scientists Show D Happens Because Foo Causes B".

The real problem is there is no disincentive to ignore the truth and publish
sensationalist drivel, but there are definitive incentives to do so: you get
more readers, you tend to embed yourself in whatever philosophical echo
chamber you are trying to reach, and, as a result, you get more money. Until
there are disincentives, the problem will never go away.

------
devindotcom
As a reporter I've found that striking a balance between accuracy and
reasonable summary can be extremely difficult, and sometimes a headline that
has to be less than 45 characters or so ends up being more suggestive than it
should. But I hate this kind of story as much as others do, and I'm proud to
say I've rejected quite a few stories the editors wanted because of salacious
write-ups, and I'm careful to use the language of the study itself or check
with the researchers, almost all of whom are happy to discuss their work. We
also have a phenomenally well-informed science/space editor where I work, who
has been in the press since they were printing the newspaper in the basement
of the building. So that helps.

It's worse to have to be the guy who points stuff like this out on Facebook,
where you end up sounding like the science equivalent of a grammar nazi - but
I've grown to be fine with it, since there's much less room for interpretation
in the results of a limited and specific piece of research.

------
jmzbond
I don't think this problem can be addressed without talking about the
fundamental difference in incentives between the groups.

Reporters are incentivized to get the story to sell. Especially as there are
more & more freelancers, competition is becoming intense. And let's face it,
by nature, we as consumers of information are drawn to the outlandish and
sensational. There was an HN a few weeks ago about someone who put out fake
crazy headlines and got crazy CTR on Twitter.

Scientists are incentivized to be objective in finding the truth. Scientists
avoid making claims about causation until the last possible moment just so
they can be sure all the variables have been controlled for and the results
are not outliers.

I don't have any well-thought through answers... but thoughts?

~~~
waltherg
Have you ever worked in scientific research yourself?

Scientists sell their story and their version of The Truth just as much as
reporters sell their news stories. This image of the objective scientist that
prevails outside of academia is mostly false.

If I remember correctly, you'll find more material to think about here (see
researching what your boss wants you to, researching what funding agencies
want you to, etc. etc.):

[https://news.ycombinator.com/item?id=7763737](https://news.ycombinator.com/item?id=7763737)

[https://news.ycombinator.com/item?id=122106](https://news.ycombinator.com/item?id=122106)

------
Florin_Andrei
You're a reporter. Your career is helped if your articles sell a lot. Of
course you have an incentive to put a bit of "kick" in the title, or even in
the content of the article, if that will help your baseline.

Money-driven outcomes are not always optimal.

~~~
spikels
I think blaming "money" for so many problems is wrong. Like any writer
reporters want to see their work published and read. This is what it means to
be a successful reporter. And while it may eventually lead to more money given
the very low compensation of the vast majority of reporters if their behavior
was really driven by money they would change professions.

~~~
Florin_Andrei
Career? Status? Worrying about the kids' college fund? They're all related.

Regardless of how you want to split this particular strand of hair, the bottom
line is that media's main incentives nowadays are NOT to inform and educate
the public.

------
Vilkku
StarTalk Radio (hosted by Neil DeGrasse Tyson) has covered this topic together
with Miles O' Brien in the two most recent episodes. I'm a fan of the show,
but I can imagine some might find it too humorous for their tastes.

[http://www.startalkradio.net/show/reporting-on-science-
part-...](http://www.startalkradio.net/show/reporting-on-science-part-1/)

EDIT: By "this topic" I meant science reporting, not marijuana.

~~~
gtremper
I listen to StarTalk also and really enjoyed these segments. It seems a lot of
these problems have been caused by the transition from "News as information"
to "News as entertainment".

------
jasontsui
To some of the older folks on HN - is this a new problem?

The last few years have brought on a whole different type of newsmedia hybrid
(the buzzfeeds, huffpos and gawkers) organization that is driven primarily by
clicks and do not hold themselves to the standards of traditional print news.
While there were dubious options on paper before (Daily Posts, National
Enquirers), the internet is far greater venue for propagating bullshit with
clickbait headlines. Some of the newer sites I'm seeing people post on
Facebook have skipped the truth part altogether, they go straight to
fabricating stories. TV has gone the same direction with news-entertainment.

I'm pretty concerned. When its too hard to find signal in all the noise, I'm
afraid folks will give up altogether. With Buzzfeed putting out longform
articles and NYT putting up quizzes, its already hard to discern who cares
about delivering real news and who will do anything for clicks.

But maybe I'm just young (25), and people have always found echo chambers, and
yellow journalism is always something we've had to wade through to find the
facts. What do you guys think? Has anything actually changed?

~~~
cafard
To varying degrees this has always been the case. Publishing and distribution
is just easier.

------
gizmo686
What makes this problem even worse is that articles almost never cite the
original research (or provide a link). If they did this then it would be far
easier to look at the original paper. Just reading the abstract of most papers
is often enough to give the lay person a relativly good understanding of the
scope and actual result of the research.

------
buckbova
I think the point should be, this study should not have been published at all.
The sample size was way too small with results open to misinterpretation.

After a scan, I think the Boston Globe article is well written.

[http://www.bostonglobe.com/lifestyle/health-
wellness/2014/04...](http://www.bostonglobe.com/lifestyle/health-
wellness/2014/04/15/casual-marijuana-use-creates-brain-changes-new-report-
shows/X1cN8A7h5pOVJkeYkXTXlJ/story.html)

The title is "Study finds brain changes in young marijuana users". Maybe it
should read "differences" instead of "changes".

~~~
idm
The sample size isn't necessarily the problem. For example: if we wanted to
compare 20 cannabis users to 20 non-users, and we measured whether or not they
felt "high", I bet our total sample of n=40 would be plenty to show a strong
effect of pot use on "feeling high," it would be statistically significant,
and it would not be a distortion of the truth at all.

This is an interaction between reporters and exploratory research. Statistical
significance works best with confirmatory research, in which there is an a
priori hypothesis, a set of falsifiable predictions, and a specific
experimental design that manipulates just one or two variables in such a
manner as to potentially falsify the predictions. However, confirmation is
only half of science.

Before the confirmation stage, there should always be exploration, and that's
what this study represents. "Let's put cannabis users in an MRI and see what
happens!" It's impossible to draw conclusions from exploratory research in the
same way we do from confirmatory research. The main problem is that there's
usually no hypothesis, nothing to falsify, and therefore no statistical test
exists to help sift through the results.

However, significance testing can still suggest whether an exploratory finding
is deviant. If one group mean is different from another group mean, it's
totally fair to report that difference. I didn't have to dig too deep into the
original article to find out the authors weren't making any outrageous claims.
It's merely that journalists reported the exploratory results as if they were
the generated by a confirmatory, falsifiable hypothesis. In fact, the
scientists were just reporting an interesting difference they observed.

Anyway, the point is not that the study should not have been published at all.
The study was fine, the work is valuable, and the reporting was
overzealous/misinformed.

Edit: updating to add that the scientists did have some ideas about what kinds
of differences they were expecting, consistent with animal models. I might
venture to say they had some clearly falsifiable hypotheses, too. The science
is fine, and so are the scientists. It's the reporting that is the problem.

------
SixSigma
> Lots of people smoke pot. They do so, presumably, because it affects their
> brains, and not despite that fact. It would be astonishing and inexplicable
> to find that getting high didn't bring about changes in the brain. But are
> those changes lasting? Are they permanent? We don't know and we'd like to
> know.

The study didn't even find that the brains had changed at all, just that they
were different.

As the sample was so small, they could just have well concluded that brown
hair made a difference or people who prefer broccoli to cheese are more likely
to smoke pot.

------
daveslash
I use this to try to show people what I mean by "correlation is not causation"

[http://twentytwowords.com/funny-graphs-show-correlation-
betw...](http://twentytwowords.com/funny-graphs-show-correlation-between-
completely-unrelated-stats-9-pictures/)

~~~
baddox
Or directly from the source:
[http://www.tylervigen.com/](http://www.tylervigen.com/)

~~~
daveslash
Thank you.

------
tom_jones
I've seen many a publication where the scientist urge restraint in
interpreting results, but then the newspapers completely ignore their pleas
and go instead to proclaim "Study of 10 subjects proves X could make you
immortal!". It's irresponsible journalism.

------
at-fates-hands
It's quite ironic for NPR to say some news outlets twist facts and headlines
to get a point across. Considering the main stream media does this on a daily
basis and nobody bats an eye about it.

If you want to talk about misreporting something, you should start there, not
a few articles on people casually using weed.

~~~
TaylorAlexander
Actually, this article is great because it proves that most media distorts
facts without venturing into controversial territory (the science paper
clearly did not say what the articles said it did). This is as good an example
as any on the phenomenon you mentioned, and it's a good topic we all remember
and may have been convinced by. My parents and I have already had a discussion
about it, where they were convinced that marijuana was much worse than
believed, and I remained open to the idea that there may be merit to the study
they heard about. But this is an article many people were convinced by because
it fed into existing bias well, which is exactly the kind of thing we need to
expose.

We all know for example that FOX News distorts facts, but if you go into a
conversation calling them out, you will lose many people who will claim you
are biased. By first pointing out specific, provable stories that have been
misreported by all media, you can get a person to believe that news is being
poorly reported. Then, after you have convinced them of that, and that it is a
problem, you can say "hmmm... now I wonder who is most guilty of perpetuating
this." You'll find FOX news in there of course, but if you actually look you
will find that all corporate-owned media has turned into fact-indifferent
"news entertainment".

Also, it isn't irony to cover something specific instead of the broader issue.
It might be ironic for Fox to cover the epidemic of misreporting in the media,
but even that would actually be somewhat expected behavior given that they
keep insisting they are the only fair network.

------
mnw21cam
Relevant XKCD comic, as required in these situations:

[http://xkcd.com/552/](http://xkcd.com/552/)

