
How to read and understand a scientific paper: a guide for non-scientists - ingve
http://blogs.lse.ac.uk/impactofsocialsciences/2016/05/09/how-to-read-and-understand-a-scientific-paper-a-guide-for-non-scientists/
======
neutronicus
I have a little "hack" that I find _extremely_ helpful for getting a sense of
specific research fields.

Journal articles, even review papers, are cramped for space and so tend to be
very dense. The author suggests methods for doing battle with this density,
but I suggest that, before doing that, you search for a class of document
that's allowed to be as expansive as the author desires, and whose authors
have recently struggled to learn and understand their content, and so tend to
_be_ expansive:

PhD Theses

Find out what research group published the research, find out which graduate
students have recently graduated from that group, and _read their theses_ (if
the author's command of the language of publication isn't what you'd prefer
... find another graduate student). I guarantee you it will function much
better as an introduction to what the group does than trying to parse any of
their journal publications. In particular, the "draw the experiment" step will
often be solved for you, with photographs, at least in the fields where I've
done this.

~~~
FredrikMeyer
This is _very_ good advice. I am a PhD student in mathematics, and every time
I try to learn a new area of math, I'm grateful when I stumble upon PhD theses
about that topic. They actually include their calculations.

~~~
neutronicus
Yeah, and if you _are_ actually in the field and trying to learn it (as you
are and this article's intended audience is not), a nice little bonus is that
PhD theses are usually formatted in a way that's very friendly to markin' 'em
up as you come to grips with the material.

------
startupdiscuss
This is a good guide, but I will tell you a trick that is faster, easier, and
more effective:

read 2 or 3 papers.

All that effort you would put into doing these steps? Instead, read 1 or 2
other papers that the author refers to in the beginning.

Science is a conversation. When you read the other papers, even if you don't
understand them at first, you will get a sense of the conversation.

Also, some writers are abysmal, and others are amazingly lucid. Hopefully one
of the 3 papers you read will be the lucid one that will help you understand
the other 2.

~~~
ouid
There is a huge variety in the quality of writing in scientific papers, but
most of it is bad, or at least totally opaque, so I'm not sure that 3 papers
will give you a sense of what is good, or even have a high probability of
containing a paper that you can use as an entry point, although I think your
premise is correct.

Probably the best evidence that a paper is a good entry point is whether or
not the author cared about the abstract. A lot of scientists treat it as a
chore, picking some key points from the premise, methodology, and conclusion
sections, and haphazardly pasting them together into a miniature version of
the paper. An abstract is a sketch of your argument. It's supposed to be how
the author thinks about the work they are doing, in terms of how it relates to
the work everyone else is doing. Look for an abstract which presents an
argument in plain english and isn't afraid to give a little background or
motivation. It might take dozens to find one though.

~~~
jacobolus
Personally I find abstracts close to useless, and just skip them entirely.
I’ve never found a particularly close correlation between what an abstract
said and how interesting/informative/well written the rest of the paper was.
YMMV.

------
closed
I love how simple and clear this post is.

As a kind of weird aside, if anyone ever emailed me about any of my journal
articles, I would 100% respond to them (assuming they weren't a machine). I
think most of my colleagues would do the same (except for articles featured in
a newspaper, which might garner a lot of weird emails).

~~~
kkylin
Me too, most especially if the email is from a student. I imagine the same
goes for many of us who write research papers.

------
lumisota
Keshav's "How to Read a Paper" [1] is a good guide, though perhaps less in the
"for non-scientists" camp.

[1]
[http://ccr.sigcomm.org/online/files/p83-keshavA.pdf](http://ccr.sigcomm.org/online/files/p83-keshavA.pdf)

------
choxi
> As you read, write down every single word that you don’t understand. You’re
> going to have to look them all up (yes, every one. I know it’s a total pain.
> But you won’t understand the paper if you don’t understand the vocabulary.
> Scientific words have extremely precise meanings).

That's a great tip. I've found that a lot of papers aren't necessarily
complicated, but the vocabulary is unfamiliar (but you experience the same
sense of confusion with both). It's interesting that we often conflate
complexity with unfamiliarity, my reading comprehension abilities improved
quite a bit by understanding the difference between the two.

------
glup
I don't understand the opposition to abstracts: dense means high information
content, so if you know the field you can learn a whole lot (like whether you
should read this paper or another one).

~~~
PeterisP
Abstracts often are misleading.

They're useful to decide whether you should read this paper or another one,
but they're often _not_ useful to get a summary of what exactly the paper
actually achieves. Often the abstract will imply a more interesting result by
leaving out key aspects and limitations (which are detailed in the paper and
its conclusions) that significantly change the impact of the paper, the
abstract often is more like an advertisement for the paper than an effective
summary. I mean, it _may_ be, but if I'd read just the abstract and go away
thinking, "oh, so now there's a way to do X", I'd often be wrong.

~~~
TazeTSchnitzel
I recently read a paper whose abstract seemed to imply to me that its content
was much more technical and specific than it actually turned out to be, which
was disappointing. It was more useful in telling you the particular area of
research than summarising its content.

------
ChuckMcM
Oh this is awesome, well presented and clear.

A couple of notes, generally if you email the author of a paper they will send
you a copy. Scholar.google.com can be used to evaluate the other papers
referenced, highly cited ones will be 'core' to the question, less highly
cited ones will address some particular aspect of the research.

For any given paper, if it cites one or two seminal papers in the field, you
can build a citation cloud to create what is best described as the 'current
best thinking on this big question'. You do that by following up the citations
and their citations for two or three hops. (kind of like a web crawler).

With something like sci-hub and some work on PDF translation, it should be
possible to feed two or three 'seed' papers to an algorithm and have it
produce a syllabus for the topic.

------
deorder
I usually first start reading or glance over papers (and non-story books) from
the end to the beginning before I read it the other way around. This has the
following benefits for me:

\- By knowing about the conclusion first I will better understand the
motivation and why certain steps are being taken.

\- I find out sooner if the paper (or book) is something I am looking for.

I like to read papers unrelated to my field to learn new thing to apply. To be
honest, some papers still take me a long time to understand because they
usually assume you already are researching the topic (for ex. certain terms,
symbols and/or variables that are not being defined).

------
nonbel
There is a difference between reading and studying a paper. Many papers I just
check the abstract for claims of A causes/correlates B (ie it is a "headline"
claim), and look for a scatter plot of A vs B (it is missing).

Then I do ctrl-F "blind" (can't find it), ctrl-F "significance" (see p-value
with nearby text indicating it has been misinterpreted). Boom, paper done in
under a minute. There is really no reason to study such papers unless they
have some very specific information you are searching for (like division rate
of a certain cell line or something).

~~~
Denvercoder9
This only works for a very small subset of studies in a subset of scientific
fields.

~~~
nonbel
Agreed, the OP was about medical research though, where it does apply.

------
olsgaard
About identifying "The Big Question", I have a story from my days as a
graduate student, where I failed to do so.

I was asked to help on a project that needed to identify humans in an audio
stream. During my literature review, I came across the field of "Voice
Activity Detection" or VAD, which concerns itself with identifying where in an
audiosignal a human voice / speech is present (as opposed to _what_ the speech
is).

I implemented several algorithms from the literature and tested it on the
primary tests sets referenced in papers and spend a few months on this until I
finally asked myself "What would happen if I gave my algorithm an audiostream
of a dog barking?"

The barking was identified as "voice".

As it turns out, the "Big Question" in Voice Activity Detection is not to find
human voices (or any voices), but to figure out when to pass on high-fidelity
signals from phone calls. So the algorithms tend to only care about audio
segments that are background noise and segments that are not background noise.

------
sn9
>I want to help people become more scientifically literate, so I wrote this
guide for how a layperson can approach reading and understanding a scientific
research paper. It’s appropriate for someone who has no background whatsoever
in science or medicine, and based on the assumption that he or she is doing
this for the purpose of getting a basic understanding of a paper and deciding
whether or not it’s a reputable study.

Better advice intended to make _layman with zero background in science_ become
more scientifically literate would be to tell them to read some textbooks.

Later on in the article, she tells people to write down each and every thing
you don't understand in an article and look them up later. And this is
excellent advice for people with a background equivalent to an advanced
undergraduate or higher, but for people with zero background it would be
better to read some textbooks and get yourself a foundation.

Honestly, even when I was in grad school in neuroscience, I asked around for
advice on reading papers and the surprisingly universal response from other
grad students was that it took 2 years to become reliably able to read and
evaluate a research paper well. And this is 2 years in a research environment
with often weekly reading groups where PIs, postdocs, grad students, and some
undergrads got together to dissect some paper. These reading groups provided
an environment in which you had regular feedback on your own ability to read
papers by seeing all the things those more experienced than you saw and that
you missed. A paper that took me 3+ hours of intense study would take a
postdoc a good half hour to get more information out of.

I feel like this article makes reading articles well seem a lighter
undertaking than it really is. It's really no wonder we see studies
misinterpreted so often on the internet, where people Google for 5 minutes and
skim an abstract.

~~~
yskmt
> it took 2 years to become reliably able to read and evaluate a research
> paper well

This completely coincides with my experience. When I started grad school, it
took me a few hours to read one paper, and I probably understood only 50% of
the materials even though I had some foundations in the research area from my
undergrad studies.

Reading textbooks is a great advice. Then one can start reading some review
papers in the area to get some more depth in his/her knowledge. I think the
difficulty is that it is hard to find good textbooks and review papers for the
subject that one is interested in, especially when the subject is in a niche
field.

------
kronos29296
As a student who needs to read research articles for my project, this article
gave some new ideas on how to approach those long boring and cryptic pieces of
text that just take days to understand. Thanks to the person who posted it.

------
luminati
A couple things I try to do when reading research papers, inspired by these
two amazing [b|v]logs.
[1][https://blog.acolyer.org/](https://blog.acolyer.org/)
[2][https://www.youtube.com/user/keeroyz](https://www.youtube.com/user/keeroyz)

I try to paraphrase the paper into a Acolyer like 'morning paper' blog post on
evernote while mentally I am directing a 'two minute paper' video on the paper
:)

------
DomreiRoam
I would like to have a digest or an overview written for a IT practitioner. I
did go SC/IT conference and enjoyed the talks and I noticed 2 things: 1/ You
learn new things and new approach that can bring value to our job 2/ It seems
that the research sector discover stuff that are already known in the
industry.

I think it would be great to have a journal/blog that would construct a bridge
between the industry and the university.

------
yamaneko
This suggestion by Michael Nielsen is also very good:
[https://news.ycombinator.com/item?id=666615](https://news.ycombinator.com/item?id=666615)

------
pitt1980
What's odd to me, is that lots of professors have blogs in which they write
quite a bit in plain language that doesn't require an instruction manual in
order to be read

------
syphilis2
Why don't the authors do these 11 steps for us?

~~~
danielalmeida
Because they are not writing for non-scientists.

~~~
pitt1980
all scientist were non-scientists first though, correct?

Look, I get that there's some natural professional context and lingo that goes
into these things, but for all the angst that goes into what esteem that
population at large holds up the science community

making their work more accessible to both novices and interested outsiders
would be a nice step in the right direction

~~~
danielalmeida
I agree with you. To put it simply, papers are optimized for the scientific
community and making them "more accessible" to outsiders has a cost. I'd
settle for better writing and presentations within the scientific community
for now. If you ever find researchers that blog about their research in simple
terms, I think it's safe to assume they're using their personal time to do
that (I know of very few; Andy Ko [1] comes to mind).

[1] [https://medium.com/bits-and-behavior](https://medium.com/bits-and-
behavior)

------
amelius
I'd like an answer to: how/where to ask the relevant community a question
about a scientific paper.

------
minademian
this is a great guide. i wish more writing on the Internet has this blend of
substance, message, tone, and grit.

------
apo
Sensible advice overall, but I completely disagree with these:

> Before you begin reading, take note of the authors and their institutional
> affiliations.

and

> Beware of questionable journals.

Institutional affiliation and journal imprimatur should have no bearing in
science. These are shortcuts for the lazy, and they introduce bias into
evaluation of the paper's contents.

Even more than that, dispensing advice along these lines perpetuates the myth
that scientific fact is dispensed from on high. If that's the case, just let
the experts do the thinking for you and don't bother your pretty little head
trying to read scientific papers.

If the author's approach to reading a paper only works by checking for stamps
of approval, maybe the approach should be reconsidered.

~~~
burkaman
They aren't shortcuts for the lazy, they're shortcuts for non-scientists who
aren't capable of fully evaluating the science alone. If you're capable of
objectively peer reviewing a paper, you're not the audience of this article.

~~~
apo
> They aren't shortcuts for the lazy, they're shortcuts for non-scientists who
> aren't capable of fully evaluating the science alone.

It's a shortcut fraught with potential for deception, as even a casual glance
through a site like Retraction Watch will demonstrate:

[http://retractionwatch.com/](http://retractionwatch.com/)

I'm not sure what you mean by "evaluating the science." A scientific paper
should present a hypothesis, the author's best attempt to disprove the
hypothesis, and an interpretation of the evidence gathered in the processes of
testing the hypothesis. There's going to be a back-story, and it's likely to
be quite involved.

The article does a good job of presenting a method for navigating a paper on
this basis. I don't see what checking credentials adds to the process. On the
contrary, it may do harm.

~~~
semi-extrinsic
While we may find the high profile cases featured on Retraction Watch mainly
in high impact journals, that's precisely because unscrupulous people deem
these journals _worth it to cheat to get into_. Nobody cheats to get their
paper into the International Journal of Architecture, Engineering and Nursing
Science - because it and it's ilk are utter pieces of crap that will accept
anything, up to and including randomly-generated text and pro-Sri-Lanka-
highly-racist-UFO-conspiracies (real example). Teaching laypeople to avoid
these is a very good idea.

