
Science is often flawed. It's time we embraced that - simonsarris
http://www.vox.com/2015/5/13/8591837/how-science-is-broken
======
deadgrey19
Academic publishing is a conversation between academics. No one paper is truly
authoritative and only with time will the highly cited well regarded
contributions make themselves known within the academic community (and
eventually outside of it too). The time horizon between publication and
"impact" for most academic work is usually (or should be) in the range of
10-20years. In the rush for funding and "impact" university PR machines (and
scientists too) are forced to make flashy statements about the latest new
sensational, tenuous at best implications of their findings. Boring, rigorous,
incremental work has no place in a media driven funding cycle. This is a real
problem and one we need to think about how to resolve.

Having said that, scientists are not perfect either. As an author, problems of
honesty and reproducibility in science trouble me greatly. That's why on our
recent papers
([http://www.cl.cam.ac.uk/research/srg/netos/qjump/pubs/2015-n...](http://www.cl.cam.ac.uk/research/srg/netos/qjump/pubs/2015-nsdi-
qjump.pdf) and
[http://www.cl.cam.ac.uk/research/srg/netos/musketeer/pubs/20...](http://www.cl.cam.ac.uk/research/srg/netos/musketeer/pubs/2015-eurosys-
musketeer.pdf)) my coauthors and I have gone out of our way to make the
original unprocessed data sets and detailed experimental descriptions easily
available. If you take a look inside of our papers, you'll find that all of
the substantial figures in the papers are clickable giving direct access to
original datasets on our websites. Although not perfect, this is a start down
the road of more easily testable and reproducible work. To our great surprise,
only a few weeks after publication we've already got requests from people
trying to reproduce the results in different environments. This suggests to me
that this kind of approach both can work, and really should become the minimum
standard in academic publishing.

~~~
wnevets
> In the rush for funding and "impact" university PR machines (and scientists
> too) are forced to make flashy statements about the latest new sensational,
> tenuous at best implications of their findings.

in a lot of cases its worse than just wanting funding. A lot of companies are
funding "studies" and getting them published so they can sell their latest
revolutionary product.

------
denzil_correa
I think the issue stems from a fact that most people overestimate the worth of
a peer reviewed article. I like to look at it a bit differently.

A research problem is a like a discussion where each peer reviewed article
provides __one discussion point __towards the solution. It by no means is
entirely self encompassed but at the same time, it doesn 't mean one should
discard it. The solution to the research problem isn't entirely contained in a
peer reviewed article. Rather, it is one data point towards making an impact
towards the larger problem. A collection of research articles is many such
discussion points. The final conclusion has to be drawn by collecting all such
arguments on the table and synthesizing them.

------
stonogo
This entire article is a criticism of the scientific journal industry, not
"science." It is disturbing that the authors appear to have a difficult time
compartmentalizing those concepts, but still consider themselves authoritative
on the matter.

------
hoopd
> When people talk about flaws in science, they're often focusing on medical
> and life sciences, as Horton is. But that might simply be because these
> fields are furthest along in auditing their own problems. Many of the
> structural problems in medical science could well apply to other fields,
> too.

Pure FUD; those fields have unusually low levels of empiricism. Rule of thumb:
as a science has greater trouble gathering enough data to settle disputes the
scientists of that field become more and more like philosophers. They decide
what's true based on arguments or personal feelings and in the worst cases
they fabricate data to fill the vacuum. The problem is that the core element
of science, that of rigorous experimentation, is missing from the process.

The question is if a field is only able to follow the aesthetics of science
but unable to adopt the rigor to what extent should we consider it science?

------
bitwize
The problem is that admitting science is flawed means that people will propose
astrology or alkaline diets or antivaccinationism as alternatives.

So yes, let's _all_ embrace that science is flawed and then work together to
fix science, and not pretend that Christianity or traditional vitalism are
better alternatives to science.

~~~
xj9
Science is not a religion, lets not compare apples to oranges.

~~~
briantakita
Belief in Science is a religion.

------
nosideeffects
How about we just ban "journalists" from publishing articles about the results
of recently published studies?

~~~
6t6t6
> ban "journalists" from publishing

Maybe not.

------
analog31
One humble idea: Include the proposed press release in the peer review
process. Maybe require two abstracts, one for the academic journal, and the
other for the lay press. And there is no loss of honor from saying that a
study is of purely intellectual interest.

------
return0
We should obviously not embrace that. Science should never be flawed
knowingly, if we break this contract, then it's best for the public to
withdraw all academic funding earlier than later and seek answers elsewhere. I
understand that too many people are working in science nowadays, and they all
have to publish something, but that's no excuse to justify "flawed" science.

