
Big Problems Facing Science - junipergreen
http://www.vox.com/2016/7/14/12016710/science-challeges-research-funding-peer-review-process
======
drauh
"Scientists often learn more from studies that fail. But failed studies can
mean career death. So instead, they’re incentivized to generate positive
results they can publish."

Not in the "hard" sciences: my background is physics, and there are lots of
papers on "failed studies", which serve to constrain the domain of
applicability of some theory or other. Or, better yet, indicate new science to
be found.

The authors note the bias in the survey: "Our survey was not a scientific
poll. For one, the respondents disproportionately hailed from the biomedical
and social sciences and English-speaking communities."

~~~
tagrun
As a physicist, I'm not happy with the trend of new research fields calling
themselves science (mainly for benefiting from the hard-earned respect physics
[or natural philosophy, as it was called] has gained throughout centuries,
which is the real problem [rather than etymology]), and recently introduced
concepts to gain some legitimacy, for justification, such as "hard science",
"natural science", etc.

[https://en.wikipedia.org/wiki/Hard_and_soft_science#Criticis...](https://en.wikipedia.org/wiki/Hard_and_soft_science#Criticism)

Now the word "science" somehow means "legitimate and respectable research".
What's worse, it's not the reality of these fields but a distortion of reality
through verbal association, and the word "science" is slowly being dragged
into mud due to non-reproducible or downright wrong published results thanks
to many such fields of "science".

I should also add that mere "data fitting and data extrapolation" with no
basic theory of fundamental understanding isn't science either.

If you're curious about the details, Feynman has defined the issue very well
at some point

[http://calteches.library.caltech.edu/51/2/CargoCult.pdf](http://calteches.library.caltech.edu/51/2/CargoCult.pdf)

~~~
James001
I disagree. I'm glad that science is becoming less associated with the "hard"
sciences and is being used more broadly. After all, science is much more than
simply the sciences that strictly use the scientific method. The "soft"
sciences like psychology are sciences even though their methods are sometimes
not easily reproducible. But we shouldn't be resigning outreslves simply
because the "soft" science methods are more difficult than the "hard"
sciences.

~~~
LoSboccacc
That's the definition for science and doing resesrch. This line of reasoning
brought us Theranos and [http://www.nature.com/news/over-half-of-psychology-
studies-f...](http://www.nature.com/news/over-half-of-psychology-studies-fail-
reproducibility-test-1.18248)

So no science as a field should remain well defined in scope and meaning, in
the same way crafting shouldn't be blurred with engineering.

~~~
James001
Right, so long as we can agree that science is not limited to the scientific
method. Science should be well defined, but not limited to arbitrary confines.

~~~
tagrun
Science _is_ well defined.

Oh, so when science is defined in the way Popper defines it, it is "arbitrary
confines", but when it is defined in a way to legitimize cargo cult science,
it is "well defined".

~~~
James001
Arbitrary confines of science is when you limit science to the scientific
method. Well-defined science is frankly an insult to science

~~~
tagrun
> Science should be well defined, but not limited to arbitrary confines.

> Well-defined science is frankly an insult to science.

Spoken like a true pseudoscientist.

Yes, it should be an insult, right? How dare they can expect a scientist to
use scientific method for all their results? I mean, it sounds totally crazy,
insulting! And more importantly, how are we gonna publish papers or get grants
then?

~~~
James001
Strawmen for days!

Just answer me this: Do you believe that the science is purely limited to the
scientific method?

Just riddle me that and I'll be content. Just keep in mind that you'll be
outing yourself as scientifically illiterate.

------
ap22213
Science just costs too much, and scientists have paid too much to be
considered scientists. The Ph.Ds that I know just want jobs that pay the
bills, that pay off their loans. They do research because it is what gives
them grants and enables the climb of the hierarchcy, not because they're
driven by it. The system has beat out their lust of curiosity and exploration.

The root problem is cost. We have to make science cheaper. We have to put it
back in the hands of the curious and adventurous. Science should be possible
by anyone - even teenagers. If science can't be done by the young, the poor,
the autodidacts, what's the point?

Currently, everyone is on the 'teach everyone programming' kick. U.S. states
are now starting to require that everyone learns programming. But, what about
science? Let's create the 'github' of science - where anyone with a hypothesis
can create a notebook, gather up like-minded people to collaborate, gather
data, analyze it, 'fork' others' research into new areas. That's how we will
make it cheaper and accessible.

~~~
patall
No, that is not the problem. The science may be expensive but the main cost is
seldomly the researcher him- or herself. Equipment is expensive and always
will be, I mean we are speaking about cutting edge science not counting birds
in the garden. When I am doing Assays in the lab, its it easily 5 bucks per
sample. But then I have 30 to 50 samples. And one Assay is only a small step
in a long procedure for a small scientific advance. Similar, when I am doing
scientific computation, I often cannot work with the nice and cheap Nvidia
1080 GTX because I am doing science and need reproducibility. Accordingly, ECC
is a must and we have to afford a Nvidia Tesla Cluster. Do you really think
scientists are not curious or exploring, thats ridiculous. It is just that you
cannot make an expedition without funding, and that is not about the money you
make, its about the ship you need. (just my two cent as someone whos working
on a dataset that is worth 3 or 4 times my yearly income and could easily be 3
or 4 (or 10) times larger)

~~~
veddox
Well put. Modern science is an incredibly high-tech business in so many
fields. Whether it's an expedition to Antarctica, building a new radio
telescope or even "just" buying an electron microscope - you're gonna have to
pay, and you're gonna have to pay _a_ _lot_.

There are still a handful of areas where amateurs could in theory do proper
research without spending a few million dollars on lab equipment - ecology for
example. But then we get to the problem of necessary knowledge: it take a few
years of hard study until you even know the right questions to ask, which ones
have already been asked, and how to interpret the results that you get.

In short: there is no short cut to modern science.

------
veddox
Yes, yes, yes - these are the things we ought to be talking about! In
academia, in government, in society! Great to see these problems summarized
here, usually one only sees an article on one of the seven.

It gives me hope that we _have_ started talking about these things - at least
in academia (as numerous think-pieces in Nature et al. testify). We need to
continue this discussion, make the public aware of it, and then start taking
steps to solve it. No, science is not doomed, but boy do we still have a lot
of work to do to get it to where it should be...

~~~
themartorana
Yeah, what I came here to say. It's been interesting to watch over the past
5-10 years as these problems seemed to gain a voice. Hopefully we'll see
several addressed in our lifetimes.

~~~
veddox
My favourite proposed solution for the paywall problem is to launch a platform
akin to iTunes - one giant repository of academic articles available to anyone
at a low cost. (Kind of like SciHub, except the people who deserve to get some
money for their work actually do.)

------
hughw
I had hoped to read one of those occasional lists of the hard, outstanding
problems scientists still need to solve, in biology, say, or physics. Instead,
it's a list of meta "problems" like science funding, or poor study designs.

~~~
danieltillett
The only problem in science is funding. Everything else is an opportunity.

~~~
veddox
If we had unlimited funding, most of the other problems would indeed take care
of themselves. The problem is the "unlimited" part...

~~~
danieltillett
The problem is not only the level of funding, but they way the funding is
distributed. Science could live happily with the current expenditure if the
way the funding was distributed was smarter.

------
jondubois
I think that this trend of decreasing funding for science research will only
continue unless some fundamental changes are made to the system.

For one, the government is becoming increasingly weak while corporations are
becoming increasingly powerful.

Unlike the government, corporations in general (and their investment
strategies) are focused on short term results - That's how executive
pay/bonuses are structured - CEOs don't want to invest in something that will
only bear fruit in 10 years so that some future CEO will get all the credit
for it. Humans are terrible at allocating credit/praise because we like to
pretend that the universe is simple and that all actions have simple,
predictable effects without unexpected side-effects.

Science research cannot exist in a corporate environment. Science can only
rely on government or philanthropy.

------
junipergreen
Daunting as this list seems, of these points and fixes are of course
interrelated. Even small tweaking can bring big improvements. Take open
science. Improvements to funding are needed to promote long-term projects, but
in absence of those kinds of grants, open science is all the more important.
Sharing partial results and details of studies that might not make it into
publications leaves a baton for others to pick up and make the most of their
limited funding time.

And speaking of open science, making research public as it's happening (not
even results, just what you're studying) can help prevent redundant concurrent
studies in multiple labs and facilitate collaboration, also making the most of
the limited funding scientists have.

------
jaz46
This list is great! Reproducibility and verification of results is key!

Pachyderm (disclaimer: my company) is building infrastructure tools to help
data scientists reproduce results by offering "Git for Data."

------
stcredzero
Could the "citizen science" movement help with the paywall problem?

What about legislation requiring that any unclassified government funded
science be available outside a paywall? Last I saw a figure for it, the
combined budgets of all of the US military bands was ~$270 million a year. I
bet a lot of science publication could be "nationalized" for that amount.

~~~
veddox
The EU has put a regulation in place that as of 2020, all published research
that funded by it must be open access ([https://techcrunch.com/2016/05/27/eu-
mandates-open-access-fo...](https://techcrunch.com/2016/05/27/eu-mandates-
open-access-for-all-publicly-funded-research-by-2020/)).

------
unabst
1\. Money. Economical issue.

2\. Poor design. Competence issue.

3\. Replication. Incentive issue.

4\. Peer review. Bad old analog system.

5\. Paywalls. Capitalism.

6\. Poor communication. Competence issue.

7\. Stress. Personal issue.

Nothing gets in the way of science except ourselves.

------
d0mine
It is an obvious propaganda piece for the benefit of those who profit on the
fossil fuel industry.

Psychology and Climate change are in different categories. The crisis of
reproducibility in _soft_ sciences has nothing to do with the overwhelming
scientific consensus on the reality of global warming.

Many mentioned issues are real in hard science too but it just an example that
you can lie with the truth. Perverse incentives, publication bias,
imperfection of peer review, etc can't invalidate established results e.g.,
Newtonian physics continues to work in the domain it is applicable for.

It is infuriating that the planet (planetary habitability) is destroyed for
the benefit of the very few.

~~~
veddox
I'm sorry but I really cannot see where you get this bizarre interpretation of
the article from. Global warming isn't mentioned _anywhere_ in here! ("Climate
change" gets exactly one mention, between two commas, and otherwise plays no
role whatsoever.)

I think you really missed the point of the article. Of course the problems the
author talks about don't invalidate already established results, the big deal
is that they potentially prevent _new_ results from being soundly established
(or discarded if necessary)!

~~~
d0mine
Are you a global warming denier?

"Climate change" is "fossil fuel"-friendly alternative to avoid saying "Global
warming" for what is happening. It is disingenuous to suggests that "Global
warming" is unrelated to the article.

The article equates the certainty with which we know results in Psychology and
Climate change (the terms are separated by commas as you've noticed).
Psychology has a very flimsy foundation: even major results can be debunked
(e.g., ego depletion). On the other hand there is no doubt that the climate
change (global warming) is happening.

The article can be used as a tool by climate change deniers. They could say:
"science have many major issues and therefore climate change is a figment of
these communist eggheads imagination."

~~~
veddox
> Are you a global warming denier?

I did not say anything to that effect in my previous comment. In fact, your
question is so mindbogglingly groundless I am not even going to bother
answering it.

> The article can be used as a tool by climate change deniers. They could say:
> "science have many major issues and therefore climate change is a figment of
> these communist eggheads imagination."

Of course it can, and nothing is going to stop them. But does that mean we
shouldn't be talking about the problems we _are_ facing in science? You aren't
going to fix anything if you refuse to talk about it.

This article isn't about climate change any more than it is about climate
change deniers. That's another discussion for another time. This article is
about _our_ problems in the way _we_ do science. So stick to the point,
please.

