
How scientists fool themselves and how they can stop - DavidSJ
http://www.nature.com/news/how-scientists-fool-themselves-and-how-they-can-stop-1.18517
======
texthompson
It's too bad that there's not a more robust discussion about the institutional
incentives that drive scientists to fool themselves.

~~~
yiyus
This is a key aspect. There is a pressure (both institutional and self-
imposed) to produce something original and unique, so trying to reproduce
other people results is seen as a lost of time.

Either you confirm previous results, which is nice but not something you can
publish or write a thesis about; or you find they are wrong and then you have
to tell the authors, which is something they may appreciate or not, and be
exhaustive to prove it, which is something in which you may be interested or
not.

What I would like to see is that PhD students were required to reproduce some
previous results as part of their thesis. This would increase the quality of
scientific results and would prepare the candidates to do their own research
(at difference of now, that they are put on front of a laboratory to get
"interesting results" and they don't know what to do with it).

~~~
NH_2
Agreed -- would also love to see an incentive structure around publishing
negative results. One of the areas of research I think is the most exciting is
artificial photosynthesis, and it's currently at the phase where the
researchers are trying to discover efficiency gains by just trying out a bunch
of different materials.

Take this recent publication [1]:

> For water oxidation, the photoanode surface was protected from corrosion by
> a 62.5 nm layer of amorphous, hole-conducting TiO2 that was grown by atomic-
> layer deposition (ALD).

> The TiO2 layer significantly improves the stability of III-V photoanodes in
> a tandem structure for water oxidation while the tandem structure produces
> sufficient photovoltage to sustain the efficient, unassisted production of
> hydrogen by water splitting in aqueous alkaline electrolytes.

They discovered that coating the photoanode with 62.5nm of TiO2 helps
stabilize the reaction. But who knows how many materials they went through to
get that one? And how many different coating thicknesses they tried before
settling of 62.5nm? This tech could be next-gen solar, would be great to see
the global rate of discovery increase. Perhaps YC Research can start this
trend?

[1] Joint Center for Artitificial Photosynthesis -
[http://authors.library.caltech.edu/59897/1/c5ee01786f.pdf](http://authors.library.caltech.edu/59897/1/c5ee01786f.pdf)

------
gaius
A scientist's first instinct is "get more funding". This cannot help but bias
their conclusions.

~~~
andyjohnson0
Unnecessarily cynical. Its a problem that science funding makes scientist's
careers so precarious, but nobody goes into science for the money.

~~~
dagw
"Get more funding" isn't the difference between making a little money vs.
making a lot of money. It's the difference between making a little money and
not having a job.

~~~
coherentpony
Depends on the scientist's position. Tenure-track and tenured faculty in the
US typically have 9 months of their salary covered by the institution
employing them. Only the remaining three months need be covered by grants (or
teaching).

~~~
scott_s
But going from tenure _track_ to _tenure_ in most research universities does
require getting grants.

------
timtas
I think it's official, we moderns owe an apology to medieval barbers and
conjurers.

~~~
venomsnake
We don't - first we have correcting mechanisms in place. Second - the self
deception ability is greater the softer/fuzzier the field is.

~~~
stewbrew
Who measures softness/fuzziness of a field? Who makes sure that measurement
isn't biased? Just because you use maths doesn't imply your conclusions and
theories are as precise and clean cut as 2+2.

~~~
nmrm2
_> Who measures softness/fuzziness of a field?_

There's an absolutely unambiguous distinction between the "hard" and "soft"
sciences

Social scientists hold themselves to fundamentally different standards on
things like repeatability and predictive power and methodolgy than other
fields. And whenever that difference can be quantified (e.g. statistical
results), "different" means "lower". It's just a fact, and it's one most
sociologists will freely admit, for example. You simply can't get the same
sorts of results on humans that you can get on e.g. chemical reactions.

Unfortunately, soft/hard comes along with a value judgement that's not
necessarily appropriate. But pretending like there isn't a clear difference
between what are now referred to as the soft and hard sciences is
disinegenious.

~~~
arcanus
Generally agree with the sentiment, but I think the line is fuzzy along the
periphery.

For instance, economics has, I would claim, far weaker levels of model
validation than engineering fields, but often is presented as a hard or
'close-to-hard' science.

------
sjg007
Until you have the technology to actually "see" something or measure it
accurately, it is almost impossible to deduce reality from correlations.

