
Aging interventions from older publications that deserve a new look - apsec112
https://srconstantin.posthaven.com/aging-interventions-from-older-publications-that-deserve-a-new-look
======
JamesBarney
> However, the reliability of the average publication has probably decreased.
> If there are indeed "diminishing returns to science" in recent decades, as
> Patrick Collison and Michael Nielsen argue [2], with roughly constant rates
> of important discoveries (as rated by experts) and flat economic
> productivity (a measure that we'd expect to correlate with technological
> progress) despite exponentially growing numbers of scientists, publications,
> and dollars devoted to science, then the quality of the average paper,
> scientist, or dollar allocated to research must have gone down. In that
> case, a randomly chosen older paper should be more trustworthy than a newer
> paper.

I don't quite follow this line of reasoning. I must be misunderstanding
something because Sarah is way smarter than I am. But it seems to me the
diminishing returns in biological science are due discoveries having a smaller
impact, not on the quality of the paper. i.e. a 1985 paper doesn't have more
of an impact than a 2019 paper, but a paper released in 1985 has a bigger
impact than that same paper released in 2019.

If older studies are more fruitful than modern studies I would imagine it's
because they were wackier. Modern studies have to deal with a more restrictive
IRB and also seem more likely to fit inside the box. I hardly ever heard of
modern studies where someone was like "I hooked up a battery to a weird part
of the brain to see what happens."

I have a theory that as colleges, PhD programs, and then professorships have
gotten more competitive that we have been selecting for higher and higher
conscientiousness which has led to greater conformity. My professors didn't
seem nearly as wacky and they described their professors/mentors.

~~~
barry-cotter
> I don't quite follow this line of reasoning. I must be misunderstanding
> something because Sarah is way smarter than I am. But it seems to me the
> diminishing returns in biological science are due discoveries having a
> smaller impact, not on the quality of the paper. i.e. a 1985 paper doesn't
> have more of an impact than a 2019 paper, but a paper released in 1985 has a
> bigger impact than that same paper released in 2019.

Less competition leads to less p-hacking, less pressure to publish or perish
leads to greater risk taking in experiments, fewer proximate measures of
impact lead to more confidence that results are real rather than an artifact
of finding _some_ intermediate measure that looks significant, and just random
fashion leading to some areas going out of style that would actually have been
fruitful if pursued further.

~~~
JamesBarney
I guess I always assumed that ignorance was a more important cause of p
hacking than cheating. But if it's mostly cheating I could see a high pressure
environment leading to more p hacking.

If that's true that would mean that a great deal of professors are dishonest.

~~~
webmaven
Rather than 'dishonesty' per-se, it is probably closer to 'wishful thinking'
most of the time, with a side-order of 'don't look a gift horse in the mouth'.

~~~
Jamwinner
Bingo. It hard to be critical when you are (or feel that you are) dependent on
the paper for your grade/diploma/career progress.

