
Peer Review is as bad as the Peers Reviewing. That’s often awful - barry-cotter
https://statmodeling.stat.columbia.edu/2020/06/11/bla-bla-bla-peer-review-bla-bla-bla/
======
barry-cotter
> Peer review is fine for what it is—it tells you that a paper is up to
> standard in its subfield. Peer reviewers can catch missing references in the
> literature review. That can be helpful! But if peer review catches anything
> that the original authors didn’t understand . . . well, that’s just lucky.
> You certainly can’t expect it.

...

> 4\. All the papers that don’t get retracted . . .

> One problem with the attention given to the fatally flawed papers that get
> retracted is that we forget all the fatally flawed papers that aren’t
> retracted.

> For example, from that long paragraph above:

> – The beauty-and-sex-ratio paper: Not retracted. 58 citations. Endorsement
> from Freakonomics never rescinded.

> – The ovulation-and-voting paper: Not retracted. 107 citations. Fortunately
> this one never got taken seriously by the news media.

> – The gremlins paper. Not retracted: 1064 citations. Might still be
> influencing some policy debates.

Etc etc etc.

> And, if we just want to look at papers published in Lancet:

> – That seriously flawed Iraq survey: Never retracted. 779 citations. Came up
> in policy debates.

> – That hopeless-from-the-start paper on gun control based on an
> unregularized regression with 50 data points, 25+ predictors, and a bunch of
> ridiculous conclusions: OK, this one had only 75 citations and, fortunately,
> it was blasted in the press when it came out. So, too bad that something
> like 75 people thought this paper was worth citing (and, no, a quick glance
> at the citations suggests that these are not 75 papers using this as an
> example to show people how not to do policy analysis). As I wrote the other
> day, the useless study is included in a meta-analysis published in JAMA—and
> one of the authors of that meta-analysis is the person who said he did not
> believe the Lancet paper when it came out! But now it’s in the literature
> now and it’s not going away.

