
“Peer review” is younger than we think - Petiver
http://sappingattention.blogspot.com/2017/09/peer-review-is-younger-than-you-think.html
======
kobeya
The journals published by the Royal Society, where modern enlightenment
science was born, was informally peer reviewed by the mid 1700's, and peer
review was systematized in the 1800's. The explosive growth of the industrial
revolution, the reordering of society in the enlightenment, and everything we
call modern and progressive is associated with a sequence of events that is
also tightly correlated with the birth of peer review. Most philosophers of
science don't consider this a coincidence. Science gave birth to exponential
progress, and science itself is dependent on peer review.

Maybe the integration of peer review into practical industries, medical
studies, and government regulation came later. But those were cases of taking
something that worked astonishingly well and apply it to other domains. It's
hardly fair to say that peer review "started" in the 1960's or 70's.

~~~
pnathan
Just for the sake of solidity, would you care to have a citation there?

~~~
kobeya
Ironically there's some given in the article.

------
murphyslab
One of the key factors missing in this discussion is that there are two
overlapping forms of peer review: One which is merely technical, checking for
correctness and completeness. The other is a much more subjective form, in
which a reviewer is asked to comment on the "merit" or potential "impact" of a
given study. The former is often necessary, as honestly there's a lot of crap
out there, but more often it comes down to minor details that the expertise of
another scientist's specialty can best address. The downside is that this can
inspire a certain pedantry and pressure to undertake what is essentially
unnecessary formatting.

The latter is essentially the real problem of science in its current state:
prestige. There is no reason to not just publish everything in a mega-journal
that doesn't consider "merit". It would mean more replication studies and less
publication bias against "negative" results (i.e. this drug didn't work as
hoped). High-prestige journals also tend to charge correspondingly higher
subscription fees, while adding nothing special to the scholarly ecosystem,
beyond delaying (often by months) the eventual publication elsewhere. However
careers are made on publishing in high-prestige journals.

A recent tweet makes one wonder what the value of high-prestige journals in
the first place, if they don't even bother to correct or improve the scholarly
record. A PNAS editor's rejection of a paper with greater statistical power
and improved methodology, but which fails to replicate a highly cited PNAS
paper was on the basis that "in these days of computer literature searches, I
see no reason a journal needs to feel compelled to publish follow-up papers on
previously published work. The follow-up paper would be found by those
interested where ever it is published"
[https://twitter.com/mendel_random/status/917115714544324609](https://twitter.com/mendel_random/status/917115714544324609)

------
anon1253
Another interesting tidbit is that Evidence Based Medicine, what we've come to
known as randomized controlled trials with systematic reviews, is /much/
younger than people think. The movement properly started in the 1960s, but
only came to fruition after Archie Cochrane (1970s). Still Evidence Based
Practice is a contested thing, and very few systematic reviews eventually end
up informing policy.

~~~
alphydan
Blind experiments in medicine were carried out a lot earlier. See for example
the revolutionary work of Claude Bernard in 1865 [0]

[0]
[https://en.wikipedia.org/wiki/Claude_Bernard](https://en.wikipedia.org/wiki/Claude_Bernard)

~~~
anon1253
Sure, but I was specifically pointing to EBM
[https://en.wikipedia.org/wiki/Evidence-
based_medicine](https://en.wikipedia.org/wiki/Evidence-based_medicine)

------
kang
Peer-review is deeply flawed and in its current form merely an advertisement
form. We should focus on formatting science in a way that accounts for merit
based on reproducibility, falsifiability and paying for real-life usage for
recipe invented. Problem is not all science work is technological application.

~~~
jonathanstrange
The problem is not primarily peer-review, the problem is the ridiculous
publication pressure we're under. I'm a postdoc in the humanities and some of
my colleagues publish 10+ bullshit papers in crappy journals per year - their
career is virtually guaranteed, even though everybody knows and agrees that
it's impossible to write that much and maintain any level of quality. It
doesn't matter how bad or insignificant your work is, if you manage to push
out 5+ Scopus-indexed articles per year, nobody will be able to fire you or
overlook you in tenure track applications, and I see more and more people
gaming the system in that way.

It's leads to a horrible display of mediocricity, of course, and anyone with a
genuine interest in substantial scientific research has a hard time competing.
Yes, some decision makers still take quality into account, but overall there
is a clearcut trend towards chickenshit and bean-counting, at least in
Portugal where I currently live.

------
ninguem2
The term "peer review" has gained usage relatively recently but the term
"refereeing" has been in usage much longer. The practice itself, as noted in
kobeya's comment, is 100's of years old.

------
arca_vorago
The next revolution needs to be in reproducing what has been "peer reviewed".
During my time at a genetics company I learned bad science abounds even in the
" top" journals.

One particular example is people "collabing" just to get their name on more
papers, which they turn around and use as a marketing tool.

Any time I hear the phrase "I've been published $number of times" these days I
tend to think they are worse scientist's not better.

------
egocodedinsol
I don’t understand the second part of the title. I don’t see anything in the
essay to suggest it can or should go away.

The first part is probably solid: peer review is new. But older scholarship
was performed in a very different context - few professional scientists, etc.

But the second part of the title, while provocative, makes no sense to me.

------
raverbashing
Has the "modern way" of doing science been evaluated and compared against
older ways of doing things?

While discoveries do go down in numbers it seems most of past advancements
wouldn't have survived modern standards (while a lot of one-off studies do go
through today)

~~~
egocodedinsol
This is an important point, but the comparison is difficult. Past advancements
occurred in a drastically different time period - few professional scientists,
for instance.

I don’t defend current peer review as optimal, but I can’t see an easy way to
carry out a fair comparison.

------
jwilk
Typography of this article was not peer-reviewed. They somehow managed to use
two different fonts within a single word.

    
    
        T<span style="font-family: LMRoman10-Regular-Identity-H;">he earliest usage
    

oO

------
galeforcewinds
While the concept of peer review of intellectual property work products may be
relatively new, there is long history of craft guilds applying peer quality
controls on physical work products in part to protect the reputation of that
class of products within the market.

------
Bromskloss
I feel that this title is putting words in my mouth.

