
The peer review industry: implausible and outrageous - jseliger
https://www.the-tls.co.uk/articles/public/peer-review-industry-implausible-outrageous/
======
SoylentOrange
Open access, as is mentioned in the article, is both important and difficult
to achieve. As a PhD student or associate professor, you work on a strict
schedule to publish or perish. The only acceptable publication venues are the
4-6 respectable journals in your field. You have no time or power to push for
open access.

Only large research institutions like (I’m familiar only with the CS research
community) MSR, Google Research, Facebook Research, ... as well as already
tenured professors can really force the issue.

I work at a smaller industrial research lab. We try to push for open access
whenever we publish, but publishing in non open access journals is important
to maintaining our perceived prestige, which in turn is important in hiring
(especially when we poach from academia).

I publish pretty regularly and try to be selective, but ultimately the push
has to come from an institutional level such as universities or large
industrial research labs to say “we will only publish in open access
journals”. This is especially true in the US where EU style legislation is a
long way off.

~~~
ma2rten
Large industrial research labs have the same problem. They need to maintain
their prestige for recruiting purposes.

~~~
drb91
At the same time, this seems to select the worst of the candidates: those who
care about prestige.

------
apo
What to do about the entrenched scientific publishing-funding complex has been
a concern since the first full text of journal articles started appearing in
the 1990s.

The response to date has been for funding agencies (nonprofits and
governments) to require publication in open access journals as a condition of
funding research. This has happened in both the US and EU.

However, I'm pessimistic this approach will address the problem, and it could
well create a number of unintended consequences. For one, publishers that have
introduced OA models can charge thousands of dollars to the research teams
publishing the work. That could mean fewer publications, and more information
silos.

The article hints at, but doesn't fully describe another, more pressing
problem. Peer review is in trouble. The number of papers the average
researcher is being asked to review is increasing while the abundance of
papers means the few bother to read them.

I suspect the solution lies somewhere else entirely. Post-publication peer
review is one option, and Fighare has built an interesting model that could be
a component of a future system:

[https://figshare.com](https://figshare.com)

Regardless, the problem is cultural, not technical. The crazy economic
incentives described in the article work because everyone involved has a
(sometimes perverse) incentive to keep it in place. Until that changes, it
will be mostly status quo ante.

~~~
buboard
> the number of papers the average researcher is being asked to review is
> increasing

The quality of review becomes proportional to the impact factor of the
journal. And given that the high impact journals have not substantially
expanded the number of articles they publish, it might mean that there are now
_more_ candidate reviewers.

What _has_ become substantially more burdensome is the number of funding
applications that (esp. young) investigators have to submit per year.
[http://www.pnas.org/content/112/2/313](http://www.pnas.org/content/112/2/313)

> Post-publication peer review is one option, and Fighare

There are interesting proposals. E.g. elifesciences proposes to publish
everything, under the condition that the reviews will be published eponymously
along with the paper

~~~
lsh
no proposals, the review feedback and author response to the review _are_
published. Here is an example:
[https://elifesciences.org/articles/35082#SA2](https://elifesciences.org/articles/35082#SA2)

Peer review at elife works slightly differently. The author(s) are not
subjected to multiple rounds with multiple reviewers, instead the review
feedback from the peer reviewers is aggregated by the editorial staff before
being sent back to the author. There may be multiple rounds of this however.

------
euske
I just realized that this analogy of the academic publishing industry is also
quite similar to that of the search engine industry.

Most web pages are provided to the public free of charge, but they typically
don't obtain them directly (technically they can, but it's often too tedious).
A search engine takes them and redistribute to the public. They do it by using
the metrics mostly provided by the (unpaid) public: PageRank. And they make a
hefty profit for doing that. They also amass a lot of power. Both industries
are essentially doing the same thing: content curation.

The difference is that search engines don't get complaints like academic
publishers.

I think the reason is that the academic publishing is much simpler and
actually more transparent than search engines; you can easily see the inner
workings and the flow of money. On the other hand, people don't really
understand how Google makes so much money. We're effectively blinded by the
complexity.

~~~
jungturk
The search engines have sidestepped grief because of their third-party pays
model - advertisers fund their content curation and discovery expenses and the
system retains open access at no (direct, monetary) cost to the consumers.

The peer review process might be more expensive/intensive than PageRank for
all I know, so perhaps journal publishers are doing much more work than search
engines.

Perhaps if journal publishers used an ad-supported model they wouldn't be
demonized so readily since they'd be incentivized for open access since
they're selling eyeballs

I'm not advocating, just working with the parent's comparison.

~~~
atq2119
> The peer review process might be more expensive/intensive than PageRank for
> all I know, so perhaps journal publishers are doing much more work than
> search engines.

The peer review process is labor intensive, but to add insult to injury,
publishers aren't the ones doing it and they don't pay reviewers. It's all
done for free by researchers.

At best publishers add some proof reading services, but in many cases not even
that.

------
titzer
> (4 paragraph analogy of implausible and outrageous scheme of bakeries which
> is total fiction). > ...Yet the story is broadly analogous to what happens
> in the publication of academic journals.

While I think there are lots of problems with the academic journal system,
please, please, please do not reason from analogy like this article does. This
is a complex topic that deserves being reasoned about directly. Presenting an
analogy which is obviously artificial just primes readers with a false,
oversimplified narrative. Don't do this. This is highly misleading, and the
bias that results from the priming is corrosive to clear reasoning.

------
confused_guy123
What I don't understand is why peer reviewers agree to perform uncompensated
labor on behalf of these gargantuan, highly profitable publishers. What is the
benefit to them, other than the warm glow of fulfilling one's professional
obligations?

~~~
jonathanstrange
Mostly it's the latter, out of a sense of duty.

But there are other motives. Regular reviews give you some perspective that
you otherwise wouldn't get. For example, you see what mistakes others make and
that can help you improve your own papers. Another motivation for me is
sometimes that I don't want the paper to be reviewed by someone less qualified
in my areas of specialization.

~~~
08-15
Ever noticed that the person who appeals to your sense of duty is always your
boss who doesn't want to give you a raise?

