
ML researcher replies to request to review paper for closed journal - robrenaud
https://plus.google.com/104362980539466846301/posts/WStiQ38Hioy
======
michael_nielsen
The earliest really large-scale instance of this occurred in 2001, when more
than 30,000 researchers signed a letter boycotting any journal which didn't
make articles open access within 6 months of publication. The boycott meant
not reviewing for, editing for, or publishing in, any such journal.
Signatories included many of the best-known scientists in the world. One of
the authors of the boycott was Nobel Laureate Harold Varmus, who had just
completed a term as Director of the US National Institutes of Health.

It's unclear how many of those researchers followed through on the boycott.
Some did -- I've met many scientists who say they send letters similar to
LeCun's. I've also met some who signed, but who now admit (often sheepishly)
that they didn't follow through.

Text of the letter is here:

<http://www.plos.org/about/what-is-plos/early-history/>

~~~
jrogers65
I feel that it's important enough to reiterate that PLOS is a non-profit
organisation and relies on donations to keep it going:

[http://www.plos.org/support-us/individual-
membership/#make-a...](http://www.plos.org/support-us/individual-
membership/#make-a-donation)

~~~
Radim
Huh? They actually charge a hefty fee for publishing:
<http://www.plos.org/publish/pricing-policy/publication-fees/>

The donations and open-access merchandise are a nice extra.

They are turning fabulous profits (almost $3 million in 2010; $4 million in
2011 -- these are _net profits_ ), but, being a non-profit, must invest them
back in the organization.

------
rrmm
The machine learning community has been pushing for open access for quite
awhile.

In 2001, the 40 editors of _Machine Learning_ (some of whom I was quite proud
to know), resigned to start the JMLR with this letter:
<http://www.sigir.org/forum/F2001/sigirFall01Letters.html> . Note also their
citation of the similar actions taken by the editors of _Logic Programming_ in
1999.

EDIT: btw, the JMLR is at <http://jmlr.csail.mit.edu/papers/> . moar edit:
<http://jmlr.csail.mit.edu/mloss/> for relevant open source code.

~~~
michael_nielsen
JMLR is an amazing success -- a great example of a very high quality open
access journal that doesn't ask authors to pay any article charges.

------
minimax
It's worth noting that Cosma Shalizi has been doing this for about six years
now, and it's becoming increasingly popular in certain areas of academia.

Background:

<http://masi.cscs.lsa.umich.edu/~crshalizi/weblog/442.html>

<http://masi.cscs.lsa.umich.edu/~crshalizi/weblog/864.html>

[http://gowers.wordpress.com/2012/01/21/elsevier-my-part-
in-i...](http://gowers.wordpress.com/2012/01/21/elsevier-my-part-in-its-
downfall/)

------
tzs
Terry Tao made a good point when he joined the boycott of Elsevier journals.
He said he would stop submitting to their journals, and stop doing editorial
work for them, but declined to stop reviewing papers.

The reason he gave is that while he is at a stage in his career where there is
no pressure to publish in any particular set of journals and so can exercise
discretion in where he publishes, he recognizes that some mathematicians, for
reasons beyond their control, are unable to do so, and he does not wish to
penalize them.

For fields where many of the high impact journals are not yet open access, I
think Tao's approach makes sense. Publishing in high impact journals greatly
helps the careers of young researchers.

~~~
Snoptic
Weird. Terry Tao posting a note on his Google+ feed saying "this is a good
paper", with few relevant +1s, should be more of a resume booster than any
journal publication.

------
alxbrun
In my opinion, Yann LeCun is one of the top celebrities in ML/Neural Net
research. His voice really has weight.

~~~
jat850
Definitely. At work today we were watching a video on Machine Learning by
Andrew Ng and he made a number of references to LeCun's involvement the field,
and listed him in the acknowledgements section of his talk. It has encouraged
me to look into more of his areas of work.

I signed up for Andrew's ML course
(<https://class.coursera.org/ml/lecture/preview> , start date is not announced
yet). Really looking forward to it.

~~~
pmelendez
I took the very first course when it was a Stanford experiment and I have to
say that was a lot of fun! It feels a bit downgraded in terms of difficulty
but quality wise was very good.

------
ElliotH
I did a course this semester at university about bioinformatics. I was very
impressed by their attitude to knowledge sharing. Almost every paper
referenced by our professor and any citations in those papers that I used were
published online for no charge. But even better than that, the data and code
used to generate their results was shared too. Our professor noted that in the
bioinformatics field a paper is considered pretty much worthless if the data
isn't made easily available in one of a few archives.

I really hope the rest of acadaemia can follow through. It seems to be we
might make more progress in all of the sciences if results are easily
replicable and research is published to all.

------
tokenadult
This is a very forthright reply by the researcher asked to review the paper.
Right now, reviewing submissions to scientific journals is anonymous, and not
well rewarded. Jelte Wicherts, writing in Frontiers of Computational
Neuroscience, (an open-access journal) suggests new procedures

Jelte M. Wicherts, Rogier A. Kievit, Marjan Bakker and Denny Borsboom. Letting
the daylight in: reviewing the reviewers and other ways to maximize
transparency in science. Front. Comput. Neurosci., 03 April 2012 doi:
10.3389/fncom.2012.00020

[http://www.frontiersin.org/Computational_Neuroscience/10.338...](http://www.frontiersin.org/Computational_Neuroscience/10.3389/fncom.2012.00020/full)

for making the peer-review process in scientific publishing more rewarding and
more reliable too. Wicherts does a lot of research on this issue to try to
reduce the number of dubious publications in his main discipline, the
psychology of human intelligence.

"With the emergence of online publishing, opportunities to maximize
transparency of scientific research have grown considerably. However, these
possibilities are still only marginally used. We argue for the implementation
of (1) peer-reviewed peer review, (2) transparent editorial hierarchies, and
(3) online data publication. First, peer-reviewed peer review entails a
community-wide review system in which reviews are published online and rated
by peers. This ensures accountability of reviewers, thereby increasing
academic quality of reviews. Second, reviewers who write many highly regarded
reviews may move to higher editorial positions. Third, online publication of
data ensures the possibility of independent verification of inferential claims
in published papers. This counters statistical errors and overly positive
reporting of statistical results. We illustrate the benefits of these
strategies by discussing an example in which the classical publication system
has gone awry, namely controversial IQ research. We argue that this case would
have likely been avoided using more transparent publication practices. We
argue that the proposed system leads to better reviews, meritocratic editorial
hierarchies, and a higher degree of replicability of statistical analyses."

------
georgeorwell
> I have pledged to no longer do any volunteer work, including reviewing, for
> non-open access publications, which unfortunately includes pretty much every
> publication from [commercial-publisher].

The volunteer aspect of peer review is the worst part. How much is an expert's
time worth? How much is a two-sentence review worth to an author? If it were
paid on an hourly basis using an honor system, it would be fantastic for
quality and consistency.

That closed journals ask for volunteer review makes it doubly bad. But I still
believe paid review for open access journals would be a good thing. I know, it
is an unrealistic fantasy.

~~~
wmf
Speaking as an author, paper reviews are very helpful (although the useful
ones are generally much more than two sentences). Speaking as a reviewer, I
don't mind doing reviews for free (I consider it part of the job) and if
volunteer reviews are needed to make open access work I think it's a worthy
tradeoff.

------
akeck
Professors and researchers should start charging for time spent helping the
closed journals at the very least.

If you think your reviews are valuable, charge accordingly. You provide value
to the publishing process and should capture that value. You are providing the
service of "expertise". If you provided that service to a lawyer, you would
earn several hundred dollars an hour.

There's probably a business opportunity in creating a clearinghouse for
reviewers, editors, etc. and managing the accounting.

Edit: Grammaring more goodly

~~~
javert
Professors are the producers and consumers of these journals. So, a group of
your peers invests time reviewing your work and giving you helpful feedback,
and you pay them back by reviewing other people's work.

Basically, it's a zero sum game. So if everybody paid for the reviews they did
and then got paid for doing reviews, a lot of money would change hands and
everybody would end up with the same amount of money they had to start with.

Of course, the companies that _publish_ the journals take a cut. But that's
what it is, a cut. There are really communities of professors participating in
a zero sum game.

~~~
afrozenator
I beg to differ a little, Professors should be paid to do reviews, and not pay
to have reviews done.

The companies that publish the journals don't take a cut, they take the lion's
share basically, paying a professor to review would work better in having them
take a share of the pie.

~~~
javert
But if you pay professors, you either pay them enough to make it worth their
time, or you pay them less than that.

In the former case, you are incentivizing people to do reviews who are not
actually qualified to do them. Or, incentivizing people to do more reviews
than they have time to do a good job on.

In the latter case, there's not really any point.

------
pitchups
This is a great first step. But for this to really make a difference to the
current traditional publishers - every researcher would need to follow this
example. The other problem is access to all the previously published papers.
So my question is : Who owns the copyright to a paper published in a major
academic journal? Is it the original authors / researchers? One would think so
since they are the authors. Or do they grant exclusive rights to the journals.
If the authors own the copyrights, what is to stop them from uploading their
papers to one of these Open access sites?

~~~
mturmon
Typically the publisher retains copyright, they make the author(s) sign it
away as a condition of publication. There are some exceptions.

By inserting themselves into the somewhat "broken" system of scientific
publishing, the large publishers have been making a killing. (It's broken from
an economic point of view because too much work is done for free.)

------
wink
I really don't want to derail the thread, but one of the people participating
in that G+-thread (Mike Taylor) has written some very good articles at
<http://reprog.wordpress.com/> (and I really can't remember if I picked up the
subscription here or elsewhere)

------
msutherl
I would love it if somebody could register a domain and create a canned reply
that we could copy paste when asked to review papers with appropriate links to
all relevant open access projects.

------
Snoptic
Who is that guy saying that publishing in arxiv is career suicide? Why are
there ML people playing paper chase at universities when they can quickly make
a fortune in industry and retire to a life of academic leisure. Of all people,
ML researchers should have noticed that the 21st century has arrived.

