
What do you get if you publish a paper in a highly-ranked journal? - ilamont
http://blogs.sciencemag.org/pipeline/archives/2017/08/25/publish-and-prosper
======
icelancer
I live in this field now. I refuse to publish in so-called "highly-ranked
journals" that take 3-6 months to find peers to review and referee your work,
drag their feet, and basically amount to a ridiculous group of cronies that
guard the gates of "science."

I've talked about it before on HN but I will only publish to open access and
_open data_ (most important) journals that require all data be scrubbed of PII
and published alongside the paper for replication purposes. I highly favor
journals that not only allow, but encourage replication (yes, many elite
journals cite "novelty" as a top factor regarding publication decisions...
embarrassing).

Science should be freely available, open access, open data, and replicable.
Otherwise it isn't science. It's primarily garbage that exists to advance the
careers and egos of a select few.

~~~
avip
Don't forget code! "we then used our in-house untested code, developed by an
undergrad we scrapped from Biology, to normalize the results. The code is
comfortably not attached"

~~~
eli
To be fair:

"All computer codes involved in the creation or analysis of data must also be
available to any reader of Science."

And

"Upon publication, Nature Journals consider it best practice to release custom
computer code in a way that allows readers to repeat the published results."

~~~
jcrites
Could we take it one step further? I'd love to see a publication format that
bundles the code, the data, and maybe even the paper together into a single
executable bundle for distribution.

Imagine that the bundle starts with a manifest file that describes the data
and the code, pointing to an authoritative, versioned copy. The user can "run"
the manifest, which will download and install the data and any necessary
dependencies. (If you're thinking about this from a containerized point of
view, the code dependencies and data may be modeled as a container or virtual
machine image.)

The manifest is like a build file that executes the code passing the data as
input, in a clear and reproducible way. Anyone running the manifest will get
the same results as the researcher did. Because the entirety of the execution
environment is versioned and captured by the manifest, including all of the
system software, the results are reproducible from the code and data phase
onward with zero effort.

For bonus points, the manifest would also "build" the paper (e.g. if it's
TeX), and would substitute the results of the code execution directly into the
paper, i.e. graphs and numbers. You could conceptualize the paper as something
like a notebook, where its values can change dynamically according with the
data you provide it.

~~~
cjbillington
This unfortunately just isn't possible if you want your research to be able to
be read in the future. "download?" from where? Links rot. "run?" which
language, compiler, cpu architecture? Dependencies? Code bit rots. And data
formats become unreadable - unless they're self-documenting like plain old
csvs, they're fine.

That's why algorithms are written in pseudocode, and mathematics is written
in, well, mathematical notation. It's expected that if you want to use it
you'll re-implement it yourself in a software setup convenient to you.

So you can't publish runnable code unless it's severely limited - being in
some standard language that is unchanging. I'd much prefer someone tell me in
words what statistical method they used, so I can type it into Python myself
than be forced to spin up some standardised but old and crufty fortran 77 or
something that they didn't enjoy writing and I didn't enjoy running (I am
aware that Python is often just calling the old fortran libs though!). Giving
me the Python in the first place also isn't feasible - some scientific
analysis code I wrote 6 months ago already doesn't run because of API changes
in matplotlib.

A few years ago I saw Python 1.5 code implementing an algorithm I wanted to
use. I couldn't run it - instead I read it, and I read the Python 1.5 docs
about the bits I was unfamiliar with. If it were in pseudocode it would have
been more self-contained.

Code and data formats, other than very crufty ones, are living. They're not
suitable as the final form to publish in. If your project and field is
ongoing, then by all means try to develop standards and use the same software
and formats as each other to help sharing data. But the actual publications
need to still be readable in a decade (or much more) time, so publishing
runnable code seems like it conflicts too much with that.

~~~
pacala
data => $0.1 / GB / Year in a public Cloud, or even $0.01 / GB / Year if using
Glacier / Coldline.

code => a Docker/CFI image.

dependencies => captured in the image.

format => the code understands the format.

~~~
pjmlp
That is short term thinking, who will guarantee any of that stuff will be
around in the next 50 years?

~~~
true_religion
Does it cost more to keep code in a standard format, with all it the
dependencies needed to run it? Yes, especially if you replicate the codebase
anywhere the journal article is stored so it has backups in case of disaster.

However, the costs are fairly comparable to what was once the cost of keeping
paper copies for everything, so I think it is a cost that can be absorbed by
academia.

~~~
pjmlp
It is not only about money, technology, infrastructure and people for keeping
it running also play an important role.

------
robotresearcher
Suggest a title change, since almost none of the comments address the contents
of the article.

E.g. "Large cash bounties paid for highly-ranked journal articles in China and
other Asian and middle eastern countries".

Most comments are answering the question by stating the traditional reward.
The article is about how very different rewards are appearing.

~~~
joe_the_user
Well then let me bite on the question of the problem with such high payouts.

"The next question is whether there’s anything wrong with this idea, and if
so, what. It makes me uncomfortable, but that’s not the appropriate measure.
As long as the journals themselves keep up their editorial standards, the main
effect would seem to be that their editorial staffs must get an awful lot of
China-derived manuscripts whose authors are hoping to get lucky."

Well, I'd offer that this analysis ignores how science has historically
worked.

Essentially, science has involved something like a "club" of individuals who
can be trusted to make a sincere effort to find the true. Certainly, the use
of exact instruments and the theoretical reproducibility of experiments were
important but _also_ dealing with individuals are reliable, systematic and
trustworthy was an important thing.

Which is to say that science journals aren't designed to filter out articles
which are wholly fraudulent, a tissue of lies cleverly created to mimic an
actual scientific breakthrough. With a clever enough person, naturally, there
is no way to see fraud from the article, one would have to lab at the
laboratory, attempt to reproduce the experiment and so-forth (and
_reproducibility_ was the key historically, most experiments _weren 't
actually reproduced_ and no journal is going to maintain a lab to reproduce
submitted articles - except perhaps in CS where a program might submittable).

And there you have it. The lab coated scientist is a movie character but be
scientist, as least once, was more than to engage in some activities. It was
to be part of the "broad march of progress" where the lesser-scientist still
helped move things forward by small, honest iterations of basic research (and
this "club" arguably was disproportionately and unfairly white, male and
first-world but every process has its weaknesses).

A move which makes the pursuit of science a purely adversarial affair, by
consigning poor performs to absolute poverty and giving fairly vast rewards to
good performs is going to break the club everywhere.

And the implications of the end of science as an idealist pursuit vary from
field to field. Fraud in CS or math might be impossible or might be rendered
impossible with suitable measures. But in social sciences or other fields,
things could get nasty indeed.

~~~
Retric
In practice math papers are basically arguments for something. It's possible
to make a bad faith argument by ignoring that A does not imply B and just
hoping nobody notices. You can even structure thing to be intentionally
misleading.

Honestly, I suspect people fudge thing in math more than you might think.

~~~
gertef
Peer-review is what caused people to notice.

~~~
Retric
Sure, but consider you just spent months working on something and your giving
up. You can toss that work away or gamble that nobody notices the error.
Further, if they catch something then you have not really lost anything at
that point anyway.

------
neurothrow
A scientist once apologized for being a jerk to me at a meeting the previous
year, after research that I presented there was published in a highly-ranked
journal. Oddly, I didn't even remember the incident in question, but the
change in opinion, apparently just because of the publication venue has stuck
with me for a long time.

------
s0rce
I published in Science and Nature during my PhD and other than some prestige
and help getting a fellowship I never received any sort of direct payout. I
don't think my supervisor got anything directly either, definitely helped
future grant applications, however.

~~~
projectramo
When I was in grad school, you were sent to the front of the line when
applying for tenure track positions. (If you got a paper or two into those two
publications).

However, people still had to like you in the interview and job talk.

~~~
s0rce
I applied, gave up, just didn't feel like it was all worth it in the end and
now work in industry.

~~~
subroutine
I'm assuming these were not 1st author publications. If you have a 1st author
Nature and a 1st author Science paper during grad school, Universities should
be rolling out the red carpet.

~~~
s0rce
They were both first author, details are on my very out of date website link
in my profile if you are interested. Definitely no red carpet rolling out, not
sure if you are in academia but those days have passed. Could I possibly have
done another postdoc and managed a faculty position somewhere in the USA,
probably, was it worth it to me, nope. Felt like I failed for a while, its
hard to escape from the academia cult.

~~~
subroutine
Whoa. Well first congrats, that is a triumphant feat for a grad student.

I am truly surprised your faculty search wasnt more fruitful, given there is
hardly much more you could have done other than maybe secure a K99 grant.

Why do you suspect your cv was not top-tier competetive? I ask because yes, I
am in Academia and from my experience (including being on faculty search
committees) suggests you would have at least been invited to job talks.

~~~
throwawaybbq1
Heh .. I don't want this to turn into an AMA. But yes .. I too have questions.
How long have you been away from academia and how do you feel about it? I had
success in grad school (papers in top CS conferences). I went the industrial
research route which paid well initially. Now, I'm nearly 40 and have no
tenure and a gap of a few years in my publication history. I am very tempted
to get out of the game but am afraid about it being a 1 way street.

~~~
s0rce
Hah, I don't really enjoying revisiting my decision making but I'll try to
answer. I've been away for just over a year and I miss the people and the
freedom but I don't miss the politics and the rat race.

Grad school was successful. However, my postdoc was a different story, I got a
relatively prestigious fellowship at a DOE lab but my supervisor quit before I
started and I tried to do my own stuff but really didn't get much support and
it wasn't much fun or productive. I quit after 2 years, about 1 year ago and
joined a small biotech company in the bay area. I'm pretty sure I already have
too large a gap in my publications to go back, I thought about going back to
academia for a bit but that would have entailed doing a 2nd postdoc but I
really wasn't interested in moving some random place for 3+ years, then
dealing with the whole faculty application process again and 5+ more years
before eventually getting tenure.

If you don't hate writing grants maybe academia can be for you.

Good luck.

~~~
subroutine
Do you have any interest in neuroscience? My lab (malinowlab.com) has two
postdoc openings right now at UCSD, and we take smart people from any STEM
field.

------
joshvm
Certainly not directly, but in most universities publication output is the
main driver for salary negotation (and advancement on tenure track). Most
places will put you on a standard scale which increments each year, and having
a top tier publication may be enough to negotiate a bump of a couple of rungs
(worth a thousand or two pa in the UK).

I would imagine the same applies to most jobs - rather than an on-the-spot
bonus, if you published in a prestigious journal as part of your job, it would
be strong grounds for some compensation at your next review.

------
efficax
In addition to the prestige and career boost, you get a a really swell awesome
feeling and get to be a permanent part of the encylopedia of knowledge. You're
in libraries now!

~~~
azag0
In chemical physics, many of the most influential papers (tens of thousands of
citations) have been published in The Journal of Chemical Physics, an
absolutely non-flashy publication with impact factor ~3 and a myriad of
uninteresting papers.

------
Vinnl
In addition to providing more incentive for manipulation of result and flashy
research, it also rewards researchers not for the contents of their research,
but for where they manage to get it published. Especially the "top tier"
journals place emphasis on noteworthiness, disincentivising e.g. replication
studies, and often with a higher number of retractions [1].

It also means that the position of the traditional, subscription-based
journals are cemented more, even though many funders are also aiming to
transition to open access publishing.

So overall, I guess I'm not that enthusiastic about this.

[1] [https://www.nature.com/news/why-high-profile-journals-
have-m...](https://www.nature.com/news/why-high-profile-journals-have-more-
retractions-1.15951)

------
moh_maya
1) For assistant professors, a better chance of tenure

2) Grants!

3) becomes easier to attract good postdocs, which helps you publish good
papers faster, which helps...

4) Grad students

------
knolan
You'll get asked to review more papers.

------
notyourday
Paper copy of the journal. Authors go absolutely apoplectic if they don't get
the paper copy mailed to them.

[Source: Pillow talk]

~~~
s0rce
I ordered extra hard-copies for my mom and grandmother, I think they cared
more about the papers in Nature/Science then anyone else.

~~~
sndean
Yeah, I was 2nd author on a Nature paper. The only person that cared was my
mom.

------
jstandard
How does intellectual property work for published research based on approaches
developed while consulting for a company?

For example, let's say you develop a novel modeling methodology for a company
who hired you. You'd like to publish the methodology and give conference talks
on it. Since you're hired, your work and the methodology would belong to the
company as their asset.

Are there certain types of licenses applicable here? How about in the case of
the article where researchers are compensated for presenting a talk on that
methodology?

~~~
icelancer
>>How does intellectual property work for published research based on
approaches developed while consulting for a company?

This is negotiable upfront for industrial research positions. You should
always review your IP contract inside your job offer packet and battle
HR/management hard on the rights to the IP and compensation surrounding it.
Most boilerplate language assigns all IP rights to the company you work for
with a maximum of $1 consideration paid for your work (to satisfy contract law
if it is ever argued in a court), with acknowledgements or minor credit to you
in the official documentation.

Needless to say, you shouldn't blindly agree to that unless your base salary
is commensurate with such a loss of control and future rights of your work.

~~~
jstandard
Thanks, that's helpful as an upfront approach. How about if a contract is
already in place and the company is okay with the methodology being published,
as long as they retain rights to it.

Is there a licensing vehicle that allows for publishing while remaining an
asset of the company?

For example, Square built their Dagger library and uses the Apache license.
[http://square.github.io/dagger/](http://square.github.io/dagger/) My
impression is that Dagger is still an asset of Square, but you can use it as
long as you meet the licensing requirements.

Is there an equivalent to this for a modeling approach or work done by a
consultant?

~~~
throwawaybbq1
IANAL. However, as an industrial researcher in CS, the standard is this. Once
you patent your work, the company is okay with you publishing. There are
exceptions. There is a story (no idea if true of not) that the folks who
published the dynamo paper almost got fired. Some ideas are so important to
the business that they may not be patented and just reserved a trade secret.
All of this is negotiable and often done on a case by case basis. Most
employers I've had have a standard reviewing process where the researchers
submit the paper (to be submitted) and we get an approval that it is okay to
publish. There is typically a checkbox that asks if you sought protection for
any IP in the paper, etc.

~~~
jstandard
Thanks, very helpful perspective. Once a paper is reviewed and approved, are
there any rights around being able to present that paper at a conference?

I'm trying to understand what kind of legal transformation takes place one a
paper on a non-patented methodology is released. I've skimmed the dynamo paper
and it seems a good analogy in that it's about general database architecture
concepts, rather than the specific implementation at Amazon. I might be wrong
though as I haven't read it deeply.

For example, let's say you're the primary author of a paper and then later
leave the company. Can there by any legal restriction preventing you or others
who have never worked at the company from giving a conference presentation on
that paper once published?

------
Animats
25 free reprint copies?

------
vixen99
A lot of action on SciHub

------
quickben
If you are not planning a career in academia, wouldnt it be better to patent
it?

Just wondering...

~~~
PeterisP
Most research is not patentable.

Technical development and design that results in some "method and apparatus"
to do a particular thing is patentable, but the research required before that
R&D that leads to understand how stuff works and what might work is not
patentable. And that's not speaking about less technical fields of science
where _nothing_ is patentable, or computer science which is not patentable in
much of the world (software patents aren't valid in EU and other places).

Also, patenting is _expensive_. Especially because you need to file a patent
in multiple major markets, you need multiple different patent lawyers, if
you're outside of USA then filing a patent in USA is a bit of a hassle, and
just for USA+main EU countries gets quite costly quick - so that's a reason
not to patent things unless you have a clear picture that some product is
going to violate that patent and thus it'll become valuable.

~~~
santaclaus
> Most research is not patentable.

In my experience, the lawyers just chuck 'A System for' in front of the
paper's title and open the firehose at the patent office. Lots of ridiculous
stuff gets through...

------
asdffdsa321
An interview at google?

------
bantersaurus
Street cred

------
abfan1127
high fives.

------
amelius
So I suppose it's often better to turn your idea into a company instead.

~~~
juskrey
It is _always_ better. The thing is most of the journal papers are useless in
the real world.

------
kazinator
If your roommate is Shelly Cooper, you get a sticker!

Possibly one depicting a kitten which says, "me-wow!"

------
paulpauper
It means a lot of status and prestige on Reddit and other 'smart' communities,
especially if the paper is in a STEM subject. Outside of academia and offline,
very little.

