
We need to talk about systematic fraud - nabla9
https://www.nature.com/articles/d41586-019-00439-9
======
twic
There was an interesting comment on a recent In The Pipeline post about the
relationship between academia and industry in drug developmemnt [1]:

 _Academics do some good work in target discovery and validation, but not
anywhere near as much as they think they do. Discovery in big pharma has to
repeat everything, because about 80% of the time, the published result doesn’t
hold up. I suspect this is the result of no job security among grad students
and post docs, because we can almost always reproduce data from Japan, where
asst profs generate the data. (And we can always reproduce published data from
other big pharmas, always.)_

[1]
[http://blogs.sciencemag.org/pipeline/archives/2019/02/01/rep...](http://blogs.sciencemag.org/pipeline/archives/2019/02/01/rep-
ocasio-cortez-and-where-drugs-come-from#comment-300356)

~~~
earthicus
I'm hesitant to take an anonymous and unsourced internet comment at face
value, but assuming it's true, that is quite damning for American academia! It
occurs to me that the claimed effect, if it exists, might also be due in part
to simple expertise. Or rather a _lack_ of expertise among grad students and
to a lesser extent post docs.

Is anyone here able to corroborate the claim made in OP's linked comment? And
does anyone know what kind of experience and oversight the researchers doing
discovery in big pharma will have?

~~~
Balgair
I'll echo the same from other commentators.

There are a LOT of issues with reproducbility in Bio [0] at the moment, and in
US based science in general. From others in 'Big Pharma' that I know, they
agree that much of bio-tech based science needs to be re-run. Others in the
comments section here point out that Mainland China has a history of suspect
research. That may have been true 5 years ago, but they are 'up to par' with
US based researchers now. Meaning, that the Chinese have gotten more rigorous
and the US has _severely_ slipped in quality.

I mean this next statement with full honestly: I don't believe half the labs
out there in the US these since about 2016 onward. From _very_ private
conversations with others, many labs are abusive, fraudulent, 'me-too'
landmines, and generally lying to grant funders, donors, and
potential/enrolled students in a strange game akin to 'Catch Me If You Can'
style con-artistry.

Major efforts are being taken to 'fix' the system. Me-Too is a perfect
example. The _Nature_ paper that is linked is another perfect example. These
efforts are very much needed and those that undertake these efforts should be
applauded and emulated. That said, Max Planck's quote is the speed at which
this change will occur, namely, one funeral at a time[2].

[0] [https://www.reuters.com/article/us-science-cancer-
idUSBRE82R...](https://www.reuters.com/article/us-science-cancer-
idUSBRE82R12P20120328) A bit old, but should get you started on more up-to-
date research.

[1] [https://www.theatlantic.com/health/archive/2018/09/what-
is-f...](https://www.theatlantic.com/health/archive/2018/09/what-is-food-
science/571105/) Food Science, but it illustrates the structure that leads to
fraud.

[2]
[https://en.wikiquote.org/wiki/Max_Planck](https://en.wikiquote.org/wiki/Max_Planck)

~~~
earthicus
> From very private conversations with others, many labs are abusive,
> fraudulent, 'me-too' landmines, and generally lying to grant funders,
> donors, and potential/enrolled students

I have a friend who worked in a neurology lab a few years ago who described
the PI in a similar way. I had assumed that it was not normal and told her to
get the hell away from that lab asap... how sad that this kind of shortsighted
idiocy is so widespread!

------
luminati
Why not call a spade a spade. Better title would be "We need to talk about
systematic fraud in China"

It's hardly a secret. Any grad student in America already knows this - about
the blatant/incredulous cheating and plagiarism that goes on between Chinese
students. But oh no, can't talk about it. 'Coz it's racist y'know.

~~~
falsedan
It is racist! Making generalised judgements about an entire group of people
based on their ethnicity background is classic racism.

~~~
pault
China's population consists of more than one race. I'm all for political
correctness but we need to distinguish between racial stereotyping and
criticisms of a nation state.

~~~
twic
Moreover, between racial stereotyping and criticisms of _certain policies of
the current government of a nation state_.

To choose the maximally flamebaity example, i might not like the current
Israeli administration's policy towards Palestinians, but that doesn't make me
an antisemite, or even anti-Israeli.

------
bounder
Wouldn't it be nice if there was a platform, some kind of hub where all
research papers could be published and visible for anyone with an interest in
the particular field. The platform would allow anyone, be it academics or just
field experts to request for a change or addition of text. Anyone could see
these (pull)requests and the initial author has the power to accept or deny
it. This allows for peer reviews and corrections being integrated into the
paper, making the whole process more organic. Any change to paper is logged
and tracked so you could see the origin of a change and it's reason.
Fraudulent papers could be marked by the research community as such.

I'm being a bit sarcastic here, but a technology such as git with a platform
such as GitHub could maybe make the process of publishing and reviewing
research papers easier to control. Maybe you have to tailor it a bit to make
it easier to work with this use case. Like open source, peers can start adding
and reviewing directly at the source, instead writing a new paper or writing
to all of the scientific journals to issue a different statement.

~~~
ordu
I have a slightly another idea. Any research activity of a scientist should be
recorded by him/her in his/her own repository. Any activity, like reading a
paper, or moments like "I have an idea", or "lets try a small scale pilot
experiment with fellow scientists of mine as a subjects", or "pilot experiment
was unsuccessful, but can be modified...".

Any publication should be forked from the main repository, selecting from any
relevant data from parent, and in that fork strict procedures of scientific
research are recorded, processed, and finally end up with an article.

In this case you can track how an idea was formed, how it was modified to
match empirical data. You can track when methods of data processing was chosen
(before gathering of data or after), and so on.

Such a repository can give enough information to decide was the research is a
fraud or no.

~~~
sn41
I am not sure about this... are you talking about experimental research? I am
a theoretician. I go through several idiotic ideas before hitting on the right
one. I'd be embarrassed to admit my silly, initial attempts before my
colleagues.

The advantage of theoretical papers is that the reasoning is a proof/argument
developed in the paper itself. With theory, there's less chance for a
fraudulent paper, even though the results may have gaps.

~~~
ordu
_> I go through several idiotic ideas before hitting on the right one. I'd be
embarrassed to admit my silly, initial attempts before my colleagues._

Thats the point. If you published your preliminary silly idea and your
colleagues didn't find immediately that it is silly, then your not sillier
than they are. :)

 _> the reasoning is a proof/argument developed in the paper itself_

Even when I studied math I found sometimes that just to have proof is not
enough to understand. The only way to understand sometimes is to learn how the
proof was devised. What the hell author thought to get an idea of some trick
in the proof? Some tricks looks as a magic, you see they are true from a
logical standpoint, but they make no sense. You cannot understand _how_ they
lead to a QED. You can verify proof by logic -- not a problem, but you unable
to understand. And if you are unable to understand, then at least you are
unable to reproduce this proof at the exam, the only way is to learn it by
heart. By I never trained my memory to learn by heart, it is really hard for
me and annoying, and moreover I do not believe in learning by heart: to master
a subject you need to understand it, not to learn it by heart.

My math experience taught me that if I cannot follow the thoughts of other
mathematician by just reading his proof, it means that he is smarter than me,
so it doesn't matter then how stupid he behaved while searching for his proof,
he outsmarted me already.

But yes, I was talking about experimental research.

------
ThePhysicist
Journals became more and more obsolete with the rise of Internet and only
managed to retain their position in the academic world due to their citation
metrics system that is used to judge academic worth and the enormous amount of
articles that they've amassed over the last centuries and that academics need
access to. I think it will take at least another 30 years until we will be
able to stop relying on closed-access journals, possibly longer since there
are still countless high-profile articles that only get published in those
journals (though some universities and funding agencies changed their policies
to demand that results of publicly funded research are freely available).

In Physics and many other fields Arxiv.org has basically replaced journals as
the primary way of getting the latest research results. Unfortunately many
high-impact-factor journals like Nature forbid you to publish a preprint of
your article there (at least that was the case in 2012) so that they can
defend their claim of publishing "cutting-edge" results only (which due to
their long review processes are often 12 months old by the time they're
actually published).

The most ridiculous thing is that the journals today do little more than act
as an intermediary between the authors of a paper and the reviewers, which
work for free and are colleagues from the same field. That and maybe styling
the papers a bit, which is (IMHO) unnecessary in most cases as the source
LaTeX version usually provides very good design already.

~~~
speedplane
There are certainly problems with journals, but they do provide one benefit:
they are a source of trust. If you read something in a major publication, you
can trust it more than if it was just posted to a website. With the internet,
reliable sources of trust are becoming more important, not less. I'm not
condoning the current extremely expensive journals that prohibit access to the
public, but a completely open model where anyone from anywhere can post
anything isn't the answer either.

~~~
username90
> If you read something in a major publication, you can trust it more than if
> it was just posted to a website.

I'm not sure that is true, I've studied a fair bit of statistics and I can
usually find flaws in most social science studies I read. So I trust an
article combined with a few hundred HN comments a lot more than I trust
anything published in scientific journals without comments, since usually if
something is wrong someone in the comment section will find it. Even on reddit
if you wade through it you can usually find some nitpickers adding good
discussion on articles with a few hundred comments.

~~~
asdff
There is still a much lower acceptance rate at a good journal than a bad
journal that might accept anything and everything that is to the page limit.
You might still get a faulty paper of course, but large journals have the
benefit of choosing.

------
choppaface
Why can’t the peer review system itself just evolve to embrace the full power
of the internet? The main reasons these fraudulent strategies are at all
effective is because (1) the review process is private and slow and (2)
studies aren’t independently reproduced as a part of review. The feedback loop
is inherently too slow. To speed it up, one must fight the inertia of senior
(tenured) faculty and major funding vehicles.

It’s laudible these researchers have invested so much in fighting fraud, but
this post shows that the offenders will essentially face no negative
consequences for fraud. The researchers worked very hard for hardly any return
/ justice.

Perhaps it’s worth re-inventing science. Open reviews. A network to connect
labs that ensures findings are reproduced. We could even re-do much of basic,
academic science in the interest of improving exposition and shedding
antiquated formulations of theory.

What if, in the face of this fraud, we simply reject tradition wholesale? The
only tradition should be an honest, reproducible study of Nature.

~~~
CJefferson
My problem is, just the "Full power of the internet" end up looking like
Facebook, or Reddit, or Twitter, where the fastest and loudest repliers often
end up having the biggest effect?

Also, independently reproducing studies is super-important, but that going to
take HUGE amounts of money, which someone is going to have to pay for. It was
have to come from the top, most governments want to see new research, not
reproductions, and scientists are guided by that direction.

~~~
ausbah
"Full power of the internet" doesn't have to be like Facebook or Twitter.
Maintaining some sort of decent moderation and having a wider "professional"
culture (can't think of a better term here) can allow for some truly
insightful interactions in the vein of HackerNews or some better known
subreddits.

------
allset_
There was a good talk about this at 35C3 [0]. Definitely worth the watch.

[0]
[https://media.ccc.de/v/35c3-9744-inside_the_fake_science_fac...](https://media.ccc.de/v/35c3-9744-inside_the_fake_science_factories)

Edit: You can change the language to English by clicking on the settings cog
in the video.

~~~
hk-mars
Thanks :)

------
OliverJones
This author and her collaborator developed a tool to flag potential issues in
papers. It appears to be FOSS, and based on a well-understood piece of
bioinformatics code. "Nature" doesn't run material from cranks and misfits, so
hopefully this will start a serious discussion.

Going forward, should journals, or the peer-reviewers they engage, test new
papers with this tool? Sure, it turns up false positives and false negatives,
so its results cannot be interpreted mindlessly. It is also possible to game a
tool like this. But it seems obvious some authors are already gaming the
system.

Undergrad teachers use software tools to check student writing for plagiarism.
Shouldn't information-based science use similar tools?

We hackers can contribute to this kind of effort by encouraging and supporting
open-source inspectable tools. Maybe there's even a business opportunity
around maintaining and using the tools.

------
JackPoach
I thing lots of wrong people go into science for the wrong reasons. This is
why you'll never really be able to do anything about this problem, unless you
have a way to screen them out.

------
justinclift
> Finally, efforts to police the literature need to be valued as highly as the
> publication of original data. It is more than ironic that systematic fraud
> is itself understudied.

Wonder if doing something like establishing "Bounties" (aka Security Bug
Bounties in software) would start things in the right direction?

Sounds like there'd be no shortage of targets and low hanging fruit though.

Also, who'd fund it?

------
JPLeRouzic
One single meaningless test: I am not a scientist and do not want to draw any
general lessons but I took a document about an ALS gene therapy that I wrote
for private use, and used the tool mentioned in the article. It found a
distant article about a tumor protein. Indeed both documents are unrelated,
mine was not published and they have very little in common, only the word
protein.

~~~
JPLeRouzic
I did it again with:

Dietary salt promotes neurovascular and cognitive dysfunction through a gut-
initiated TH17 response

[https://doi.org/10.1038/s41593-017-0059-z](https://doi.org/10.1038/s41593-017-0059-z)

And it found:

Knockdown of NOB1 expression inhibits the malignant transformation of human
prostate cancer cells

doi: 10.1007/s11010-014-2126-z

~~~
sgt101
Apologies: I'm being obtuse.... Which tool?

~~~
JPLeRouzic
This one:

[http://scigendetection.imag.fr/TPD52/](http://scigendetection.imag.fr/TPD52/)

Citation: Informatician Cyril Labbé at Grenoble Alps University in France and
I have developed a tool, Seek & Blastn (go.nature.com/2hsk06q), ___to identify
such papers_ __on the basis of wrongly identified nucleotide sequences.

So far, our work has uncovered dozens of papers and resulted in 17
retractions, with several investigations pending (see Nature 551, 422–423;
2017).

------
kyloon
The scientific process typically involves making inferences and conclusions
based on an observation, whereby the result often include biases and in some
cases even ill-constructed hypotheses that are caused by assumptions and
imperfect information.

One way to reduce 'fraud' and also increase accountability of the individuals
and organizations is to secure raw data at its source which the veracity of
the data that will be subsequently used to support downstream conclusions can
be easily verified and attributed to its source. I'd say a bottom up approach
here is more holistic as it incentivizes people to produce high quality
research that future work can then be reliably be built on top of.

------
nostrebored
Why does the author assume that this isn't a norm among researchers? The sheer
number of irreproducible studies we have should indicate that this behavior is
way more common than people would like to admit.

------
hkon
Every time i see a headline with that wording "we need to talk about x" I just
ignore it. Often it's just click bait into something that is not really up for
serious discussion and should rather be "I need to tell you what to think".
I'm writing this to reduce the usage of it. And no, I will not write an
article with that title about that subject.

~~~
joppy
However, this is an article written by a researcher in a particular field,
published on the website of the best-known and most reputable journal in that
field, trying to get a message out to members of that same field, about a
topic that nobody wants to discuss. It seems like a pretty appropriate use of
"We need to talk about ..." to me.

------
rmrfrmrf
> Finally, efforts to police the literature need to be valued as highly as the
> publication of original data.

I don't think even the author would agree with this if it were about anyone
other than them.

------
master_yoda_1
What about fraud in machine learning and artificial research research? Just to
get a high paying industry job these professors do fraud research and claim
them self as AI expert. I think the next bubble will bust when these
fraudsters sitting in industry won’t deliver anything. Because all the company
foolishly spending so much money on fraudulent research without any productive
result. Look at Facebook ai research. They can’t even control fake news and
their researcher are trying to solve ‘general artificial intelligence’ :)

------
dqpb
Is it fraud or disinformation?

------
djsumdog
I think back to the recent group of academics that wanted to prove how most
Grievance Studies journals don't property review submissions. One of the
papers they got accepted had chapters from Mein Kampf rewritten with
terminology from modern social cause rhetoric.

Instead of accepting the issues, with the entire process, these professors are
now facing disciplinary hearings for submitting false research. In a way I can
understand this; they did after all submit papers they knew weren't academic
(in bad faith?) in order to prove a greater point. But does that justify what
they did?

~~~
ATsch
I think it's surprising that on a site such as HN the narrative of these
authors is so readily accepted without any second thought.

The claim is that they just made up trash and submitted it to high-quality
peer-reviewed journals and they accepted them. The reality is much less
outragous.

Of the 20 papers, 4 were published and 3 more accepted for publication. They
spent a _year_ doing this. The authors note that they submitted the papers to
"top journals in the field", leaving out the fact that all of those top
journals rejected the 20 papers and they had to revise the papers and send
them to shoddier journals instead.

The review quoted by the authors was from an unsure grad student doing their
first review of a paper. Said student later said they knew the paper was crap,
but still tried to be constructive.

The assertion of the authors is that this stuff getting published means the
whole field is worthless. Which is of course ridiculous. Medicine didn't
become obsolete when a journal published a paper with an abstract ending in
"The fact that these last sentences appear in the published paper tell you,
dear reader, exactly how seriously the editorial process has been taken".
Computer Science didn't implode when three MIT students made an AI to generate
random papers and 120(!) were accepted by the reputable IEEE and Springer. The
blame was back then correctly put on the publishers and the general flaws of
the current scientific process instread.

So, as people have pointed out, it's ironic that their essay itself is
incredibly scientifically unrigorous. No controls, unsourced claims, biased
authors and conclusions that aren't supported by the data.

~~~
prepend
“The assertion of the authors is that this stuff getting published means the
whole field is worthless.”

I don’t get this at all from the authors. My take is that they call out
serious flaws in the rigor of how papers are reviewed. I don’t think it’s
reasonable to extend that to say the field is worthless. I did read through
many papers in the referenced journals and it seems really surprising to me.
I’d love to see some of the research reproduced. But I’m not even sure how you
could scientifically prove a field worthless.

~~~
ATsch
> I don’t get this at all from the authors. My take is that they call out
> serious flaws in the rigor of how papers are reviewed. I don’t think it’s
> reasonable to extend that to say the field is worthless.

I think it's rather undeniable that the Authors essay was not a good-faith
attempt at criticizing the peer-review process, but an attempt to discredit
certain fields that they didn't like, even by their own admission. I mean, the
title of the essay alone should make this very clear. Here's some quotes
nonetheless:

> We spent that time writing academic papers and publishing them in respected
> peer-reviewed journals associated with fields of scholarship loosely known
> as “cultural studies” or “identity studies” > As a result of this work, we
> have come to call these fields “grievance studies” > We undertook this
> project to study, understand, and expose the reality of grievance studies,
> which is corrupting academic research > The biggest difference between us
> and the scholarship we are studying by emulation is that we know we made
> things up. > these fields of study do not continue the important and noble
> liberal work of the civil rights movements; they corrupt it while trading
> upon their good names to keep pushing a kind of social snake oil onto a
> public that keeps getting sicker

[Note also that this essentially an academic way of phrasing the same things
you'll hear people like the far-right Stefan Molyneux (on who's show one of
the authors has been a guest several times) say.]

Of importance here that it's not the scientific practices that are "corrupting
academia" but the fact that these fields exist at all. I'm sure you can find
more if you really want to. Compare this to say the "reproducibility in
Psychology" study, which unsurprisingly doesn't dedicate half of their text to
talking about why the authors think psychology is bad.

Don't be mistaken: The whole thing was an attempt to cash in on the general
crisis the scientific process is facing to score some easy political points
and not a neutral and constructive attempt at improving the quality of
research in the field. Or rather, their proposal at improving the quality
would involve getting rid of the fields that they don't personally like.

------
hk-mars
Good talk about xx community’s intention.

