
80% of current drug applications in China failed to meet analysis requirements - ting_bu_dung
http://www.sciencealert.com/80-of-the-data-in-chinese-clinical-trial-is-fabricated
======
projektfu
In the couple years I spent doing biology research, about 8 years ago, I was
skeptical of most research from China because it already had a reputation
already of being untrustworthy. But why stop at China?

I spent a good part of a year trying and failing to replicate an English
study, and looking around for materials referenced in another study that
simply don't exist, finally being told by the principal on that study that
they mischaracterized the materials in the paper. Sometimes when you get the
results you are hoping for, you publish, and you don't care if they were
wrong, right?

Perhaps it would be better if scientists generally took a more skeptical view
of others' publications and tried to replicate them in house. At conferences,
instead of asking, "Where do we go from here", with the first study, ask,
"What do I need to do to replicate this?"

~~~
nonbel
I had a very similar experience. And you are correct that these problems are
not at all limited to "China", although those authors tend to be a little less
sophisticated in their (what may generously be called) shortcuts.

For example, if you do a _real_ literature review on a medical topic (not just
credulously take their word for it) you are almost sure to find that one of
the primary assumptions/claims was actually never studied. One paper refs a
second which refs a third that speculated about it in the introduction, but
now "everyone knows it".

~~~
projektfu
So true. This is especially important as people study ways of improving
metrics of markers of disease instead of actually improving disease endpoints.
For example, there is a supplement marketed in the veterinary space that may
lower blood urea nitrogen in uremic animals, according to a study. Whoop-de-
do! I don't care about urea, it doesn't cause the disease of uremia. It is
just a marker. You could convert all the urea to ammonia and now have a
patient with two diseases and a BUN of 0.

------
eveningcoffee
Considering past occurrences
[https://en.wikipedia.org/wiki/2008_Chinese_milk_scandal](https://en.wikipedia.org/wiki/2008_Chinese_milk_scandal)
and general behaviour in online market places (my last experience was that the
product was advertised to be shipped from Germany (expect fast delivery), but
arrived a month later from China) then it should not come as very big surprise
(I am not saying that this is normal or acceptable, but that it is not very
surprising).

My question is that what is the cause of this general fraudulent behaviour?
Does it have some historical roots? Does it come from lack of actual
functioning oversight?

Another anecdotal case: try asking for directions in China. Even when person
does not know the correct directions, they do not mind to direct you, and when
confronted, they refuse to admit that they do not know. Is this mindset result
of constant fear of repressions, or it runs deeper into the history of the
society (say it is also present in Taiwan?).

I am not accusing anybody in anything, I just would like to understand.

~~~
hzhou321
Historically scientists are lonely people who are (mostly) motivated by
seeking truth or knowledge and satisfying curiosity instead of money or fame.
Most people are practical and are motivated by money or fame more than
satisfying curiosity. So most people are pursuing jobs rather than science.
Fraudulent behavior is simply a result of seeking shortest path (shortcut) to
a goal. When the goal is (short term) money and fame, and when the gain of
taking the shortcut of fraud is bigger than the negative impact, then the
popularity of the short cut is well expected.

So the problem is we reward so-called scientist too much (in terms of money or
status). Scientists were historically not rewarded by money or status. In fact
I don't think there is traditionally a career of science. People then were
doing science on the side of their regular jobs (such as engineers or teachers
or nobles who don't really need a job). Only when the reward of doing science
is detached (to certain degree) from the reward of money and fame, can we
truly prevent the fraudulent behavior.

On the other hand, the allure of money and fame probably can never be truly
detached for any normal human, so fraudulent behavior is part of the human
nature. It ebbs and flows. There is no need to make a big fuss about it.

PS: from what I heard, China today has so much (relative to its capacity)
science money floating around and that got to distort the goal of science
significantly.

~~~
maxander
I would have a hard time swallowing the argument that we pay scientists too
much, or that society respects them too much. A distant observer might _think_
this could be the case since, from their perspective, what they see as
"scientists" are celebrated professors, showered with job stability, upper-
middle-class incomes and book deals. But those lucky sods are vastly
outnumbered by postdocs and tenure-track faculty, who work thankless hours at
minimal pay and who could put their brains to work _anywhere else_ for better
reward. Those are the "typical scientist," on average, these days, and if we
paid them any less they could barely afford to eat.

The problem is the imbalance- a postdoc _really really wants_ to get tenure
someday, because the only alternative is for his career to peter out and to
end up working ignominiously in industry (as a second-rate software engineer,
most likely.) And the number of tenure positions that could possibly be
available is an order of magnitude fewer than the number of smart people
competing for them. So he'll possibly "cut corners" to get there, despite the
devotion to his field that made him choose that career and not something
reasonable, like finance or software engineering. Academia is tenure or bust-
there's no stable middle option.

There is a correction under way as people start to realize that academia is no
longer a feasible career path, in the same way that the NFL isn't. So I'm
hopeful that things will stabilize in regards to scientific integrity
regardless- but, then we'll have to figure out who is going to _do the
science_ when we can't rely on an army of poorly paid geniuses.

~~~
hzhou321
I won't deny the reality of modern day science worker. But the fact of they
are paid/respected at minimum does not justify their motivation. Just like a
typical minimum wage worker is not doing the work out of pleasure, a typical
postdoc is not pursuing a career of pure knowledge. Of course there are some
component in it, but from what I see, they are pursuing a career -- a tenure
as you put it, an end result that is secure in income and high in status of
respect and rich in freedom -- even when that is more of an illusion. So the
bottom line is I don't believe there are as much people as many scientists
today that are motivated by curiosity alone. They are pursuing money and fame,
even when that is an illusion (to certain extent).

The current science career is more of an engineering career, where people have
clear goals with constant feasibility assessment and motivated and encouraged
to seek short cuts.

So I am not saying we are paying scientists too much. I am saying scientists
don't need to be paid, only need to be sustained. Not a many people would be
happy for a career that is merely sustained, but not many people are truly
born to be a scientist -- think about Einstein being happy at a patent office.

EDIT:

So I agree that look at the way today we do science, it is of concern.
However, I think that is largely a mislabeling. Today's science career is more
of a engineering practice; and look at the way we do engineering, we are doing
fine today. Some place some people build a shoddy bridge, it is something of
gossip, but not much of concern.

Think about it: science is not supposed to produce a product (medicine in this
context) but it is supposed to answer some questions (not given by the society
but of one's own). To answer a demand or solve a question (with a belief it
can be answered), that is engineering.

Of course, engineering is important. And there is nothing wrong for a few
people pursuing science on the side while doing their engineer jobs.

~~~
maxander
Ok, the vision of scientists as an ascetic class only appealing to the dyed-
in-the-wool seekers of knowledge has some appeal. But, there's a problem
there, too- sure, Einstein would still have published his breakthroughs, but
many of the thousands of researchers whose works were the basis for his
wouldn't have. The pace of scientific progress we're accustomed to now is the
product of the work of thousands upon thousands of scientists; the
contribution of celebrated geniuses like Einstein are dramatic but represent a
rounding error compared to the whole rate of progress. We need a way to
recruit and retain the rank-and-file ordinary-human researchers as well, and
that requires compensation somewhat proportional to the effort they're putting
in.

~~~
hzhou321
I wouldn't go out and say Einstein is a rounding error :).

When we say science progress is built on top of giant's shoulder, that giant
don't have to be and often is not just other/previous scientists. Engineering,
culture, or even witch crafting all have contributed to science.

We need a lot of rank-and-file ordinary-human doing their ordinary works out
of ordinary motivations, they will provide the basis necessary for the a few
true scientists to question the known and explore the unknown.

------
brudgers
_Radio Free Asia (RFA) is a private, nonprofit international broadcasting
agency of the United States government[2] that broadcasts and publishes online
news, information, and commentary to listeners in East Asia while "advancing
the goals of U.S. foreign policy."_

[https://en.wikipedia.org/wiki/Radio_Free_Asia](https://en.wikipedia.org/wiki/Radio_Free_Asia)

The source cited is Radio Free Asia not the report itself. Radio Free Asia
does not link to any additional sources.

That is not to say that the story is inaccurate, but rather that the claims
are not well substantiated and that the source is not apolitical.

~~~
eveningcoffee
This is good catch, but considering the history of such programs
([https://en.wikipedia.org/wiki/Radio_Free_Europe/Radio_Libert...](https://en.wikipedia.org/wiki/Radio_Free_Europe/Radio_Liberty)),
it is that they publish rather information that has been suspended by local
governments than something that has been made up.

Of course you should be cautious with what they publish, but this applies to
every published information.

~~~
brudgers
From the RFA story:

 _data gathered during clinical trials were incomplete, failed to meet
analysis requirements or were untraceable, the paper cited a source in the
agency as saying._

Outside of China, clinical data is often not shared, suppressed when it does
not support the claims, and p-value hacking is used to justify 'off-label'
use. This is not usually spun as fabricated, and in commercial drug
development my intuition is that 80% of proposed drugs having at least one of
these features would be a plausible hypothesis

------
Odenwaelder
In chinese culture, losing one's "face" is one of the worst things that can
happen to a person. I'm wondering whether the fact that 80% of data has been
fabricated is a result of that. As in, "I have to make my findings significant
to not lose my face", or, "Presenting negative findings to my superior will
make me lose my face". Maybe someone more familiar with this culture could
elaborate.

~~~
finid
Wait a minute!

That seems to be backward. Doesn't "losing face" come from being caught doing
the wrong thing or being dishonest, so that the incentive to fabricate data
for something as important as scientific research should be very, very low?

~~~
HillRat
I'm not sure that reputational risk is the same thing; I suspect one way to
look at it is that a researcher who returns the null case is disrupting the
organization by putting a lot of other peoples' work and assurances at risk.
So there would be a significant amount of implicit social pressure to show
results and maintain organizational harmony.

~~~
794CD01
That sounds like academia anywhere.

~~~
finid
Yep, _publish or perish_ applies to academia everywhere.

------
canada_dry
No wonder wealthy Chinese go out of their way to buy not-made-in-China food
and medicine.

------
DanielBMarkham
Science is having an astounding crisis -- I don't think there's been anything
like it in history before.

If this is true about Chinese clinical data? More generally, if there is any
simple tag that covers fraud at this scale? Other people working under that
tag will experience deleterious effects for years to come.

The only way I see out of this is a rigorous, self-imposed body of standards.
Even then I'm not sure that this will be sufficient.

Think of all those people, all of that money, all of the hundreds of thousands
of hours of research. For nothing. Actually, it's worse than nothing because
it's been flooding the system with fake data.

~~~
Odenwaelder
It's not only about fake data, it's also, maybe even most importantly, the low
quality data. Publications with low data quality decrease the signal-to-noise
ratio, leading to a slower scientific progress. Purely from a scientific
standpoint: If you identify fake data, you just take them out of your model
training set. But identifying low quality data is much harder.

~~~
DanielBMarkham
Agreed. Poor quality data is worse.

A couple of years ago, as this story started to break, I would mention it to
scientist friends. "Don't worry," they'd say, "We always have the meta
analysis to fall back on."

I don't hear that anymore.

ADD: A critical issue here, which you touch on tangentially, is the mix of
motivation and milieu. If I'm making stuff up -- bad data -- other people can
identify it and delete it. But if my goal is to _appear to be doing good
science_ instead of actually doing it, then it can become extremely
problematic both to identify and remedy what I've done.

A lot of "normal" science is done around the margins, with not-so-incredible
hypotheses and pedestrian-looking data. Bad science with really poor data fits
seamlessly into that model without having any distinguishing characteristics.

------
tlb
It mentions 1622 current clinical trials. Of those, how many would be expected
to be approved in a rational system?

I wonder if the huge %age is due to a lot of low-quality applications mixed in
with the serious ones. If the barrier to apply is sufficiently low, you would
expect a Sturgeon's law dynamic.

~~~
Alex3917
> I wonder if the huge %age is due to a lot of low-quality applications mixed
> in with the serious ones.

I doubt it. Most clinical trial data in the U.S. is similarly fraudulent, but
there isn't really any concept of high-quality vs low-quality applications
like you might see with YC or whatever.

Instead, pharma companies tend to rely on the same set of 20 or 30 tricks to
take chemical compounds that might be no better than placebos and make them
look like wonder drugs. I've been keeping a tally of these tricks on my
website:

[http://www.alexkrupp.com/Citevault.html#pharmaceuticals](http://www.alexkrupp.com/Citevault.html#pharmaceuticals)

------
ksk
Title should make it clear that this article is about currently pending
applications, not about all clinical trials.

~~~
sctb
Thanks, we've updated the title.

------
msie
Ting bu dung...

~~~
jza00425
lol, after checking comments and stories of this guy, what a deep love to
China

