
Diederik Stapel’s Audacious Academic Fraud - gruseom
http://www.nytimes.com/2013/04/28/magazine/diederik-stapels-audacious-academic-fraud.html?pagewanted=all#
======
niggler
I was going to write a response, but the following comment from the site
summarizes my thoughts: "If Staple's ambition says a lot about science as a
business, then the credulity of his reviewers and colleagues speaks volumes
about science as a religion. The sacred trust people place in science and the
scientific method is sadly misplaced. The awe evoked by science popularizers
like Carl Sagan and Neil deGrasse Tyson isn't all that different than the pomp
and ceremony that surrounds the pope. And yet, it really should be the night
sky that moves us, not the flattering self-portrait that science paints of
ourselves and our intellects.

Simply put, science isn't all that different than other human endeavours
wherein people compete for prizes. Stop putting scientists on a pedestal."

~~~
freditup
Another powerful quote (this one from the article), especially taken in the
secular atmosphere of Europe:

'“People think of scientists as monks in a monastery looking out for the
truth,” [Staple] said. “People have lost faith in the church, but they haven’t
lost faith in science. My behavior shows that science is not holy.”'

------
plg
Academic science (at least what I have seen in North American research-
intensive universities, as it has changed over the past 24 years) is no longer
about scientific truth. The system is constructed (partially by design and
partially due to circumstance, i.e. such scarce resources) such that
scientific truth is no longer the goal. The goal now is self preservation and
self promotion. The truth today is that it is no longer good enough to "just"
be a good (or even great) scientist. Now you have to sparkle. Now in order to
get postdoc funding you have to show how your science is not just great but
also how it links up with industry. Now in order to get hired into a tenure
track stream you not only need to have published your PhD work in prestigious
journals but you must also show evidence to your hiring university of your
ability to attract money. Now in order to get tenure you need much more than
"just" a solid, respected track record of doing science. You need articles in
high-end journals. You need patents. You need links with industry. You need to
have been featured in the popular press. You need a track record of attracting
money to the university. The more of this you produce, the higher your salary,
the more perks you get at your home university.

Lots of people excuse the system by saying something like: well, sure, the
system has changed, boo hoo, but after you get tenure you can stop being
concerned about all the BS and return to just doing good science.

The thing is, people don't do that. People build a "brand" as a professor-
scientist, and as their profile gets higher, so does their salary, so do their
perks.

At my university, perversely, the highest-profile scientists, with the highest
salaries, (1) teach zero. I mean zero. (2) travel 70% of the time away from
home base. (3) supervise grad students by, truthfully, farming out the
supervision duties to their postdocs, who by the way, often write the grants
for the supervisor as a sort of write-of-passage (sic); (4) are trotted out by
the university anytime they need to boast about how good the uni is.

The system has been twisted so that the goal is to get money and the means to
that goal is to publish high-profile papers with simple stories in high-
profile journals, and the means to THAT is to "do science".

The goal ought to be good science, and the means to that ought to be funding
(money). It's backwards.

PS, I speak from experience, I got my PhD in 1999 and I've been tenured since
2006. I'm not a high-flyer (relative to the academic celebrities in my
faculty) but I am on the A team, so to speak.

PPS I'm not excusing scientific fraud ... but it's important to understand the
context in which it occurs.

~~~
Zklozenblarg
This doesn't translate well to Dutch academia, where Stapel was working.
Private funding is an irrelevant percentage of their income, and each school,
prestigious or not, gets the exact same amount of money per student. The man
had a tenure and kept doing it. He loved the media attention, loved appearing
in TV shows to talk about every remarkable finding he published. He was more
in the business of producing mass entertainment than knowledge, and simply
didn't want to take the time to actually conduct the experiments his papers
were supposedly based on.

To this day he continues to write more episodes to his reality TV show at a
staggering pace. He has been writing books about how ashamed he is and keeps
begging journalists to interview him about his downfall. I'm sure reading an
article about himself in the New York Times gave him a massive hardon.

~~~
plg
Yeah part of me thought, as I was reading the NYT article, why am I supporting
this guy's personality disorder by reading what amounts to an extended
biography of him?

------
maurits
I am not to worried about Stapel. He is an outlier, and just as all outliers
he is getting his share of attention in the media.

I am in general much more worried about the dubious statistics and protocols
that are deployed on real data. The torturing of the data, shoddy experiment
design, the unpublished negative results, and the somewhat sobering
realization that research is actually hard. Ioannidis formulates this quite
elegantly in "why most research findings are false" [1]

One thing that is painstakingly obvious, is that this whole affair was only
possible because people do not, and are not obliged to, share their data and
code.

[1]:
[http://www.plosmedicine.org/article/info:doi/10.1371/journal...](http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124)

~~~
andreasvc
I'm all for sharing data and code but don't you think that someone as
determined to cheat as Stapel would have simply been a bit more careful in
preparing his spreadsheet? Publishing carefully crafted fake data does not
help, only replicating the experiment would but you can't do that for every
publication. You can also demand that some kind of notary is present while
performing the actual experiment, but ultimately you need a large amount of
trust in science.

------
w1ntermute
> The experiment — and others like it — didn’t give Stapel the desired
> results, he said. He had the choice of abandoning the work or redoing the
> experiment. But he had already spent a lot of time on the research and was
> convinced his hypothesis was valid. “I said — you know what, I am going to
> create the data set,” he told me.

> Sitting at his kitchen table in Groningen, he began typing numbers into his
> laptop that would give him the outcome he wanted. He knew that the effect he
> was looking for had to be small in order to be believable; even the most
> successful psychology experiments rarely yield significant results. The math
> had to be done in reverse order: the individual attractiveness scores that
> subjects gave themselves on a 0-7 scale needed to be such that Stapel would
> get a small but significant difference in the average scores for each of the
> two conditions he was comparing. He made up individual scores like 4, 5, 3,
> 3 for subjects who were shown the attractive face. “I tried to make it
> random, which of course was very hard to do,” Stapel told me.

This sort of misconduct is shockingly common in academia, such that it is
often not even seen as misconduct.

~~~
Kylekramer
Really? The other elements (stopping once your data fits your hypothesis,
ignoring contradicting data in analysis) are fairly common, but does any
really think that making up data isn't misconduct?

~~~
a_bonobo
As someone who works in science, I've never ever ever met someone who
confessed to making up data (nor have I made up data myself).

Generating data and passing that of as measured data = fraud = loss of
contract.

~~~
spikels
Contract? Are we taking about business or science?

Now I am really worried. Is the goal to produce "tangible results" (papers,
press, professional aclaim, popular fame) or try and maybe, even probably,
fail to uncover something new but true?

Is everything corrupt?

~~~
roel_v
"Contract? Are we taking about business or science?"

Are you living under a rock? How do you think research is funded? The people
actually doing research are funded through short-term contracts with "Tenure"
being dangled in front of them as the proverbial carrot. However 1) most
people starting out on this track never get tenure; and 2) those who do,
mostly give up the nitty-gritty of research, because of a combination of
reasons.

Sometimes I find the apparent lack of understanding of how academia actually
works (as opposed to how undergrads and the general public with a university
degree but no real exposure of the behind-the-curtains of academic research
think it is, or feel it should be) just as jarring as the way things go on
universities.

~~~
spikels
Yes I must have been living under a rock while academic research (we are not
talking about corporate R&D) became completely dominated by the persuit of
funding. Yes I've heard that grant writing has become a key "research" skill.
And I know about all the academic scandals, the frequent inability to
reproduce published results, the bias against publishing negative results and
general abuse of statistics.

However as a non-academic science lover I still held on to the ideal that
science was mostly about the pursuit of knowledge. I certainly got that
impression from the few great researchers in medicine, economics, physics and
math I happened to get to know. I guess they were just exceptions.

Thanks for opening my eyes. Science is a business. And academic research is
just as corrupt as every other human activity. Why should I have ever though
anything else.

~~~
roel_v
Look man, I don't mean to rag on you. It's just - how do people think
academics make a living? They're normal people just like the rest of us, with
grocery bills and mortgages, and daydreams of having enough time to watch
their kids grow up. Money is tight everywhere, and has been before the crisis,
because so little research can be shown to have actual uses. Dreary-eyed grad
students flock to academia hoping to not have to think about such mundane
things as _gasp_ money, full of youthful idealism about Advancing Human
Knowledge(tm) only to find out that their contributions, in all likelihood,
will not matter at all, and after 10 years in their university coming to terms
with the fact that they'd rather find other ways to give meaning to their life
rather than wasting it on more of the same publish-or-perish grind.

Obviously this is exaggerating things - some research is genuinely useful, and
overall the state of the art advances in all fields, even if in very small
increments; and very few academics (at least of the ones I know, which is a
considerable number) drag themselves to their desks every day thinking about
shooting themselves in the head. But the core point that academia isn't the
romantic ideal that some have of it are undeniable, and won't be denied by
anyone living the life (because that's what it is - a lifestyle, with
advantages and drawbacks like any other).

------
aashaykumar92
Can anyone shed light on how a paper is published? I am guessing the process
requires some higher authority to review the paper to check for legitimacy. If
so, he committed fraud in 55 of his papers, not 1 or 2! So what irked me while
I was reading is, "how was fraud missed so many times?"

Someone else said that this is common in academia...if so, something HAS to be
done to stop it. That's why I would like to know the process behind a paper
being successfully published--it seems like a broken system if fraud like this
occurs remotely often.

~~~
shmageggy
Typically papers are reviewed by anonymous peers (other researchers in the
field) and the journal's editors. This is why the fraud review board chastised
the field of Psychology as a whole.

Note that it would have been nearly impossible to detect fraud in a single,
isolated paper His data were fabricated in such a way as to seem believable,
and it's not like researchers are expected to record video evidence of their
experiments actually taking place or anything like that; they just report the
results. He was ultimately revealed by those who were familiar with his work
noticing patterns that appeared over many papers. I don't think there is any
formal mechanism in place to review entire bodies of work as a whole.

~~~
lmm
Missing raw data should surely be a big red flag; maybe not video but surely
you're expected to include the individual response numbers so that other
people can check your analysis, and it sounds like he claimed to not even have
those.

~~~
jorleif
Usually these kinds of data are not part of the publication or up for review.
I don't know why exactly, but I suppose it is a combination of: 1) paper
publishing tradition, where distributing this material is too much work. 2)
too much work for reviewers to comb through raw data, the real solution would
be to require sharing of the raw data, so readers rather than the reviewers
would find mistakes, but this is often difficult because of privacy issues.
E.g. videos of the Utrecht train station study would make fraud more
difficult, but publishing the videos would probably require permission from
all participants 3) Trust in the authors' integrity. Most scientists would not
make up data, but they might make logical errors in their reasoning, or use
bad procedures. Peer review can find these errors, but if the authors lie
about their procedures, it is very hard to check. Peer review is about
internal consistency.

------
academia
This article inspired me to write an essay about my disillusionment with
academia after completing a PhD. I've posted it at
<http://mytimeinacademia.pen.io>

HN comment thread: <https://news.ycombinator.com/item?id=5624903> \- I'd be
grateful for your feedback

------
argumentum
In situations like this, people should take an empathetic stance. Sure, what
Stapel did was wrong, and he deserves no _sympathy_ , but everyone deserves
_empathy_ , even criminals.

I'm not one who "blames the system" for malicious actions of individuals, as
for anything to make sense, we must have free will. The system is not _at
fault_ , but it may be poorly constructed in such a way that individuals prone
to abuse it can easily do so.

Regardless of whether I keep my front door unlocked, if I'm robbed, the robber
deserves legal punishment. But this doesn't mean I should keep the door
unlocked.

Ultimately, Stapel has paid for his actions. He's completely discredited as a
scientist and researcher, and will never regain his reputation as an academic.
But that's not the important question.

What we must ask is why and how this got so far. How can we change the system
so people like Stapel will not be able to abuse it?

------
denzil_correa
CICLing : Computational Linguistics and Intelligent Text Processing (Natural
Language Processing) conference [0] uses an open data/software method to
eliminate such frauds. They have a _Verifiability, reproducibility, and
working description policy_ which reads as follows

 __ _Starting from 2011, CICLing implements a policy of giving preference to
papers with verifiable and reproducible results:

If the authors claim to have obtained some results, we encourage them to make
all the input data necessary to verify and reproduce the results available to
the community.

If the authors claim to advance human knowledge by introducing an algorithm,
we encourage them to make the algorithm itself -- and not only its (usually
vague and incomplete) description -- available to the public. We do not
require any demo or tool based on your paper, but instead a form of proof and
working description of the algorithm in addition to the verbal description
given in your paper. An approximation of the idea is the code submitted with
Church & Umemura's paper to be permanently hosted at CICLing servers, and
cited in the paper (see last line): you see, we don't mean anything
complicated. Obviously, you are also encouraged to show demo programs or tools
based on your method, either as part of your talk or (better) at the demo
session, and we will also be happy to host on our servers such software that
complements your paper. However, this is not required. In contrast, we do
believe that a publicly published scientific paper must be accompanied by a
minimal working description of the algorithm, open-source and available to the
community.

Also, we do not ask for impossible: if you present a large system, especially
commercially distributed or a property of your company, then we do not expect
you to provide the system. Our point is that when the software and data can be
provided, it should be provided.

We do not yet have specific rules: we hope to elaborate the rules basing on
this year's experience, so please use common sense. See the problems this
policy is to address, as well as the list of software reviewing committee and
instructions for the reviewers._ __

[0]<http://www.cicling.org/2013/>

~~~
roel_v
Right, and which is perfectly useless to prevent what Stapel did. "We
_encourage_ them to make <etc>"? Instead of spending 5 days fabricating
coherent data, you spend 6 - an additional day to come up with a plausible
reason for not having to divulge raw data. I'm not a proponent of requiring
everybody to submit their lab notes with every paper submission, in fact I
have no credible solution to the whole problem at all - but feel-good policies
like this are no more than something for editors of a journal to point at
later when fraud is found, to be able to say "yeah but we did _something_ "
(i.e., the classical politician's fallacy).

~~~
denzil_correa
> Right, and which is perfectly useless to prevent what Stapel did.

Nothing of that sort is claimed; just because you do not have a fool proof
method doesn't mean you take limited steps to stop it. The least you can do is
put hygiene checks. Most reputed conferences wouldn't even do that. Also, how
and why is this a feel-good policy? The program committee of the conference is
reputed enough to check for fraud/discrepancies. After all, the Stapel fraud
(like many others) was detected by his peers. FYI - this is not a journal
either.

~~~
jhdevos
You used the phrase 'to eliminate such frauds.' The suggestion (to me) is that
this is a fool proof method that eliminates such frauds. I would contend this
is not true; fraud is still relatively easy. The question is how much this
promotes a false sense of security.

Stapel's fraud was only detected after many years, and after his results had
been widely published. I still hear some of his conclusions quoted now and
then as if it were established truth, by people who have not yet realised that
it is one of the famous Stapel ones.

~~~
denzil_correa
Some clarifications -

\+ No method/suggestion is fool proof in the world of publications; simply
can't be

\+ Stapel's fraud was "data fraud" which is eliminated if your work on
publicly available datasets

\+ NLP community encourages work on public data and you can't get Stapel level
of recognition if you do not have your data/tools out in the open

\+ Any suggestion/method if not fool proof can be said to promote a false
sense of security; it is important to start somewhere rather than maintain
status quo

------
dmckeon
No doubt many scientists are ethical and do good work. Others may produce
reports that are either not honest or not accurate.

I wonder how we could encourage honesty and accuracy - perhaps by getting
funding agencies to increase the rewards for people who show others' work to
have been deeply inaccurate, or worse, fraudulently wrong.

Consider: what were the global costs of Rogoff & Reinhart's error? How - and
how much - will we reward Herndon, Ash, and Pollin for their diligence in
uncovering that inaccuracy?

What would the long-term effects on science be if the revelation and
correction of such inaccuracies were better rewarded, and the perpetration of
any scientific fraud involved more effective penalties?

~~~
shardling
Ah, a way to encourage _more_ academic infighting. What could go wrong! :P

I don't think an incentive like this would work very well. What you need is
more people attempting to reproduce work, which means funding the _attempt_ ,
not the result.

~~~
dmckeon
Less would _be_ wrong. Plenty could still go wrong.

Consider whistle-blowing by grad students & post-docs - should that be
rewarded as increasing accuracy, or punished as embarrassing to an
institution?

Also, I mean science to include the (often better funded) areas beyond
academia - think medicine and health care.

 _funding the attempt, not the result._

Possible outcomes: 1) Reproduces as expected (all real, or all faked?) 2)
Reproduction fails (failed attempt, or not reproducible?) 3) Exposes fakery in
previous experiments.

We should fund all of those, but reward 3) better afterward.

