
A Criminologist Accused of Cooking the Books - Hooke
https://www.chronicle.com/interactives/20190924-Criminology?key=mi0Bff1vaLHL09_no2Emgy5Y-dAeAI88Vkl3OR4ZPajd6ssrFAxNVnafDFjqu4AWZkpfVDdwM3pGT1E3SjBzbm5pVUNTc0FPRXB6UHY4UTctY0xWOC0xUVlIWQ
======
classicsnoot
I got my major credits for both History and Political Science but was a few
credits short for my core requirements so I decided to pick up a minor in
Criminal Justice during my last semester. I thought PoliSci was bad in terms
of opinions being treated as facts, alchemical postulations being regarded as
high science, and a general culture of trendy cults around particular
texts/authors being the deciding factor in who and what gets taught and
believed; CJ makes PoliSci look like Biology.

Sociology in general is far closer, metaphorically speaking, to alchemy than
chemistry. The problem is that Criminology is used to make very important
decisions about criminal law, incarceration, sentencing, community standards.
One could make a claim that PoliSci has a similar relationship with policy
making, but it doesn't hold up. Political Scientists spend more time doing
interviews and tweeting than they do running countries, or even experiments
(case in point: my idea of using North Dakota and South Dakota as a
geographical laboratory to run 10 year long, competitive governing structure
experiments was perceived as a joke). Criminologists are actively involved in
policing standards/regs, sentencing, etc.

With the stuff I've come into contact with, the allegations leveled in this
article do not seem surprising.

~~~
misterdoubt
A minor in Criminal Justice is not necessarily a useful point of reference for
the field. Most undergraduate Criminal Justice degrees are geared more
pragmatically toward training police, parole officers, and such. CJ majors
(and certainly minors) can graduate without ever being in shouting distance of
actual criminological research.

That's not to say that the field is without problems, but what you're
describing is not well representative of what we're taking about when we talk
about the journal Criminology.

~~~
classicsnoot
A fair point, and something I considered before posting. There are a few
factors that lead me to believe my experience is quite representative.

1) Every single teacher I have (5) are PhDs in criminology.

2) 4 out of 5 classes use their readings/assignments exclusively
criminological research papers and peer reviewed journal articles. The fifth
is exclusively case law and comments/treatises thereof.

3) Each of the 4 classes covers a rough average of 15 papers, the lowest being
ten and the highest 20 or more.

4) The institution I am at is a leader in the field of criminology. In
addition, the CJ program is paired very tightly with the law school.

5) The institution I am at is decidedly anti-LEO in both form and content.
Though there are a number of matriculating cops and a smattering of future
LEOs, the vast majority of undergrads in the program are sociology students
aiming for lawschool, social work, or criminology.

I understand your point, and I would wager that it is generally true, but not
for my case. TBH, it is the only reason I posted.

I also shared this article with every one of my professors, a few of the guest
lecturers, and the dean of the school.

EDIT: clarity of point 02

~~~
jeegsy
I appreciate your taking the time to respond to this idea that somehow because
you were only a 'minor' in CJ, your opinion is 'not a useful point of
reference'.

~~~
misterdoubt
"Not _necessarily_ ". The following comment was a very useful qualification.

For reference, I was a philosophy major, and at no point would I have
considered myself qualified to opine on the quality of current philosophy.
Similarly, a close friend who was a physics major has now completed a postdoc
in a subfield and _still_ hesitates to make sweeping statements about the
state of "physics."

And lest anyone is left with the wrong impression, criminologists are not
running much of anything in the administration of criminal justice in the
United States.

~~~
classicsnoot
As someone who has claimed to be a part of the wider world of criminology, I
am a bit puzzled how you could postulate that criminologists are not deeply
involved in the criminal justice structure in the United States.
Criminologists have been structurally incorporated into CJ at every level of
policy making and implementation. Sociologists formulate the very rules for
conducting studies (via IRB), sociologists gather, collate, shape, and deliver
the data, sociologists write the papers, journals, and articles that use the
data, and sociologists run the bodies and organizations that advise law
enforcement, government, and universities that define values, decide policy,
and force conformity (A/B/C C/B/A order).

Probation over incarceration was a criminological theory made policy.
Decriminalization of (x) are criminologists postulating about causal
relationships. More importantly, criminologists are heavily consulted by the
entertainment industry in terms of form and content.

Regardless, I feel completely comfortable to comment on the state of abysmal
reasoning I see in my school. I did it to the PoliSci department for the same
reasons. People who cannot define/explain the mathematical and/or
computational logic underpinning their methodology have no place in relating
conclusions and analysis from said studies. The methodology for encoding of
life factors, interactions, and ethno-social details alone should be the
biggest red flag. There are so many little biases, arguments from consensus,
and cognitively dissonant perspectives that it becomes death by a thousand
errors.

I hope these "data thugs" rip sociology to shreds. It is a good thing. These
are all my opinions and observations, of course, but I truly believe that all
of the social sciences are at an alchemical state; whoever can convince the
patrons that they can cull prosperity/equality/justice/peace/wellness from the
data gets the support and favor. But it is bullshit. Rather, it isn't science.
Not yet. There are too many taboos, too much common knowledge, too much
politicking, currently. Sociology needs to be culled in a compassionately
dispassionate way by external forces, with the hope of the dross being scraped
away so the concrete value, however much or little is there, can be recognized
and built upon.

------
_edo
If you look at the author's published papers[0] just about every one involves
highly sensitive political and social topics. That means they're likely to be
quoted outside of the field where people will say things like "Look, it's
scientific it was published in a peer-reviewed journal!"

Young adults have this guy as a professor and they believe that surely their
professor knows what he's talking about.

This story is a few months old, the retraction request looks serious[1], and
I'm left thinking that either Picket has gone off the rails or this entire
field looks awful.

The paper in question has 78 citations and Stewart (the one accused of
fabricating data) has 5,712 citations according to Google Scholar.

[0] - [http://criminology.fsu.edu/research/type/eric-
stewart/](http://criminology.fsu.edu/research/type/eric-stewart/) [1] -
[https://cj.fiu.edu/student-resources/resources-for-
graduate-...](https://cj.fiu.edu/student-resources/resources-for-graduate-
students/pickett-requested-retraction.pdf)

edit: this thread of some sociologists discussing this issue is interesting -
[https://www.socjobrumors.com/topic/co-author-requests-
retrac...](https://www.socjobrumors.com/topic/co-author-requests-retraction-
from-top-crim-journal-1)

~~~
misterdoubt
The field doesn't look great. But in most corners there is serious concern,
e.g., [http://grumpy.skardhamar.no/2019/09/25/the-former-
flagship-j...](http://grumpy.skardhamar.no/2019/09/25/the-former-flagship-
journal-criminology/)

From that post: "I do not have any solutions to the systemic problems here,
but improvements should be easy. Criminology as a field has to improve in
terms of making data available with full documentation and reproducible code.
That would make errors detectable sooner."

Anyone not agreeing with this needs the boot.

~~~
_edo
I think the field has a cultural problem that's causing their data problem:
impact is valued over empiricism.

~~~
EricE
Sadly this is a predominant trend in Academia. Some would argue it's always
been there, but it does seem to be significantly more prevalent and unashamed
in it's presentation these days.

------
spangry
It's quite shocking that the editor in chief of the 'Criminology' journal
seems unconcerned about papers in his journal that rely on fabricated data. It
calls into question the veracity of all papers published in his journal. Does
anyone have any insight into criminology as an academic field?

~~~
misterdoubt
Yes. He hasn't been editor in chief for long; the Crim editorship rotates
around through top departments every few years. I wouldn't take his position
as particularly determinative.

Shawn Bushway's take is probably most representative of the field. If we're
going to be taken seriously (insert Arrested Development shot) we need to be
able to deal with this.

Criminology has thus far mostly avoided the replication crisis, so there
hasn't been much reckoning with sloppy research -- much less with a major
figure in the field just making up data. And the co-author network here
touches on several of the top programs, so there's plenty of drama.

------
jessaustin
_“I don’t even quite know what retraction is. I imagine that it could occur. I
would think there would be legalistic implications.”_

\-- David McDowall, Editor-in-chief of _Criminology_

~~~
brokenkebab
That's horrible.

Also, "McDowall is no fan of the move toward more scrutiny in the social
sciences". Pretty damning.

------
Simulacra
I think this is indicative of the publish or perish mentality that
universities have instilled among their researchers and professors. It creates
an incentive to fake data, to fake papers, anything to publish. Those who get
upset at closer inspection of their papers and their data sound more like they
have something to hide, or they are afraid that the spotlight of closer
inspection will turn on them. Then again this could speak to the larger issue
that universities are having with aversion to critique, and disagreement.

~~~
elliekelly
The author collecting the data is a pretty clear conflict of interest that I
guess I’ve never given much thought. Considering how difficult/expensive/time
consuming it is to interview people it’s kind of odd the publications don’t
require some sort of certification from whichever organization collected
and/or funded the collection of the data.

~~~
Nasrudith
I suspect it hasn't been hiven much thought because of how logistically
difficult it would be communications wise and the value of context. Take even
a simple yet still complex example as say Microsoft declares that coding and
software design should be kept absolutely separate on a personal level. You
can make a design or write of anything but not both. That would be a major
pain for no concrete gain - like trying to produce bottled water from hydrogen
and oxygen burning on the grounds it is "purer" that way - ignoring that
apparatus also introduces comparable parts per billion contaminants as a
reverse osmosis filter.

As for interviews what would another layer of gatekeeping even add that
calling one or both parties to verify that the transcript wasn't from a
satirical news site add?

------
justinjlynn
I am dumbfounded that respectable journals still don't regularly engage in
data escrow with third party adjudication precisely for this reason. A
credible question about the veracity or interpretation of the data, in terms
of objectively presented statistical heuristics, should automatically grant
confidential data access for further peer review. This could be accomplished
easily with threshold-type cryptographic tools, even amongst hierarchies of
authors. These studies are the basis of our science and are its direct
credibility. It's absurd we don't take precautions against the malefactors who
would pervert it.

~~~
dekhn
even without any of the overly complicated tech you describe, some journals
maintain statisticians on staff who evaluate everything in every paper and
then help correct errors before publication. My stats prof, Stan Glantz did
this for a major cardiology journal. He said (and I agree, having reviewed
plenty of papers myself) that statistical errors that affect the conclusion of
the paper were found in the majority of papers.

~~~
justinjlynn
> overly complicated tech

It solves a problem, just like any other engineering solution. There are
trade-offs involved like always. Obviously you understand the problem I'm
trying to solve, since you're passing judgement on my suggestion - do you have
a better one? Even given all of your verbiage, I'm afraid I couldn't devine
one.

------
Nasrudith
I haven't taken any Criminonology classes so I cannot comment on the level of
field rigor in even a subset but this resembles a stereotypical case of "non-
Mathematics Liberal arts majors who think they were done with math".

There is a sadly significant culturally innumerate population who will look at
you with scorn and anger for even daring to bring up numbers even for
something straightforward like "does a year of typical catfood or small dog
breed dogfood cost more" instead of trying to guess based on concepts like
"beef is more expensive than fish" or "crazy cat ladies are willing to pay
more". Their reaction seems more ignorance than "caught in a lie bullshit
more".

It also sounds like TV news screenshots mocked online for poll percentages
which overflow 100%. I have done research and replicated sources in high
school. They often took numbers from multiple related sources and questions
and miscombined them. "Like they had the unpaid interns who knew how to use a
computer source the data" was a trope of an explanation but it may well have
been the senior professionals who did so.

------
LocalTrust
recently saw a tweet by an academic (political) economist who proposed "you
are not a real empirical social scientist unless you have published a null
results paper." science cannot advance without them, but an individual cannot
build career on them.

------
ironSkillet
The sad truth is that the majority of social sciences likely have similar
problems surrounding data integrity. Good data is often expensive or hard to
acquire, and once you have it, you often have to apply some sort of ad hoc
interpretation in order to make it quantifiable for performing statistics.
E.g. in psychology there are entire manuals for how you "encode" interactions
between a therapist and a patient along various dimensions.

And then once you've got quantitative data, many social scientists don't
actually apply statistics correctly, either out of ignorance or willful
desperation to have some sort of positive result.

We probably need to have to have some sort of standard for what constitutes
acceptable quality data, and have a government funded repository of such data
to encourage reproducibility. But that may be way easier said than done.

I think I'll just stick to mathematics.

~~~
bjourne
If you think the problem is limited to social sciences then you are sadly
mistaken. It affects all disciplines which deal with statistical evidence,
from comp sci to medicine to criminology. Not pure math though, but that field
has other problems such as proofs being published that are incorrect and never
gets corrected.

------
hdfbdtbcdg
One of the features of academia is that those on the inside of the club know
which papers are to good to be true - but no one says anything publicly.

------
EricE
Anyone who says anything as inane as "The science is settled" deserves the
highest form of ridicule and mockery, yet as you can see in this article the
wagon circling is more about politics and power than performing good science.

The more that gets revealed about the whole academic journal scene points to
it being more a scam and house of cards credibility wise vs. sincerely
advancing human knowledge.

And we aren't talking about "journals" like these:
[https://www.nytimes.com/2018/10/04/arts/academic-journals-
ho...](https://www.nytimes.com/2018/10/04/arts/academic-journals-hoax.html)

Sigh...

------
ineedasername
This isn't just an issue about fabricating a research paper, it also
represents an enormous failure of the peer review process. 2 or 3 reviewers is
common, and yet none of them pushed back on any of the issues presented here.
And yet when others took a look the issues became apparent. It's hard not to
come away from this with the idea that the system is broken. I also don't
understand why some level of sharing anonymous data isn't a requirement for
all research. If replication is a cornerstone of scientific research, the very
first piece of replication should be obtaining the same results from the same
set of data.

------
ineedasername
I really don't understand, especially given the risk of being found out, why
someone would go through all of the trouble to fabricate research like this.
With a little extra effort it should be possible to produce legitimate
research. Heck, even if you're results are negative you can write the research
and say "Results here showed no significant difference on the question of X.
However there is a need for further research, and these results simply
represent the first step along that path" blah blah blah. It's really not that
hard!

------
Merrill
Does Wiley-Blackwell as publisher have any responsibility, or are they just a
print shop?

