
No funding for uncomfortable results - fanf2
https://www.johndcook.com/blog/2018/12/07/no-funding-for-unwanted-news/
======
bjourne
In the 80:s, when I was a child, I was part of study about something called
DAMP - Deficits in Attention Motor control and Perception). The DAMP diagnosis
was roughly similar to ADHD, and kind of a precursor to it, invented by
psychologists in the university in the city I grew up on. Back then, this
Swedish university was very influential in the area of child psychology.
Several renowned text books are written by researches on this university and
so on. Luckily for me, I was part of the control group and not among the kids
they thought suffered from DAMP.

Anyway, Gillberg the professor who led the study had promised all
participating families total anonymity. But in the mid nineties, other
researchers started to doubt the veracity of Gillberg's results. They thought
they had found flaws in his methodology and wanted access to the raw material
to double-check his results. Gillberg refused, realizing that anonymizing the
data sufficiently so as to not leak any identifying details would be
impossible.

Eventually, the other researchers went to court and got a court order
demanding that Gillberg give up his data. At which point he destroyed all the
raw data thereby completely removing any scientific underpinnings the DAMP
diagnosis had. There is a lot more to be said about this controversy, but one
thing is clear and that is that you shouldn't promise anonymity if you can't
keep it.

~~~
user982
_> …one thing is clear and that is that you shouldn't promise anonymity if you
can't keep it._

Although it sounds like he did keep it, at significant cost.

------
new299
The statement in the original article linked is more interesting:

“In the few experiments a decade ago where publication would have been
possible, academic journals refused to do so for reasons having nothing to do
with the scientific quality of the work. Computer-science publications refused
to publish re-identification experiments unless the paper also included a
technological solution, notwithstanding assertions that publishing these
experiments would inspire technological innovation to address the real-world
problem. Health-policy publications refused to publish re-identification
experiments related to health data from fear that reaction might make data
sharing more difficult, despite assertions that because technology was
fostering unprecedented levels of data-sharing, it was timely to
scientifically re-examine data-sharing practices. Even my Weld example and
related demographic analyses, despite making significant contributions to
privacy regulations worldwide, were refused publication by more than 20
academic publications at the time.”

Personally if feels like the work was an important journalistic effort, but
its scientific merit is less clear.

~~~
dash2
That's not how I would read it. Instead, the article fell between the gaps
between two different disciplines. The computer science guys wanted a tech fix
- reasonably enough, that's kind of what computer science is about. The health
policy guys' reaction is harder to understand: they didn't want to publish it
because they feared the consequences of publicity, rather than because of
flaws in the methods. That's almost never a legitimate reason to reject a
paper.

~~~
new299
That’s what the author said, but it seems suspect to me. I’d like to read the
rejections and understand if this is just the authors interpretation.

But looking at what was done, it doesn’t seem novel. It’s just combining two
datasets, and while journalistically important, doesn’t seem to be
sufficiently novel for publication in most journals.

~~~
naasking
> doesn’t seem to be sufficiently novel for publication in most journals.

The question is why you think novelty is relevant. Replication studies aren't
novel but are critically important to real science. Novelty is not and should
not be mandatory.

~~~
new299
Maybe, but it generally is. In most scientific disciplines I think you’d find
it hard to publish simple replication of someone else’s work. You’d need to
add some novel aspect, or extend the work slightly.

For a healthcare replication study, I would guess you be hard pressed to get a
replication study published for the same demographic, and same sample size as
a previous study.

~~~
naasking
> I think you’d find it hard to publish simple replication of someone else’s
> work.

Yes, but this shouldn't be the case. The replication crisis clearly points out
the problems with this.

------
sgt101
I thought this was famous work that stopped a lot of anonymized data being
released and was part of the reason that the programme around differential
privacy kicked off?

~~~
gumby
It became famous because she was clever enough to get the _governor 's_ data,
which meant her work became interesting to the general press. The specialist
press remained uninterested for reasons discussed in the article and elsewhere
in this discussion.

------
meowfly
One thing I don't quite understand is why aren't the wealthiest citizens
lobbying for privacy legislation? I'm sure there are plenty of wealthy people
who have a lot to lose having their medical or financial information leaked.
This seems like the kind of bipartisan cross-class issue nearly everyone can
get behind.

~~~
guitarbill
Because they can pay to either insulate themselves (private clinic, fake
names, etc), or pay to have such things silenced/changed/denied would be my
guess? Especially if their businesses somehow benefit from not having privacy
legislations...

------
asciident
I see no evidence that the problem is that people were unwilling to fund
uncomfortable results, but that they didn't see the point of funding a
calculation that as the article notes, could be done on the back of the
envelope, and without any proposed solution. Basically, what would the funding
add to this statistic?

~~~
mirimir
What's significant is this:

> Over 20 journals turned down her paper on the Weld study ...

And this:

> A decade ago, funding sources refused to fund re-identification experiments
> unless there was a promise that results would likely show that no risk
> existed or that all problems could be solved by some promising new
> theoretical technology under development.

And she didn't just do a "back-of-the-envelope calculation". She re-identified
an actual public dataset, including the then governor of Massachusetts :)

~~~
new299
In order to publish in a scientific journal the work needs to be novel. Based
on this article, the work doesn’t seem novel...

I’d like to see the rejections, my guess is they mostly said “this is not
novel or unexpected”.

To make the work novel you might suggest a new approach to anonymizing data
for example...

~~~
extra88
I think it was only trivial in hindsight. Scientific publishing is rife with
articles the reveal a problem without offering a solution; to expect one seems
outright weird.

