> “Our work is not fake news,” Filippo Menczer, a professor of informatics and computer science at Indiana University and a coauthor of the study, told BuzzFeed News. “In the moment that we found the error, we immediately contacted the editors to retract it. It was our own initiative. We were not trying to trick anyone. This is how science works.”
>> we immediately contacted the editors to retract it.
And that's how the most effective fake news works. A tiny fraction of the people who see (and spread) the original article will ever see the retraction. And ever fewer will spread it.
That’s true, but it’s an unavoidable consequence of research publication. You don’t produce new science and hold onto it for years waiting to see if someone challenges your non-published result.
Alternatively, I think fake news is artificially engineered to achieve the effect. So unless this group was deliberately producing fake conclusions I personally wouldn’t call them fake news.
I'm not even talking about research publication. I'm talking about today's mainstream press. The algorithm there is simple: issue a bombastic, click grabbing "story", let people generate traffic (and ad revenue) for a few days. Then either change the article, or delete it, or issue a milquetoast "retraction".
A team I ran at one of my past jobs was interviewed by NYTimes once. What was published was extremely editorialized to drive clicks, and as a result bore little resemblance to what people actually said. We didn't even bother asking for a retraction, but after that incident I've been extremely distrustful of basically everything I read in the media, no matter the source, unless I see direct, unedited evidence. And even then I'm distrustful if evidence appears to be taken out of context, which it is at least 90% of the time. I just wish the "journalists" would stop killing their own profession, and start behaving like adults. The only prominent voice I trust these days is Glenn Greenwald.
I don't know about that. I would say most people know that the anti-vaccine research was rescinded. And especially if you search for info you are more likely to read about the fact that it was wrong than not. If the impact is large enough there will be a maybe-not-equal, but opposite reaction.
They weren't that careful. It needs to be standard to require "independent" (as far as that is possible) replication of all studies at least once before they are taken seriously. If that means publishing half the current number of original research papers, that is fine.
This simple practice would take care of all sorts of issues (bugs, fraud, bias, random fluctuations).
Well, the way research is usually run they look for "statistical significance" and try to have "power" of ~80% to find it.[1] The power is controlled by the sample size, which is a big determinant of how much is spent on the study so it is chosen to be the lowest acceptable value.
Anyway, my point is if there is an 80% chance of getting a result and you run two trials with those same odds, then only about 64% of the studies will replicate with the correct result (in R):
p = 0.8
res = t(replicate(1e5, sample(0:1, 2, prob = c(1 - p, p), replace = T)))
# 0 = Replicated wrong result
# 1 = Non-replicated result
# 2 = Replicated correct result
> table(rowSums(res))
0 1 2
3944 32135 63921
So, the standard practice is designed to generate a large percentage of conflicting results.
[1] 80% is the target, but is probably usually optimistic
How do you independently replicate research that hasn't been published yet?
And when it's finally published, do you publish both papers and credit both teams? If not what's the incentive for the replicators to consider a non (yet) influential work?
Both projects are funded simultaneously as part of the same grant. If it is worth doing once, it is worth replicating. Vice versa: If it isn't worth funding the replication then the project isn't worth doing.
Obviously pre-registration is required if both groups will follow the same plan.
If not enough is known to do that, then there needs to be a pilot study done to figure it out. Basically what is being published now is a bunch of pilot studies.
He never said replicate before publish. He said replicate before taking seriously.
Of course, it’s not in the hands of the author that they be taken seriously. Essentially what op is saying is the collective community needs to be more scrutinizing, a statement laced with many levels of irony.
Honestly after a group publishes it’s necessarily someone else’s responsibility to reproduce. So publish quickly, and hope it gets enough attention that a reproduction comes quickly.
Honestly this seems like more of a media issue. They take a story and run with it and don’t care if it’s right.
https://www.buzzfeednews.com/article/stephaniemlee/fake-news...