Hacker News new | comments | ask | show | jobs | submit login

My first reaction to this headline was "duh." Of course we should hold off on accepting scientific claims (i.e., predictions about the natural world) that to date have been verified only by the same person making those claims!

My next reaction was, "wow, it's a sad state of affairs when a postdoctoral research fellow at Harvard Medical School feels he has to spell this out in a blog post." It implies that even at the prestigious institution in which he works, he is coming across people who treat science like religion.

Pretty much everyone in science and academia is saying we need to reproduce results. It is a sad state of affairs, but I'm not sure why having someone from Harvard say it is surprising or different than anywhere else.

The real issue is that there are anti-incentives to reproducing other people's results. All scientists want to see it, and nobody is able to actually do it, because they'll lose status, publication opportunities, and funding. It's viewed as career suicide. Unfortunately, this article doesn't suggest any solutions to that problem.

It would be best to not suggest this problem is somehow due to "people who treat science like religion". That association isn't called for, nor is it largely true or applicable here. Ahmed even said very early in the article "the majority of irreproducible research stems from a complex matrix of statistical, technical, and psychological biases that are rampant within the scientific community."

Statistical and technical biases are common human errors that affect everyone equally and don't amount to religion. Even beliefs don't amount to religion. You and I both believe electrons run through our computers, yet I haven't verified electrons exist and I've never seen one - have you? Almost everything we know is belief based on what others have told us, whether it's science or not. Even if scientific results are reproduced, unless you're doing the reproducing, you're still subject to believing the results. It's a more believable story when two independent people verify some result, but that doesn't mean that believing one story or one paper demonstrating a result is somehow akin to blind faith, ritual, and deity worship.

I don't have anything constructive to add other than sadness. This bums me out.

Culturally academia is responsible for a lot of cult-like approaches to things. It's just a humanity problem that we have to acknowledge and use science to fight against.

My reaction was the same. I would argue that the requirement is even stronger: the hypothesis must be verified by different scientists over time. One of the key roles of the scientific method is error correction for bugs in our mental machinery (i.e., cognitive biases, of which there are many ).

I look at science as an essentially evolutionary process: a single study has a very low probability of being accurate, if the results survive additional testing over time, the probability of accuracy increases.

" he is coming across people who treat science like religion."

The only difference between Scientists and people in other fields is that Scientists completely lack self-awareness.

Science is obviously systematically biased.

Have you ever been to a good school? Every word a prof says is designed to make you think he is smart. That's the only way they make careers. They 'live in the identity' of being smart.

It's laughable and hyper competitive.

Bad studies ensue.

It's obvious to anyone with a basic grasp of human behaviour.

The only thing that surprises me is how they are unwilling to admit the problem.

Of course you as a reader of said claims confirm at the very least that they've been independently reproduced, right?

(If so, this shouldn't be news.)

Unfortunately, there is no easy way to do it. Confirmation studies are not easily accepted by impactful journals/conferences, thus nearly nobody bothers to do them. Even if there is one, it can be surprisingly hard to find it.

As a point of anecdata: my wife's master thesis was a confirmation study of using LLDA for face recognition. I remember seeing it included in some book by the university press. I gave up Googling for it after 5 minutes.

There needs to be better scientific protocol. More linking through data instead of annoying cites. I think anyway.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact