Hacker Newsnew | comments | show | ask | jobs | submit login

An article claiming that only 11% of findings could be reproduced doesn't cite its sources, thus rendering the findings of the article unreproducible. Nice work!



Nature wrote an editorial justifying the situation, you can read it here http://www.nature.com/nature/journal/v485/n7396/full/485041e...

-----


> "those authors required them to sign an agreement that they would not disclose their findings about specific papers"

I don't know why this doesn't cause a huge stir in the scientific community. Seems like everyone is fine with people sweeping their negative data under the rug. Shameful is a very mild word for that.

-----


How were those authors able to enforce this request? To be a proper paper it should not be necessary to contact the author in order to reproduce it.

If you had to ask the author for more details then their paper was incomplete - perhaps that's why it could not be reproduced.

-----


If you want to maximize your chances of reproducing the results, you will use identical apparatus and methods to what the original study used. That is often not possible to do just from reading the paper.

-----


If the identical apparatus is essential to reproducing the result, then the description of that apparatus is part of the experiment. Omitting that means it's not a proper paper.

-----


If only it were so. Often important parts of the protocol exist only in a hand written entries in a lab notebook. Because very few people are reproducing experiments and it is lots of extra work and jornals have limited space all the required details are rarely included. This is a big part of the problem. The typical work around is to contact the original reaseach to fill in the missing details.

-----


Well then this article would be testing how well the papers were written, not whether or not it was possible to reproduce the results.

-----


Agreed, but this study was specifically designed to highlight the problem not solve it. And the results would be useless, or non-existent, if they restricted themselves to scientists that would co-operate fully with the process and the possible "shaming" afterward.

-----


I think it also highlights the underlying mentality that prestige matters more than integrity and "if everyone does it then it's fine".

If tax-funded scientists can't uphold high standards for themselves, the public is entitled to demand them to do so.

-----


You might be being a little harsh. There is a huge difference between "hey, I'm having a problem reproducing your results, want to help me either reproduce or disprove your results?", which legitimate scientists will and do participate in, and "I'm doing a study on unreproducible results and want to publish your paper and name in the list afterwards".

Notice they say they got help without the shaming part so the scientists were willing to participate, just not get listed.

I personally know scientists who have helped out with disproving their own findings, it's not uncommon.

-----


Exactly. This should be the top comment. Are we just supposed to assume their work was true based on trust? Perhaps this 'study' was actually just a fleecing meant to show that we take too many studies to be true based on simple trust.

-----


It should not be the top comment. The original article is an explanation of WHY this was done. If a comment is posted explaining why the study was a bad idea or should not have been published despite this explanation THAT should be the top comment.

-----




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact

Search: