Hacker News new | past | comments | ask | show | jobs | submit login
An error made in 1925 led to a crisis in modern science (qz.com)
14 points by imarg on Aug 30, 2017 | hide | past | favorite | 7 comments



The problem is not the value 0.05 but researchers cheating with their data. If researchers were honest, by the definition of statistical test 5% of papers should turn out to be actually false. If that number is more like 30% it only means that some 25% of researchers were not honest, did selection, sanitized their data, or were guilty of some other form of data dredging/p-hacking. Switching p-value requirement to 0.005 will only mean that these dishonest researchers will now have to spend a bit more time fishing for data that matches their preconceived claim. Statistics will always remain only statistics and it heavily depends on whether people participating have enough discipline to not cheat. With current structure of incentives in science I suspect many will still be tempted (perhaps even subconsciously) to p-hack.


I've seen researchers trying different statistical tests until they get a "significant" result. I was a bit shocked; it's essentially unobjective and dishonest. The other one was removal of "obvious outliers" which is equally unobjective. It's part of the distribution of your data whether you like it or not. Add more replicates or do a better experiment, but don't manipulate the data and/or use inappropriate tests when the best one for your data gives you a P-value you don't like.


> The exact proposal of the authors is that findings with p-values of between .05 and .005 would now be referred to as “suggestive” evidence

> The paper presents research suggesting that it could reduce the number of false results in economics and psychology by half

> Benjamin admitted that choosing .005 was also a bit arbitrary

Not so much "suggestive" evidence then


While this topic should be well understood already by most HN readers, I upvoted this because it's a pretty clear layperson's explanation that deserves a wider audience.


I also like the suggestion made in various places that research not be publishable until it's independently replicated.


Not publishable at all may be too hard a requirement. But I approve if we make it less strict: "not replicated/replicatable" should give a paper similar bad reputatation or worse than "not peer reviewed"


Clickbait title.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: