I agree with what you're saying in general, but there's also problems in play in academics that are broader than just bad statistics. The article tries to convey the nature of this problem, but the "chasing a unreplicable effect" or "science sometimes takes awhile to work itself out" is just the tip of the iceberg.
This touches close to home because it's in my area of research, and for years I had many discussions with colleagues about this very same genetic effect and its problems. This SLC6A4 candidate gene research was not just a fluke of incompetence (unless by incompetence you mean much of an entire biomedical field of researchers), and it persisted wildly, with huge amounts of methodological research and money behind it.
Papers advocating for this type of research (and even more statistically problematic research) were published in Nature, with lots of methodological arguments, by established quantitative experts. This doesn't mean it's correct, just that by all superficial indications, it was solid. You had to question these authority figures, and a body of research, in good journals, the actual nature of the argument, and even then you were branded a naysayer or curmudgeon.
Even when people started questioning the effect, then you had people start advocating for more complex interactions (as intimated at in the article) that just amounted to unintentional (or intentional) data fishing and p-hacking with a theoretical cover.
When I started pointing out how problematic this all was to my colleagues, I had some of them outright explain to me that they thought it might be bunk, but it was popular, and if they found something significant, and it landed them a paper in a prestigious journal, why wouldn't they publish it? That is, you gotta publish what's popular because that's how you build an academic career.
I can't begin to explain all the shady stuff I've seen with the SLC6A4 effect being discussed in this article. Some of it was probably completely unintentional, and some of it probably amounts to conscious p-hacking and fishing.
The worst part about this, that's hard to convey, is that, yes, science probably works itself out eventually most of the time? But there's such a focus on popularity and prestige, fads, regardless of veridicality, and much less so on boring rigor and correctness, that entire careers can be made or broken on complete nonsense. The person who catapaults a completely empty finding to fad status has a career elevated permanently, to a nice named-chair-full professor position. The person who is trying to be rigorous, maybe even dispute the finding or disprove it? Much less clear what happens to them, and often it's a thankless task. That is, the fad makes a career, and even after the fad is discredited, people shrug and say "oh well, that person just happened to have a good idea that was wrong." The people who do the hard work of replicating it, disproving it? Well, that's not interesting or worthwhile to reward.
Academics is really broken, at least in many fields.
I thought it was interesting and amusing, but filed it away in my head and forgot about it.
Then, one day, I got an email something to the effect of "fax photocopies of your drivers license and credit card or we, Amazon, will shut down your Amazon account permanently. Do not send emails, only fax, to X number."
At first I naturally thought it was spam. However, I kept getting systematic emails of this sort, with a date attached to them. It seemed weird for spam to be so deliberate, and as phishing attempts go, also sort of weird. There also seemed something strange about the content of the emails that made me think it might be serious.
So I called Amazon customer support (totally independently of anything in the email), and they said "Yes, that's a real email. You need to respond to it or they will shut down your account." I was puzzled by this, because nothing unusual had happened from my end. So, after talking to someone who made it clear they knew nothing about it other than that it was real, I asked to speak to a supervisor.
This supervisor sort of chuckled and said that the email was from the security division, that customer support knows nothing about them, that they cannot access anything about why I was getting the email, and that this division only communicates via fax, including with customer support. So basically I was being threatened to have my Amazon account shut down for a reason Amazon couldn't explain, because they themselves can only communicate with the people who know via fax, and I can only communicate with them via fax.
So I figure, ok, fine. I search out an office supply store (because I can't use work fax for personal business, and don't have a fax machine). I send them photocopies of my drivers license and credit card. I make sure it's completely visible, and include information on the cards in the fax, in text.
A few days later I get an email saying "we received your fax, but the photocopy wasn't legible enough." I was like WTF??? because it could not have been more legible. Also, any info not legible (even though it strains credulity to be considered illegible) was in the fax. So I tried again.
Again I receive a similar email saying "we received another fax but again it's illegible." I called customer support again, and again they threw up their hands and said they can't communicate with that division either.
At this point I gave up because what was I going to do? Amazon's own customer service can't even communicate with this shadowy fax-only communicating security division, I know nothing about why my account is being shut down, there's no recourse or appeal, and when I try to comply, I capriciously am told it's not sufficient.
About a week later my account gets shut down.
About that time I remembered the article, which was eerily similar to my experience. So nothing had changed in those years.
I kept my materials (email printouts, including the faxes); I think I still have them, but am not sure as we moved in the interim.
The whole experience convinced me Amazon simply has zero incentive to care about customer service after an experience like that (which apparently is not the first time this has happened).
Wow! Relying on FAX sounds so ridiculous, especially coming from a company that was supposed to have designed API for every internal operation -even for some that didn't make sense, long before APIs where even a thing.
This touches close to home because it's in my area of research, and for years I had many discussions with colleagues about this very same genetic effect and its problems. This SLC6A4 candidate gene research was not just a fluke of incompetence (unless by incompetence you mean much of an entire biomedical field of researchers), and it persisted wildly, with huge amounts of methodological research and money behind it.
Papers advocating for this type of research (and even more statistically problematic research) were published in Nature, with lots of methodological arguments, by established quantitative experts. This doesn't mean it's correct, just that by all superficial indications, it was solid. You had to question these authority figures, and a body of research, in good journals, the actual nature of the argument, and even then you were branded a naysayer or curmudgeon.
Even when people started questioning the effect, then you had people start advocating for more complex interactions (as intimated at in the article) that just amounted to unintentional (or intentional) data fishing and p-hacking with a theoretical cover.
When I started pointing out how problematic this all was to my colleagues, I had some of them outright explain to me that they thought it might be bunk, but it was popular, and if they found something significant, and it landed them a paper in a prestigious journal, why wouldn't they publish it? That is, you gotta publish what's popular because that's how you build an academic career.
I can't begin to explain all the shady stuff I've seen with the SLC6A4 effect being discussed in this article. Some of it was probably completely unintentional, and some of it probably amounts to conscious p-hacking and fishing.
The worst part about this, that's hard to convey, is that, yes, science probably works itself out eventually most of the time? But there's such a focus on popularity and prestige, fads, regardless of veridicality, and much less so on boring rigor and correctness, that entire careers can be made or broken on complete nonsense. The person who catapaults a completely empty finding to fad status has a career elevated permanently, to a nice named-chair-full professor position. The person who is trying to be rigorous, maybe even dispute the finding or disprove it? Much less clear what happens to them, and often it's a thankless task. That is, the fad makes a career, and even after the fad is discredited, people shrug and say "oh well, that person just happened to have a good idea that was wrong." The people who do the hard work of replicating it, disproving it? Well, that's not interesting or worthwhile to reward.
Academics is really broken, at least in many fields.