Look past the false dichotomy - there can still be a gigantic problem even though this particular hoax didn't do a good job of exposing it.
I mean, the social science journals targeted by this hoax could be awful, I have no idea. But the hoax started with the thesis "if we submit an awful paper it will get accepted" and that thesis turned out to be false. This implies that the journals are not as bad as the hoax authors originally expected, right? They could still be bad, but the hoax doesn't establish that.
The fact that the authors later got other papers accepted, I don't think we can say much about. If an experimenter's methodology is "keep iterating until X happens", then obviously the fact that X eventually happened can't prove anything about their thesis.
There is a LOT of motivated reasoning in here. What method of experimentation are you proposing where if the first attempt at validating a hypothesis fails, you simply quit instead of tweaking the hypothesis and trying again?
The hoax proved that you do not need more than a couple weeks of training to become a published author in these journals. I do not expect that these authors could, without any training or expertise, revise-and-submit a half dozen papers to be published in physics or chemistry. That's significant, whether you like it or not.
You have things backwards. I agree with the hoaxers' motivation, and I assume that what they were trying to prove is presumably true. But their experiment didn't actually support those conclusions.
> What method of experimentation are you proposing ..
Lots of methodologies are valid, but if the whole thing is presented as science then the conclusions need to be consistent with the methodology used!
In the hoaxers' writeup, they said they submitted a bunch of absurd papers, but they all got rejected, so they gradually toned things down bit by bit until some of them got accepted. That's fine, insofar as it goes, but you cannot then turn around and claim to have proved that the journals will will publish absurd papers, which is what the authors did. You can't call a result significant if your methodology is to gradually lower the bar for "significant" until you get the result you wanted!
What the hoax actually proved was that the authors could publish a paper that they considered absurd, but which the journal considered fine. Which is neat - but isn't that already true of the regular papers these journals normally publish? Then what in the world did the hoax accomplish?
> The hoax proved that you do not need more than a couple weeks of training to become a published author in these journals
I don't think the authors claimed anything like that, where are you getting that? Their thesis was just that the content of the papers was absurd, not the times or credentials involved.
> You can't call a result significant if your methodology is to gradually lower the bar for "significant" until you get the result you wanted!
This didn't happen. You're the one who has post hoc decided that there is some abstract bar of "significance" that they're moving up or down. The point is to publish absurd papers in ostensibly serious journals. There is not some absurdity equivalent to p < 0.05 that they manipulated. They just published absurd papers in ostensibly serious journals. It's not like the absurdity of Dog Rape Culture would be lessened if the first papers they submitted had been the output of /dev/random.
> I don't think the authors claimed anything like that, where are you getting that? Their thesis was just that the content of the papers was absurd, not the times or credentials involved.
I'm getting it from the literal facts of what happened. They are not trained in these fields, and papers they wrote were published in these journals. Therefore, the hoax shows that you do not need to be trained in these fields to publish in these journals.
> They just published absurd papers in ostensibly serious journals.
You and I can say the papers were absurd, but an argument is only meaningful to the extent that it has persuasive power to someone who doesn't already agree with it. For anyone who disagrees that the papers were absurd, the hoax has no argument to make - that's where it falls down compared to Sokal, or SciGen-style hoaxes, etc.
I mean honestly, did you read the authors' writeup? I remember it had a bit in the summary like: "so what has this experiment proved exactly? Well, we'll leave that for you to decide". If that's a study that's proved its thesis, I'm a banana.
> Therefore, the hoax shows that you do not need to be trained in these fields to publish in these journals.
I don't know what special training an academic from one branch of the social sciences would need to publish in another, but again you're claiming the study proves something the study authors didn't even discuss. That's not a tenable position to hold, and you might reconsider why you hold it.
I mean, the social science journals targeted by this hoax could be awful, I have no idea. But the hoax started with the thesis "if we submit an awful paper it will get accepted" and that thesis turned out to be false. This implies that the journals are not as bad as the hoax authors originally expected, right? They could still be bad, but the hoax doesn't establish that.
The fact that the authors later got other papers accepted, I don't think we can say much about. If an experimenter's methodology is "keep iterating until X happens", then obviously the fact that X eventually happened can't prove anything about their thesis.