On: No publication without confirmation

[RyanZAG]

That doesn't rule out that it's a blanket pass for some papers. How about if we phrase it this way:

In some cases, peer review works correctly with constructive feedback. In some cases, peer review is abused to "rough up the competition" and slow down the progress of science. In yet more cases, peer review is a blanket pass when the correct actors all align on that paper. There is no way to determine which of these cases apply to any particular paper.

Or put another way, all of these experiences can be true and correct at the same time.

[mattkrause → RyanZAG]

I'm willing to believe that the skids could be greased for a some papers, based on the trendiness of the topic or the authors' reputations. I'm also willing to believe that this is hard for people without the relevant expertise to detect.

However, I think describing peer review, generally, as "a blanket pass" is going much too far. If anything, I wish it were harder on actual methodological errors while being much, much more permissive of (openly-disclosed) ambiguities in the data or gaps in the theories. Right now, people tend to 'write around' issues in their data, lest a reviewer argue that this invalidates the entire experiment. Looking at papers from the 1980s and 1990s, it's amazing to me how much more frank the authors were.

[caseysoftware → mattkrause]

I didn't say "generally a blanket pass" and in fact, I was very specific to say that it was very selective.

But you've successfully shown how to "write around" the facts to make your point.

I'll choose to interpret your comment as performance art.. and in that case, you nailed it! Well done.

[mattkrause → caseysoftware]

You literally said "The more subtle problem is that in some circles, it isn't even that....it's a blanket pass." It's right up there!

[caseysoftware → mattkrause]

some !== general

I'm happy to argue my own points but you misrepresenting them into strawmen isn't productive.

[mattkrause → caseysoftware]

Sorry! I'm not trying to, but I'm also not sure how else to interpret them. As I said, I'm willing to believe that papers occasionally slip through the cracks (the arsenic life thing from 2011 could have been caught in review, for example), but the idea that a decent number of papers papers just slide through the peer review process is totally unlike the experience I've had with my own and my friends' and colleagues' papers.