I'm reminded of Feymann's Cargo Cult Science:
"I was shocked to hear of an experiment done at the big accelerator at the National Accelerator
Laboratory, where a person used deuterium. In order to compare his heavy hydrogen results to what might happen with light hydrogen he had to use data from someone else's experiment on light hydrogen, which was done on different apparatus. When asked why, he said it was because he couldn't get time on the program (because there's so little time and it's such expensive apparatus) to do the experiment with light hydrogen on this apparatus because there
wouldn't be any new result. And so the men in charge of programs at NAL are so anxious for new results, in order to get more money to keep the thing going for public relations purposes, they are destroying--possibly--the value of the experiments themselves, which is the whole purpose of the thing. It is often hard for the experimenters there to complete their work as their scientific integrity demands."
For anyone who hasn't read it, the whole thing is excellent
The theories that are powerful and worthwhile are the ones that are rarely or never wrong. Can't always get "never". It's a complicated world and we aren't all physicists. But we at least ought to be able to get "rarely", and if you can't, well, I guess it's not science then. That's OK. Unfortunately, not everything is amenable to science, though you can still approach it in this spirit of trying to see how you might be wrong rather than proving yourself right.
Once you start looking around with that standard, it's not hard to see how little science is really being done. Why are we publishing these dubious studies? Because for all the scientific trappings we claim, with statistics and p-values and carefully-written recordings of their putative procedure written in precisely the right way to make it sound like everything was recorded (while still leaving out an arbitrary number of relevant details), we've created a system where we are telling people to look for reasons why they are right... or we won't publish their results. Guess what kind of results we get with that?
If you start from the idea that you need to look for why you are wrong, the scientific method will fall out of that, along with any local adjustments and elaborations you may need, and every discipline, sub-discipline, and indeed at times even individual experiments need adjustments. If you start with "The Scientific Method", but you don't understand where it came from, how to use it, or what it is really telling you, you'll never get true science, just... noise.
I share your worldview. Makes intuitive sense to me. It's intellectually honest. And if I'm wrong about something and no one corrects me, I get kinda grumpy.
Maybe everyone else already knows, but I learned relatively late that it's called Popperian.
Reproducibility is one of the main principles of the scientific method (those papers shouldn't even been accepted if they don't contain enough information about how to reproduce the experiments they describe)
If an experiment is so complex that it can be compared to redoing someone else's thesis it can be either that the thesis is very simple or the experiment is so complex that it probably proves nothing.
In every field you have the established base theory, the bleeding edge you work on, and a bunch of preliminary work in-between. Checking out references is important, but this is a recurrent process (as you need to check the references of those works too), and you only have so much time.
So what really seems to happen is only a few select works in any subfield achieve large citation index, and those stand a decent chance of being verified at some point, or at least they do have enough work trying to build up on their results that a systematic inconsistency would show up.