The problem is human nature, specifically that we become emotionally attached to our ideas and our methods/tools and form cliques around them. If that weren't the case, a "crackpot" who followed their own peculiar methods and a systematic logical/positivist type could get along just fine and could even be objective about each others ideas. I also blame monotheism. A polytheist culture might have an easier time with the idea that there are multiple ways of thinking that can prove fruitful in different situations.
I've seen people who can pull this off, and it results in profoundly interesting debates. But most people can't. For most people their beliefs equal their identity and their sense of self, and other beliefs are a threat.
I think you might be overestimating the role emotions and identity play in maintaining the integrity of a system of epistemic statements. There are structural reasons beyond simply how much I like my ideas or other incentives to keep my ideas that make scientific reasoning hard. Devoid of emotion and human nature concerns, we still end up with optimization problems which feel fuzzy because they're fuzzy, not because there's anything clouding our vision.
The epicycle models that dominated cosmology's explanations of orbits comes to mind, if only because it's a good example that is obvious enough in hindsight as to be uncontroversial. Epicycles carry such strong descriptive power that it does make sense to add more epicycles if you're merely trying to explain your data. But because epicycles don't require information from gravitation, it feels like crackpottery to introduce such information even if it results in a simple model.
Another example that's more obviously structural is building set theory without any notion of self-reflective statements. Russell's paradox undid the first iteration of set theory despite that system being quite descriptive.
If you built a whole system of thought on a powerful, incorrect statement, you're fooling yourself but it's understandable.
I can't evaluate it well enough to say if it's true, but some crackpots are coming up with models of black whole phenomena without event horizons, based on a similar problem as epicycles might of had. That's all I'll say because I think I've exhausted the controversy I can put into one post. :)
I've seen people who can pull this off, and it results in profoundly interesting debates. But most people can't. For most people their beliefs equal their identity and their sense of self, and other beliefs are a threat.