Let's say that there are two races, orange and blue. Alice is orange and goes to a majority-blue church and gets mistreated because of her race. Bob is blue and goes to a majority-orange church and gets mistreated because of his race.
Alice and Bob are both upset about their experiences. They submit articles to "The American Truth," an orange-supremacist newspaper whose stated mission is to convince orange people that they'll never be happy in a society where blue people are treated as equals. Naturally, they publish Alice's article and not Bob's.
Carol is orange and goes to a different majority-blue church and things are totally fine; Dan is blue and goes to a different majority-orange church and things are also totally fine. Eve, who is orange, and Mallory, who is blue, both go to the same church which has a good mix of folks from the two races, and they love it. "The American Truth" is of course totally uninterested in hearing any of these stories.
If you read "The American Truth," you'll think that there's a problem with blue people being intolerant, and you won't be aware of problems with orange people being intolerant, nor will you be aware that, quite possibly, these are both exceptional cases and most orange people and blue people alike are quite tolerant and welcoming and they tend to get along with each other.
I like your explanation, but it seems you're arguing in favor of reading "The American Truth", since it will give you both perspectives if you're currently only reading blue-supremacist newspapers?
And you'll be fine as long as you recognize the biases on both sides and verify claims.
Not really, because the blue-supremacist newspapers won't publish any of the other perspectives - they're also going to say that the blue man cannot survive in a society with orange people. You won't hear any of the stories where things are fine.
The problem with p-hacking is you need to honestly report the negative results too, not that you need to also find "statistically significant" results reporting the opposite effect.
But yes, if you consciously make an effort to find extremist sources from all possible points of view (and in the real world, that's rarely the same as "both points of view"), and if you make a point of reading them all critically and skeptically, then that is likely to get you a more balanced perspective about things on the margins. Can you find a story about life as a black teacher in a majority-white school published by a black supremacist/separatist organization?
Let's say that there are two races, orange and blue. Alice is orange and goes to a majority-blue church and gets mistreated because of her race. Bob is blue and goes to a majority-orange church and gets mistreated because of his race.
Alice and Bob are both upset about their experiences. They submit articles to "The American Truth," an orange-supremacist newspaper whose stated mission is to convince orange people that they'll never be happy in a society where blue people are treated as equals. Naturally, they publish Alice's article and not Bob's.
Carol is orange and goes to a different majority-blue church and things are totally fine; Dan is blue and goes to a different majority-orange church and things are also totally fine. Eve, who is orange, and Mallory, who is blue, both go to the same church which has a good mix of folks from the two races, and they love it. "The American Truth" is of course totally uninterested in hearing any of these stories.
If you read "The American Truth," you'll think that there's a problem with blue people being intolerant, and you won't be aware of problems with orange people being intolerant, nor will you be aware that, quite possibly, these are both exceptional cases and most orange people and blue people alike are quite tolerant and welcoming and they tend to get along with each other.