> That said, particle physics history is full of 3 sigma particle "discoveries" that disappeared with more data.
The idea is that because 3 sigma means a ~1/1000 chance of the thing being explained by random chance, 1 in 1000 experiments will produce a bogus 3 sigma result, and we do many thousands of experiments.
and also because a result always has the caveats of "if we did our experimental design and math right". A 1/1000 rate of experimental/code design errors will double the number of incorrect 3 sigma results.
The idea is that because 3 sigma means a ~1/1000 chance of the thing being explained by random chance, 1 in 1000 experiments will produce a bogus 3 sigma result, and we do many thousands of experiments.