We're not talking about different things. We're talking about one very specific claim that you made:
> if something exists with non-zero probability in a finite set there has to be at least one example
This is a false claim. See my #1 through #4 above. I strongly suspect if you carefully re-read those Erdos papers, you'll discover that he never relies on my #2 or #3 above to prove anything; only #1 and #4.
If a property P holds over a sample space with positive probability (i.e. the measure of the subset of elements satisfying P is positive), this implies that there is at least one element of the sample space satisfying P. This is easy to prove - it follows formally from sigma-additivity of the measure.
I think you are confusing this with the notion of taking repeated samples from a probability distribution (which is what your coin example is all about).
> i.e. the measure of the subset of elements satisfying P is positive
Question: do any finite sets ever have positive (nonzero) measure? If not, then your sample space is infinite and you're in #4. Haven't you just restricted our conversation only to infinite sets via this assumption? Remember that this entire discussion started because of your claim about finite sets!
That depends on the measure, obviously. It's trivial to construct examples in which finite sets have positive measure - just take the uniform probability distribution over a finite set. Or for a more thematic example, the probability space of random graphs on n vertices. Or really any finite measure you like!
There are also myriad examples of measures over infinite spaces in which there exist finite sets with positive measure - left as an exercise to the reader.
I don't mean this to be rude, but I think you would benefit from reading an undergraduate level textbook on probability theory (or measure theory). Especially before making confident claims about people like Erdos are wielding it.
> if something exists with non-zero probability in a finite set there has to be at least one example
This is a false claim. See my #1 through #4 above. I strongly suspect if you carefully re-read those Erdos papers, you'll discover that he never relies on my #2 or #3 above to prove anything; only #1 and #4.