As for the axioms in use, I think the big reasons they were chosen is: They lead to results we already wanted/proved to be true.
Another thing to keep in mind, not everyone works with the same sets of axioms. Which, as someone with a formalist view on mathematics, I find interesting. For example, not everyone studying logic assumes the principle of the excluded middle. One of the consequences of this is that you can no longer do proofs by contradiction.
The axiom of choice is another example of this where two groups of mathematicians accept it or not. I'm a formalist, so I don't have issues with this (as long as both sets of axioms are interesting and "intuitive"), other philosophies of maths might.