Because that's a feature of human perception. How would you handle social life without at least some labeling of others? You brain would overload with all the unnecessary information.
Labeling is basically saving brain's RAM and CPU power.
a few million years of genetic programming and the fact that presentation and chromosome type correlate in >99.9% of cases. gender may not be a fact of the universe but it's about as close as complex cultural traits get.
