Oh, come on, do you really think insurance companies don't break out all the different factors? They're in a cutthroat business where every tiny margin they make over their competitors is a big advantage. But if they guess wrong, they'll cut premiums too aggressively and lose money on accident claims.
Car size and gender are both factors. It's possible they're partly correlated, but actuaries can still incorporate both factors into their models.
Charging higher premiums for men is fair if you only care about accident rates. If you feel it's a form of discrimination and want to level the playing field, that has to be done via government regulation.