I don't think it's terribly controversial to say that someone who has a history of making inappropriate comments about women would make it less likely that women would be interested in participating in an organization where that person is in leadership. Do you disagree?
They certainly _shouldn't_ be in a leadership position... (where that "inexcusable behavior" becomes a barrier to participation for various groups)