Very few of us are selecting our mates based on their prowess in the gym. Usually its social convenience alone and proximity, these aren't traits that drive us into some more efficient species. Plus consider the fecundity of people. Do fit, and intelligent people have more kids? Studies say these people typically have fewer kids on average, that means the resulting next generations offspring is underrepresented by those from intelligent and fit parents and overrepresented by those types of parents that tend to have more kids on average.