Those votes are constrained though. Many more people would live in urban areas if the housing there was more supportive. Tons of my friends who lived in NYC moved out when they had kids only because they couldn't afford a 2 or 3-bedroom apartment. There's also a lack of amenities in urban areas like playgrounds. One of my friends moved from a suburb to Berlin, and loved it because apparently Berlin has fantastic playgrounds, while still being a nice urban area.
And even for people who want to live in suburban or rural areas, enabling more people to live in urban areas is beneficial, because it means shorter commutes as the infrastructure is less overburdened, and less land is occupied.
 These days, the sprawl of suburban Texas seems a lot more alive and vibrant to me—you can walk into an Applebee’s and see tons of diverse families with children in tow—than the walkable, well-manicured, childless streets of downtown DC.
It’s got a strange mix of inner city poverty and people moving to the suburbs to be able to afford rent. The horrifically designed street layouts likely play a major role in this. IMO, Fix those streets and more of the city would be viable, resulting in both gentrification and more affordable middle class rents.
PS: Personally, I am not willing to give up the vote to live there, but that’s another story.