current home owners who sell, end up buying another home most of the time. there isn't any significantly less inventory. housing shortage comes from more.people wanting to buy homes, and new inventory lagging the increase in want to be buyers
For people with a ton of money, sure it doesn't matter. But most people buy nicer and newer houses throughout life, freeing up smaller and older houses for the next generation. When that quits happening, everything gets kinda logjammed.