The argument is that you shouldn't be forcing people out of their homes. The reason you don't get the argument is that you view people's property as something you should optimize for others, while the people making the argument are believers in strong property rights, especially that they want to be left alone without oppressive government actions. The other half of the argument is that your terms are vague. Not every million dollar house is huge. It could be used to price people out of normal SFH just because the land is zoned in a way that allows higher density housing.
You're literally proposing a scheme that views removing people from their homes as beneficial. Your values don't match with others. That's the disconnect.
And by the way, that increase in value over the years is because other people want to live in that area. Perhaps we should disincentivize that instead of punishing the people who made it a nice area to live in.
You're literally proposing a scheme that views removing people from their homes as beneficial. Your values don't match with others. That's the disconnect.
And by the way, that increase in value over the years is because other people want to live in that area. Perhaps we should disincentivize that instead of punishing the people who made it a nice area to live in.