Maybe we’re too focused on the word “home” rather than “real estate”. That lot you built your vacant vacation home on could be someone else’s primary residence lot. As the saying goes, “they aren’t making any more land” so there is at least a theoretical supply constraint within city limits.
