It's crucial to understand that if we let a person that build a shelter keep it for eternity then that person can build many shelters such that there will be no land left for other people to build their shelters on.
How they can make something they can trade, if they have no land on which they can make it? Where can they go, if private property exists in most of the world?