I only give away the land, not the property on it. I assume in the real world we'd get some financial engineering to holding companies and peppercorn rents and so on - but as with every other tax there is a simply an inflection point where receipts drop to zero (if you tax income at 100%, people will simply stop working).
If you give away the land to someone else, but still own the building on it, the person you gave the land to would have to pay land value tax and they would pass that onto you in rent.