If you bought it in California, say, in 1984 it was more like $100k, and would now be worth more like $900k[0]. Your outlier valuation is a terrible basis upon which to institute yet another tax.
The outlier valuation is exactly the point! That's why it's a lottery ticket. Some people get $900k, others get $3m, not because of anything they did, not because of improvements made to the property, but just because they picked the lucky patch of dirt.
As discussed in the article, LVT can be implemented in a revenue-neutral way where most homeowners would pay lower property taxes so your handwringing about "yet another tax" is unconvincing.
[0] https://www.noradarealestate.com/blog/california-housing-mar...