(All of the above is completely without source or further arguments or detailed analysis available, just my thoughts.)
In order to earn a positive rate of return without simply taking wealth off others, the overall world economy has to grow. This growth appears to be slowing, and also significant concentrations of wealth are being stashed away (e.g. Chinese investers in Vancouver and other cities).
Not to mention the huge "negative growth" risks presented by climate change. There's going to be significant investment needed to reduce CO2 and/or mitigate the impacts of these, just to maintain the same level of economic output! An exogenous source of negative growth.
Basically the wealthy have to choose between negative interest rates, voluntary charity, wealth taxes, sudden confiscation, or invalidation of worth due to collapse. You can't take it with you, as the saying goes.