More than that, the method actually appears to be an iterative "prox" method. These things are very well studied in the convex analysis literature. I wouldn't be surprised if this already appears as a special case of an algorithm in the literature somewhere.
Is this just a case of PCA being ill-suited to the analysis of these datasets in astronomy? Or is it a more general problem that PCA can reduce datasets to arbitrary component vectors but those vectors may not contain easily-quanitifiable physical information (but might contain predictive power, if an understanding of the underlying physical system is not the goal)?
Regularized regression such as ridge regression can achieve the same thing without the risk of throwing away small but significant variation. So I think a current trend is to replace PCA+regression with ridge regression. This paper seems to take that trend a step further by replacing PCA itself with ridge regression even when the desired outcome is PCA, which is neat.
you can also do online PCA (updating the model as you get new data) but i'm not sure about runtime/computation requirements.
You learn the projection from an initial dataset. That is "slow". You apply the projection to new data. That is "fast".