
Principal Component Projection Without Principal Component Analysis - beefman
http://arxiv.org/abs/1602.06872
======
__mbm__
I don't really see a compelling case for this method. The empirical
convergence rates are clearly polynomial, while subspace iteration (e.g.,
Krylov) are "in theory" exponential and in practice pretty good. There is a
bit of hand-waving about why Kyrlov methods are not as good (the theory is
supposedly not robust to noise), but the techniques for robustifying Krylov-
type subspace iterations, such as restarts, are pretty mature.

More than that, the method actually appears to be an iterative "prox" method.
These things are very well studied in the convex analysis literature. I
wouldn't be surprised if this already appears as a special case of an
algorithm in the literature somewhere.

------
privong
A bit of a meta question, but what is the current thinking regarding the use
of PCA in understanding data(sets)? My recollection is that PCA was somewhat
en vogue in astronomy 8–10 years ago. I saw it applied to various mid-infrared
data, but it seemed difficult to actually translate the principal components
into useful physical knoweldge about the datasets or the astronomical objects.
Since then, I rarely see astronomy papers with PCA analysis, and even then,
the PCA analysis doesn't seem to contribute much to the physical understanding
of the objects being studied.

Is this just a case of PCA being ill-suited to the analysis of these datasets
in astronomy? Or is it a more general problem that PCA can reduce datasets to
arbitrary component vectors but those vectors may not contain easily-
quanitifiable physical information (but might contain predictive power, if an
understanding of the underlying physical system is not the goal)?

~~~
thang
PCA is useful as a pre-processing step to reduce the size of a data set to be
used as the input to a machine learning algorithm.

~~~
eggie5
But then do you have to PCA all your input test data in the future too?

~~~
huac
the output of a PCA process is a system of linear combinations of the data. so
yes, you'd take the principal components of your future inputs, but the
process would be super fast (it's a linear transformation with known
parameters) since you don't need to rerun anything.

you can also do online PCA (updating the model as you get new data) but i'm
not sure about runtime/computation requirements.

~~~
eggie5
I think the paper disagrees w/ you: "Computing principal components can be an
expensive task..."

~~~
hiddencost
The result of PCA is a linear projection matrix. Applying that projection is
fast.

You learn the projection from an initial dataset. That is "slow". You apply
the projection to new data. That is "fast".

~~~
eggie5
thanks for explaining

------
eggie5
Some background on classic PCA I just wrote
up[http://www.eggie5.com/69-dimensionality-reduction-using-
pca](http://www.eggie5.com/69-dimensionality-reduction-using-pca) interesting
read in light of this...

------
JupiterMoon
NIPALs has been available for decades to do this. I don't see it referenced in
this manuscript.

