

Linear Methods (PCA) vs. Deep Learning (Autoencoder) - nkurz
http://danluu.com/linear-hammer/

======
StefanKarpinski
Nice post, Dan. I'm a sucker for linear methods myself, but I do think they
tend to be way less good for classification than for regression and inference
(e.g. collaborative filtering). The LSA example is a bit unfair since you're
clearly going to need the number of dimensions to be proportional to the
number categories for that to work at all well (and if you need to figure out
the number of dimensions to use, a scree plot should help). The ability of the
deep learning approach to get good separation in only two dimensions is very
impressive.

------
Irishsteve
Pretty cool to see the viz. I've never found a great improvement with LSI /
PCA on text if trying to classify information.

A lot of the work in IR seems to suggest improvements in NLP don't improve
various ranking models (TF,BM,LM).

I wonder do people just revert to lsi / plsi / pca because its fast to
compute?

