
How to Use t-SNE Effectively - allenleein
http://distill.pub/2016/misread-tsne/
======
denzil_correa
Has anyone here used t-SNE for visualization of their high dimensional data
for machine learning? Were you able to ensure that your feature space was good
via t-SNE?

~~~
colah3
t-SNE is used very widely in ML, both to analyze input data and to analyze
learned representations.

To me, the canonical example of people using t-SNE this way is visualizing
word embeddings, like this: [http://metaoptimize.s3.amazonaws.com/cw-
embeddings-ACL2010/e...](http://metaoptimize.s3.amazonaws.com/cw-embeddings-
ACL2010/embeddings-mostcommon.EMBEDDING_SIZE=50.png)

Also, some lovely examples of using t-SNE to visualize conv net
representations:
[http://cs.stanford.edu/people/karpathy/cnnembed/](http://cs.stanford.edu/people/karpathy/cnnembed/)

I think people have found it very useful as one of the main tools for
understanding what deep models are doing (along with optimization-based
feature visualization) and to just check that your model is learning. I
haven't really heard about people using it to pick input features, but that's
probably mostly because I don't really work with anyone doing feature
engineering.

~~~
denzil_correa
> t-SNE is used very widely in ML, both to analyze input data and to analyze
> learned representations.

I was wondering if t-SNE could be used to know the "goodness" of your selected
features for the prediction task. In the sense, t-SNE could be used as an
indicator for feature selection. I was wondering if some people have
successfully used t-SNE for feature selection and in what cases.

Edit - Off Topic : Cool, homepage/blog.

------
_jamesm_
The article mentions that:

"There may not be one perplexity value that will capture distances across all
clusters—and sadly perplexity is a global parameter. Fixing this problem might
be an interesting area for future research."

There are some suggestions in the literature for fixing this. Michel
Verleysen's group suggested a "multi-scale" approach:

[https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2014...](https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2014-64.pdf)

[http://dx.doi.org/10.1016/j.neucom.2014.12.095](http://dx.doi.org/10.1016/j.neucom.2014.12.095)
(more details in this one, but behind a paywall)

Their approach is to calculate the input probabilities using multiple
perplexities and use the average. They also suggest tweaking the output
probabilities, but it uses a free parameter that isn't present in the standard
formulation of t-SNE (their suggested algorithm takes the same approach as
t-SNE, but uses a different cost function and output weighting function).

------
discardorama
Are there standalone packages that can perform t-sne, or are we limited to
using the versions in R, etc. ?

~~~
Eridrus
There are plenty; This page contains a partial list:
[https://lvdmaaten.github.io/tsne/](https://lvdmaaten.github.io/tsne/)

