
HyperTools: A Python toolbox for gaining insights into high-dimensional data - schuetze
http://hypertools.readthedocs.io/en/latest/#
======
sanxiyn
Quote of the day from HyperTools README
[https://github.com/ContextLab/hypertools](https://github.com/ContextLab/hypertools)

"To deal with hyper-planes in a 14 dimensional space, visualize a 3D space and
say "fourteen" very loudly. Everyone does it." \-- Geoff Hinton

~~~
kgwgk
Don’t be silly. One can visualize the N-dimensional case and then it is just a
matter of setting N=14.

------
legel
Looks neat and wonderful. It’s always impossible to compress all structure
from high dimensions to 2D or 3D via something like PCA or t-SNE, and the
focus on “geometric” insight is also encouraging. Cool/appropriate name too,
nice choice of well-supported dependencies, and minimalist design are all
appealing. This could become a go-to toolkit for early stage exploration,
while it’s also pretty and smart enough to wow a coworker / investor. Hope it
continues to really develop conceptually and practically!

~~~
SubiculumCode
Do you happen to know of a good graphing package for high dimensional data in
R?

~~~
wodenokoto
As far as I can tell, HyperTools are a bunch of convenience functions for
doing something along the lines of the following dummy R code

    
    
        df %>% dim_reduction_function() %>%
           ggplot() + geom_*()
    

So in a sense, I think you should be looking for a dimensionality reduction
library, rather than a plotting library.

~~~
SubiculumCode
so it seems, looking at their gallery, and I'd rather just write the lines
calling pca, or whatever.

------
Myrmornis
The first example

[http://hypertools.readthedocs.io/en/latest/auto_examples/plo...](http://hypertools.readthedocs.io/en/latest/auto_examples/plot_sotus.html#sphx-
glr-auto-examples-plot-sotus-py)

seems a little disappointing -- it seems to exemplify more than solve the
problems of representing 3D data on a 2D screen. (Is it interactively
rotatable or something? I did run it locally in a notebook and it didn't
appear to be.)

~~~
Blahah
Indeed, all the examples appear to be showing datasets that would benefit from
good data visualisation and in some cases dimension reduction, but all of them
are terribly represented. I don't think this necessarily a problem with the
library, but the examples certainly do not exemplify good visualisation of
high-dimensional data.

~~~
daveguy
When you say it doesn't exemplify good visualization of high-dimensional data
do you have alternate examples in mind? I would like to be able to see
examples of good visualization of high dimensional data.

~~~
Blahah
I have many examples in mind. Here's an old article with good examples:
[https://cacm.acm.org/magazines/2010/6/92482-a-tour-
through-t...](https://cacm.acm.org/magazines/2010/6/92482-a-tour-through-the-
visualization-zoo/fulltext)

In general I'd say that for any dataset, the ideal visualisation depends on
the features and meaning of the data, but almost always some combination of
geometry, colour, shape and size can capture all the dimensions. Beyond a
small number of categories, colour, shape and size are much less visually
informative than geometry. Most often creative use of geometry on different
scales is the best first stop for high-dimensional data, for example using
faceting (like with ggplot2's facet capability:
[http://ggplot2.tidyverse.org/reference/facet_grid.html](http://ggplot2.tidyverse.org/reference/facet_grid.html)).
Simulating 3D is almost never required or optimal.

------
gaius
EdX has a good course on this (tho' taught in R)
[https://www.edx.org/course/high-dimensional-data-analysis-
ha...](https://www.edx.org/course/high-dimensional-data-analysis-harvardx-
ph525-4x-1)

------
Blahah
I can't see anything in the tutorials or examples that makes this a good
library for visualisation of high-dimensional data. It seems somewhat
equivalent to base graphics in R, which is _much_ less useful for visualising
high-dimensional data than anything based on Wilkinson's grammar of graphics,
like ggplot2 in R.

------
bagrow
Would this be useful for word vectors/embeddings?

