
Hinton's 'Capsule Networks' Unveiled - cbennett
https://www.wired.com/story/googles-ai-wizard-unveils-a-new-twist-on-neural-networks/
======
nshm
There is also code [https://github.com/naturomics/CapsNet-
Tensorflow](https://github.com/naturomics/CapsNet-Tensorflow) and pretty cool
explanation of ideas [https://hackernoon.com/what-is-a-capsnet-or-capsule-
network-...](https://hackernoon.com/what-is-a-capsnet-or-capsule-
network-2bfbe48769cc)

~~~
cbennett
Thanks for that. the hackernoon piece in particular is really worth the read.
Two salient points/questions/comments that jumped out to me after reading
(these are really about the theory and not practice):

-The fact that you have a bunch of capsules in each layer, and each one of them is intrinsically performing a non-linear filter function (a particular squashing function shown as an image in that blog), seems like both a great asset (it looks like a 'meta-network') and also a potential problem. If you need to tune weights within these caps by the derivative of such a composite function, it doesn't seem straightforward.

-The 'routing by agreement' feature is interesting, but I dont quite get why it is superior to max pooling. If the feature is simply that it punishes weak links rather than selecting only the strongest, one interesting analogy is that it seems a bit Hebbian , and related to a concept in un-supervised learning called STDP (spike timing dependent plasticity).

------
cbennett
To save you clicks, the two pre-prints are at :
[https://arxiv.org/pdf/1710.09829.pdf](https://arxiv.org/pdf/1710.09829.pdf)
&&
[https://openreview.net/pdf?id=HJWLfGWRb](https://openreview.net/pdf?id=HJWLfGWRb)

------
Aron
Is this the only reference to this on HN? I'd be surprised but I didn't see
it. Hinton is a powerhouse and he's been wanting to make progress on this for
years.

------
ttul
Can someone ELI-not-a-genius-at-neural-networks?

