
Theano Implementation of Tree Recursive Neural Networks - sndean
https://github.com/ofirnachum/tree_rnn
======
vonnik
Recursive neural networks have not seen wide adoption in the deep learning
community. To my limited understanding, this is because their performance
depends on the quality of the parser you use, which makes them fragile. Has
anyone had experience using them?

~~~
Eridrus
Not really my area, but RNN variants such as LSTMs/GRUs were (are?) the state
of the art in NLP tasks.

~~~
mrdrozdov
Recursive Neural Networks are not the same as Recurrent Neural Networks
(RNNs).

~~~
FreakLegion
The repo uses RNN for _Recursive_ Neural Network and specifically calls
attention to its LSTM implementation.

~~~
mrdrozdov
Thanks for pointing this out. The same acronym is used to refer to both
versions. I like the description of Recursive Neural Nets versus Recurrent
Neural Nets in this Stack Overflow post [1].

> A recursive network is just a generalization of a recurrent network. In a
> recurrent network the weights are shared (and dimensionality remains
> constant) along the length of the sequence because how would you deal with
> position-dependent weights when you encounter a sequence at test-time of
> different length to any you saw at train-time. In a recursive network the
> weights are shared (and dimensionality remains constant) at every node for
> the same reason.

In the TreeRNN (Tree Recursive Neural Nets) Github project and associated
paper [2], there is indeed an implementation of a TreeLSTM which is a
recursive LSTM inspired by the recurrent version.

[1]:
[http://stats.stackexchange.com/a/158995/92040](http://stats.stackexchange.com/a/158995/92040)

[2]:
[http://www.aclweb.org/anthology/P15-1150](http://www.aclweb.org/anthology/P15-1150)

------
cjfont
Link to white paper:

[http://www.aclweb.org/anthology/P15-1150](http://www.aclweb.org/anthology/P15-1150)

~~~
rch
There's also a video:

[http://techtalks.tv/talks/improved-semantic-
representations-...](http://techtalks.tv/talks/improved-semantic-
representations-from-tree-structured-long-short-tem-memory-networks/61849/)

