
Training Recurrent Neural Networks [pdf] - e19293001
https://www.cs.utoronto.ca/%7Eilya/pubs/ilya_sutskever_phd_thesis.pdf
======
e19293001
I found this from neural-networks course in coursera[0]. The author of this
paper had been discussed as an example of what recurrent neural nets can now
do.

Here's the description from the slide:

    
    
         •  Ilya Sutskever (2011) trained a special type of recurrent
          neural net to predict the next character in a sequence.
    
         •  After training for a long time on a string of half a billion
          characters from English Wikipedia, he got it to generate new
          text.
    
               – It generates by predicting the probability distribution
                for the next character and then sampling a character from
                this distribution.
    
               – The next slide shows an example of the kind of text it
                generates.
    
         Notice how much it knows!
         
         Some text generated one character at a time by Ilya Sutskever’s
         recurrent neural network:
         
         In 1974 Northern Denver had been overshadowed by CNL, and several
         Irish intelligence agencies in the Mediterranean region. However,
         on the Victoria, Kings Hebrew stated that Charles decided to
         escape during an alliance. The mansion house was completed in
         1882, the second in its bridge are omitted, while closing is the
         proton reticulum composed below it aims, such that it is the
         blurring of appearing on any well-paid type of box printer.
    

[0] - [https://www.coursera.org/learn/neural-
networks/](https://www.coursera.org/learn/neural-networks/)

~~~
florianletsch
You probably know this already, but if you are just looking for an illustrated
demonstration of what RNNs are capable of in the text domain, probably the
best brief article is the post by Andrej Karpathy about [The Unreasonable
Effectiveness of Recurrent Neural
Networks]([http://karpathy.github.io/2015/05/21/rnn-
effectiveness/](http://karpathy.github.io/2015/05/21/rnn-effectiveness/)).

------
joe_the_user
Also needs 2013 designation,

Especially important as neural net knowledge seems to be evolving quickly.

And perhaps someone can explain why paper matters relative to the plethora of
papers and approaches "out there".

------
the8472
> The certificate is only valid for the following names: www.cs.toronto.edu,
> cs.toronto.edu

~~~
walty
You can just change the URL to
[https://www.cs.toronto.edu/~ilya/pubs/ilya_sutskever_phd_the...](https://www.cs.toronto.edu/~ilya/pubs/ilya_sutskever_phd_thesis.pdf)
to get valid https.

~~~
Jimbo_029
Could you explain how this works?

~~~
nayuki
The University of Toronto owns both the domains toronto.edu and utoronto.ca.
Some sub-sites work on both domains, and some don't. Apparently they
misconfigured their servers so that they are using the toronto.edu certificate
on a utoronto.ca site.

------
eveningcoffee
Considering that it is from 2013 and from Hinton team then are the Restricted
Boltzmann Machines actually necessary for this?

