
Basics of Neural Networks with example codes and illustrations - coderjack
http://natureofcode.com/book/chapter-10-neural-networks/
======
JDDunn9
Does anyone have any examples of areas where neural networks beat out
statistical based methods, other than maybe image recognition? I can't even
think of another major area where they dominate.

\- Search engines use algorithms, not neural nets.

\- The most popular algorithm on Kaggle (data analysis competitions) is random
forests

\- Google's self-driving car uses statistical-based methods

I can't imagine commercial aircraft would use a neural net. What happened if
one crashed? They would analyze the data and ask questions like, Q: "What
happened?" A: "I don't know" Q: "Can we fix it so it doesn't happen again?" A:
"I don't know".

~~~
dave_sullivan
Glad you asked...

Definitely image recognition:
[http://www.cs.toronto.edu/~hinton/absps/imagenet.pdf](http://www.cs.toronto.edu/~hinton/absps/imagenet.pdf)

Speech recognition:
[http://www.cs.toronto.edu/~hinton/absps/RNN13.pdf](http://www.cs.toronto.edu/~hinton/absps/RNN13.pdf)

Natural language processing:
[http://www.socher.org/index.php/DeepLearningTutorial/DeepLea...](http://www.socher.org/index.php/DeepLearningTutorial/DeepLearningTutorial),
[http://aclweb.org/anthology/N/N13/N13-1090.pdf](http://aclweb.org/anthology/N/N13/N13-1090.pdf)

If you're into kaggle competitions: [http://blog.kaggle.com/2012/11/01/deep-
learning-how-i-did-it...](http://blog.kaggle.com/2012/11/01/deep-learning-how-
i-did-it-merck-1st-place-interview/)

I don't think there are going to be any further major advances in eg SVMs or
random forests (famous last words maybe...) Neural nets, on the other hand,
are just scratching the surface of what's possible. So _right now_ they are
state of the art in some historically very difficult areas. But these are
early days still.

~~~
lightcatcher
You wrote the answer I was just a little bit too lazy to write...

As to the GP: Geoff Hinton (probably the most well-known neural networks
researcher) said in his Coursera course that neural networks thrive at
problems with a lot of structure that could be encoded, while simpler models
like SVMs or Gaussian processes might be better for problems without as much
deep structure to discover.

Also, a lot of the current research with neural networks involves using neural
networks to learn better representations of data. These cleaner
representations of data (which can be thought about as a sort of semantic PCA)
often make classification far easier, which explains the great results.
Learning representations also makes transfer learning (transferring knowledge
from one domain to another) much easier/more possible.

------
nrox
brain.js library is a NN implementation in JavaScript. It's very easy to use.
[https://github.com/harthur/brain](https://github.com/harthur/brain)

Here is a test with a model of a robotic arm:
[https://assemblino.com/show/public20123372.html](https://assemblino.com/show/public20123372.html)

------
Lambdanaut
This book is everything I've ever wanted in a programming text. I'm sorry that
I don't have much of anything substantial to say except praise, but seriously,
thank you for writing this.

~~~
coderjack
Thanks for liking this post. But the actual credits must go to the author of
this text as I am just another fan of this book like you are now.

------
mekarpeles
The experience (specifically the careful choice of mediums + examples +
presentation though which the concepts are conveyed) is pretty fantastic.

~~~
coderjack
You may also want to read the chapter on fractal programming in this book. Its
pretty intuitive too.

------
NKCSS
An awesome book; I've now started reading from the beginning of the book :)

One thing I've noticed though, is that img 10 of chapter 1 is missing.

[http://natureofcode.com/book/chapter-1-vectors/imgs/chapter0...](http://natureofcode.com/book/chapter-1-vectors/imgs/chapter01/ch01_10.png)

~~~
CodeCube
You can submit a pull request ;) [https://github.com/shiffman/The-Nature-of-
Code](https://github.com/shiffman/The-Nature-of-Code)

I think that's so amazingly awesome, that it can evolve as a living document
in this way.

------
pests
In regards to the first interactive demo, it seems to be adjusting the line to
be parallel to the one drawn on the background. Was this intentional or are
the supposed to converge?

~~~
coderjack
The line in the background is the function towards which the neural net is
expected to converge.

~~~
pests
When I first posted my reply every time I watched that demo the neural net
line was converging on a line perpendicular to the line in the background.

(rereading my original comment I think I described what I was seeing
incorrectly but this new reply correctly explains what I was originally
seeing)

------
bcuccioli
I wrote a simple neural network about a year ago for doing optical character
recognition as a class project. I think looking over the code could be good
for learning, as it has a pretty simple OOP structure:
[https://github.com/bcuccioli/neural-ocr](https://github.com/bcuccioli/neural-
ocr)

------
catshirt
cool. just bought the book on Amazon! i know some small amount about neural
networks (i was able to skim the article), but the book as a whole looks
stellar.

------
gustavodemari
nice article

