
A Neural Network in 11 Lines of Python - adamnemecek
http://iamtrask.github.io/2015/07/12/basic-python-network/?
======
danso
At the risk of being pedantic...if you're not familiar with data/math work in
Python, the `np` word refers to "numpy", which is an extension to Python that
includes array and matrix math...so the OP needs 12 lines, the first being:

    
    
           import numpy as np

~~~
ksenzee
No, definitely not pedantic. In fact, I have only just now realized that
"numpy" means "NumPy," instead of being a nonsense word that rhymes with
"lumpy." Sigh.

~~~
Namrog84
I will still always call it nump-pee, and never num-pie

Lumpy Numpy FTW

~~~
sciguy77
Me too! It drives professors nuts, but its just so much more fun to say.

------
elyase
Just for comparison this is almost the same using a library [1]:

    
    
      import numpy as np
      X = np.array([ [0,0,1],[0,1,1],[1,0,1],[1,1,1]])
      y = np.array([[0,0,1,1]]).T
    
      from keras.models import Sequential
      from keras.layers.core import Dense
    
      model = Sequential([Dense(3, 1, init='uniform', activation='sigmoid')])
      model.compile(loss='mean_absolute_error', optimizer='sgd')
      model.fit(X, y, nb_epoch=10000)
      model.predict(X)
    

[1] [http://keras.io](http://keras.io)

~~~
avyfain
But this is not nearly as clear for a beginner, or someone who doesn't know
what Sequential, compile, fit and predict are doing.

~~~
nudpiedo
But this is the tested, maintained solution by experts in the topic.
Hopefully.

~~~
w23j
_" Do Not Reinvent The Wheel Unless You Plan On Learning More About Wheels"_

------
avyfain
Thanks for this. It is really well explained. However, it did feel like it
went downhill when you explained how the updating happens. At least it was
less of a walkthrough than the previous sections.

Also, lines 22 and 23 in the three layer network are not intuitive for a
beginner. Before, syn0 was (3,1) and now its (3,4), and the new syn1 is (4,1).
Your previous explanation for this initialization was much better.

But again, awesome work, just being nitpicky. Really appreciate it!

~~~
matt1
I recently spent some time learning how backpropagation works and wound up
putting together this step by step tutorial that you might find helpful:
[http://mattmazur.com/2015/03/17/a-step-by-step-
backpropagati...](http://mattmazur.com/2015/03/17/a-step-by-step-
backpropagation-example/)

~~~
mrborgen
This is brilliant. I've been looking for an explanation like this, which truly
understands the perspective of a beginner. Thanks!

------
fizixer
Also recommended, 7 part video series (link is video 1):

[https://www.youtube.com/watch?v=bxe2T-V8XRs](https://www.youtube.com/watch?v=bxe2T-V8XRs)

~~~
jbssm
Can you suggest any good video explanation about NN but with some code with
Python but using an actual NN library (like theano)?

~~~
fizixer
Honestly I'm a beginner myself. Other then google search that you can do
yourself, here's what might be helpful:

[https://triangleinequality.wordpress.com/2014/08/12/theano-a...](https://triangleinequality.wordpress.com/2014/08/12/theano-
autoencoders-and-mnist/)

------
shubhamjain
Great work! I have been reading a practical guide to neural networks [1], and
I am amazed that it is so comprehensible. I had assumed ML to be a lot more
complicated. Although, I won't classify it as an easy craft but (maybe) it
isn't so hard as it looks.

[1]:
[http://neuralnetworksanddeeplearning.com/chap1.html](http://neuralnetworksanddeeplearning.com/chap1.html)

------
bakul
Some of you may like seeing the two level neural network code translated to
kona (or k3)!

    
    
        nonlin:{:[y;x*1-x;1%1+_exp -x]}
        x:(0 0 1;0 1 1;1 0 1;1 1 1)
        y:+,0 0 1 1
        syn0:-0.5+3 1 _draw 0
        step:{L1:nonlin[x _mul syn0;0]
              L1err:y-L1
              L1delta:L1err*nonlin[L1;1]
              syn0+:(+x)_mul L1delta;L1}
        do[10000;L1:step[x;y]]
        L1
      (,0.009668795
       ,0.007862982
       ,0.9935916
       ,0.9921171)
    

k3 does this in 140ms, Kona in 2030ms on the same machine.

------
BrainInAJar
This is a really fantastic article, I've been looking for a code-based "Neural
network implementation for idiots" thing like this.

------
toxik
I wonder if it'd work better with a leaky ReLU. Somebody wanna try? It's a
_very_ simple mechanism. [1] Basically, instead of a sigmoid, let f(z·w) be
the activation due to inputs z and weight w, then a(z) = z·w if z·w > 0 else
0.01 * z·w. Sigmoid for activation have several problems, most notably they
suffer from vanishing gradient problems with bigger networks.

[1]
[https://en.wikipedia.org/wiki/Rectifier_(neural_networks)#Le...](https://en.wikipedia.org/wiki/Rectifier_\(neural_networks\)#Leaky_ReLUs)

------
arpgy
Shouldn't the derivative of sigmod(x) be sigmoid(x) _(1-sigmoid(x)) and not x_
(1-x). The code seems to implement the second.

------
compostor42
Can HN recommend any good neural net tutorials? (Preferably in Python)

~~~
vonnik
This is an excellent course no matter which language you prefer:

[https://class.coursera.org/neuralnets-2012-001/lecture](https://class.coursera.org/neuralnets-2012-001/lecture)

~~~
monstruoso
Why do you say it is an excellent course? I started it and so far I found the
professor to be uninspiring and the lectures lacking in preparation and
detail.

I much rather read a book.

------
bodokaiser
I done something similar some months ago. Maybe this helps someone though it
should have same content: [http://nbviewer.ipython.org/github/bodokaiser/data-
science/b...](http://nbviewer.ipython.org/github/bodokaiser/data-
science/blob/master/algorithms/perceptron.ipynb)

------
mjcohen
Interesting fact about the sigmoid function: If you scale it so its slope at
the origin is the same as the cumulative error function, it differs from that
function by at most 0.017.

------
robmcm
I don't get the point of these "x in n lines" posts.

Fewer lines of code doesn't always equate to simple, comprehensible code.

~~~
choudanu4
Well, I personally presumed that neural networks would require two or three
orders of magnitude more of code. When its revealed that the code is actually
much more manageable, individuals (such as myself) take the dive into the
subject, knowing that it is something we can at least get through.

------
ratsimihah
The final "Line 36" in your explanation should be "Line 39."

------
ratsimihah
YES! Thank you!

------
blazespin
This is amazing stuff. Just want more!

------
junto
Anyone fancy translating this to C#?

~~~
cryowaffle
Pretty trivial if you have a math library, I don't see anything here that is
special. Why not give it a shot yourself?

~~~
candu
To help you get started, some relevant documentation:

[http://docs.scipy.org/doc/numpy/reference/ufuncs.html](http://docs.scipy.org/doc/numpy/reference/ufuncs.html)

[http://docs.scipy.org/doc/numpy/reference/routines.random.ht...](http://docs.scipy.org/doc/numpy/reference/routines.random.html)

[http://docs.scipy.org/doc/numpy/reference/generated/numpy.nd...](http://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.T.html#numpy.ndarray.T)

;)

------
nikmobi
very nice, thank you!

