
Visualize Algorithms based on Backpropagation - thibaut_barrere
http://neupy.com/2015/07/04/visualize_backpropagation_algorithms.html
======
hacker42
Would be interesting to see this for various test functions with various
starting parameters. Here is a large compilation of various test functions:
[http://infinity77.net/global_optimization/genindex.html](http://infinity77.net/global_optimization/genindex.html)

~~~
p1esk
The functions in your link are not gradient based, not sure how they are
applicable to neural networks with backprop.

~~~
hacker42
Don't know what fraction of them, but most of them look differentiable (as
long it is not exactly some variant of the Weierstrass function).

~~~
p1esk
True, but I think these functions are intended for non-gradient based
optimization algorithms.

------
dpratt71
Does someone want to take on the challenge of explaining the basics of this to
a lay person? For starters, I don't understand the relationship between the
scatter plot and the contour plot. I would guess that the blue (?) dots relate
to the darker areas of the contour and the red dots to the lighter areas.

~~~
chestervonwinch
There are two classes in the scatter plot -- blue and red points. These can be
separated by a line. A line has two parameters, for example, a slope and an
intercept (although, other parameterizations are possible). These line
parameters are the x and y axis. The lighter and darker areas of the heat map
indicate better and worse lines, respectively, for separating the two classes
of points.

~~~
dpratt71
That makes sense, thank you.

------
choosername
I was looking for a blogpost here that used similar, orange visualisation. The
graphs seem to be an intuitively general application in geometry or analysis
or whatever - I don't have the basics down; this should help.

------
lormayna
What is the advantage of using Neopy instead that others neural network Python
library like Pybrain?

~~~
itdxer
There are a couple of advantages:

1\. It's based on Theano, so it's fast. Also you are able to run your code on
CPU and GPU.

2\. NeuPy supports a lot of algorithms and different layer types
([http://neupy.com/docs/cheatsheet.html](http://neupy.com/docs/cheatsheet.html)),
so you can easily construct deep neural networks.

3\. In the NeuPy it's easy to read and understand network's structure. One of
the newest features is a Subnetworks
([http://neupy.com/docs/layers/basics.html#subnetworks](http://neupy.com/docs/layers/basics.html#subnetworks)).

4\. Neural Network Surgery is another new feature
([http://neupy.com/docs/surgery.html](http://neupy.com/docs/surgery.html))

