
A Basic Introduction to Neural Networks (2007) - vengefulduck
http://pages.cs.wisc.edu/~bolo/shipyard/neural/local.html
======
annnnd
Looking at headers:

    
    
        Last-Modified: Tue, 30 Apr 1996 18:53:31 GMT
    

Still, looks good, if a bit dated. For me, these two tutorials were the ones
that helped me understand NN the most:

\-
[http://neuralnetworksanddeeplearning.com/](http://neuralnetworksanddeeplearning.com/)

\- [https://mattmazur.com/2015/03/17/a-step-by-step-
backpropagat...](https://mattmazur.com/2015/03/17/a-step-by-step-
backpropagation-example/)

~~~
nwj
Just adding a +1 for
[http://neuralnetworksanddeeplearning.com/](http://neuralnetworksanddeeplearning.com/).
It is really excellent. Clear, but mathematically rigorous. Highly
recommended.

~~~
annnnd
Agreed - it gives a very solid overview of the field. Truth be told, it was a
bit _too_ mathematically rigorous for my taste, so I had trouble producing
code for backpropagation. However with the second link (backpropagation step-
by-step, with numbers so you can check progress) and then TensorFlow Jupyter
notebooks I think I cracked it. Now I just an excuse to use ANN at work... ;)

------
argonaut
This is neither basic nor an introduction, and has several inaccuracies. My
guess is the info is from the 90s era of neural networks.

Reply to below: it uses pretty complicated language and doesn't explain many
terms (e.g. activation functions, supervised learning, etc. are not
explained). Errors: the statements the article makes about the error surface
are really misleading and in some places wrong. Same for the statement about
sigmoids. Some of their definitions are also wrong today, such as "epoch." The
list goes on.

~~~
amelius
What actually is new since the 90s, besides convnets?

Is ML becoming more prevalent because of theoretical breakthroughs, or is it
because of hardware improvements, or perhaps because there is more training
data available now?

~~~
thegeomaster
What enabled the ML revolution we're witnessing now is mostly the advent of
powerful and low-cost GPGPUs and the abundance of training data, as you
mentioned. Of course, there are theoretical breakthroughs and interesting new
applications [1], but it wasn't the deciding factor. Even ConvNets aren't new.
For example, AlexNet is basically a much larger reimplementation of the
original Lenet idea [2] from 1998.

This is not to discredit any new work that's being done. I think it's awesome
to see so much progress in a field that's certainly on its way of defining an
era. I'm just pointing out that we have known about most of the fundamentals
for a long time.

[1]: [https://tryolabs.com/blog/2016/12/06/major-advancements-
deep...](https://tryolabs.com/blog/2016/12/06/major-advancements-deep-
learning-2016/)

[2]:
[http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf](http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf)

------
ge96
I have to do some research. I'm primarily commenting on this to keep track of
it.

Ultimately I want technology to be my better half. Fill in my gaps if you
will. I want my vitals linked to a server, a "CRON" job monitoring my
sleep/wake cycles. I've been doing some scraping. I'd like to develop my own
thing runs "autonomously and grows"

Gotta read, between this and machine learning. Gotta focus though. Not sure
when I'll come back to this. Thanks for posting this.

~~~
ge96
After reading that article and what I just described, it does not seem like
neural networks is what I'm after. Still it is something to use.

I wanted to write a "quick search" scrape pages related to a search (though
last time I checked, the Google API wasn't available anymore where you could
search Google back end.)

Anyway. Maybe a source like Wikipedia. Still parsing words and assigning them
values...

That part about using ANNs for finding regularities in patterns. That sounds
interesting. It seems many successes came by determining the next step in some
form of evolution whether it was a product or service. Blockbuster to Netflix,
Blackberry's to iPhones, etc... Maybe look at something like that.

At any rate, gotta read. That was quite informative. The whole thing of "not
knowing what it actually does" is pretty nuts too. Give it data and watch it
go!

------
rawnlq
Better off learning from the man himself:
[https://www.coursera.org/learn/neural-
networks](https://www.coursera.org/learn/neural-networks)

------
derekmcloughlin
For a good basic intro, this is excellent:

[https://www.amazon.com/Make-Your-Own-Neural-Network-
ebook/dp...](https://www.amazon.com/Make-Your-Own-Neural-Network-
ebook/dp/B01EER4Z4G)

------
vonnik
We wrote a decent intro to neural nets here:
[https://deeplearning4j.org/neuralnet-
overview](https://deeplearning4j.org/neuralnet-overview)

