
Artificial Neural Networks for Beginners - rdudekul
http://blogs.mathworks.com/loren/2015/08/04/artificial-neural-networks-for-beginners/
======
ching_wow_ka
If you're trying to learn about deep learning, I highly suggest using
Python(Theano) or Lua(Torch). They're free and used by the experts in the
field for research.

Even if you don't want to use the frameworks, you'll still have access to fast
linear algebra routines.

~~~
raphaelj
Could someone can recommend me a book about deep learning and/or machine
learning for this kind of open-source library ? I do not have any background
in ML nor DL.

~~~
wodenokoto
Then you might actually want to start in Matlab / Octave with Mchael Ng's
coursera course on ML.

~~~
criddell
I think you are talking about Andrew Ng's course.

I completed it and can't recommend it more highly. It is a really excellent,
dense course and Ng is a very good teacher.

[https://www.coursera.org/learn/machine-
learning](https://www.coursera.org/learn/machine-learning)

~~~
artmageddon
I've completed the course as well - have you used any of the knowledge from it
on anything in particular after you completed the course?

~~~
zardo
I'm taking the Coursera course right now. The course page at Stanford has a
lot of student projects. The breadth of applications is pretty huge,
definitely worth a check if you're looking for an idea.

[http://cs229.stanford.edu](http://cs229.stanford.edu)

------
akshayB
Are there any good Neural Network frameworks written in Ruby? The ones I have
used (ruby-fann and AI4r) dramatically slow down when you use them on large
amount of data.

------
an4rchy
Nice article. If anyone is interested in understanding the theory and also dig
deeper, the machine learning course on Coursera is a great place to start as
well.

------
omegote
Matlab? Thanks, but no thanks.

~~~
fphilipe
At times my mechanical engineering courses at university felt like MATLAB
tutorials. You had to use it, no way around it. Good luck once you're out of
university and want to start your own thing, you won't be able to afford it.
The computer science courses in contrast preferred open source tools over
proprietary ones.

~~~
tacos
The guy cutting my lawn spent more on his tools than MATLAB costs.

~~~
MadManE
I also spent more on video games in the last year than it costs. What's your
point? That it's so cheap he should just buy it, even if there are better,
cheaper tools out there? That seems like a waste to me.

~~~
tacos
I was refuting this silly argument: "Good luck once you're out of university
and want to start your own thing, you won't be able to afford it."

Yup, startups have costs. Go figure. And sometimes you get what you pay for.

I wish I'd learned Matlab sooner. I still love the Python ecosystem, but
Matlab's replaced a LOT of dicking around in Python for me. It does completely
different things, and certain things are trivial or impossible in each place.
Worth learning both.

------
LesZedCB
Cached version since the database seems to be having issues

[http://webcache.googleusercontent.com/search?q=cache:UhEgP6_...](http://webcache.googleusercontent.com/search?q=cache:UhEgP6_8fB4J:blogs.mathworks.com/loren/2015/08/04/artificial-
neural-networks-for-beginners/+&cd=1&hl=en&ct=clnk&gl=us)

------
moron4hire
ANNs are great for the right application. But I'm starting to fear "Deep
Learning" is the new "Big Data" buzzword.

I believe ANNs are Turing Complete, meaning they should be able to compute
anything (EDIT: + "that is computable by any other Turing Machine"). The
questions are, can a training regimen be created to create the right ANN to
solve "any" problem, and if so, is it an efficient means to solve that
problem?

For example, it's fairly trivial to build an ANN to spit out the right results
for a given polynomial function, i.e. "f(x, y, z) = ax + by + cz". Knowing the
polynomial function ahead of time, you just generate a ton of input/output
sets and feed them into the training of the ANN, and then from there on the
ANN will spit them back out.

The problem with that is, you didn't learn anything new. You didn't learn how
to solve a new problem. It's somewhat useful for teaching people how to
program ANNs, but I personally think it's garbage for teaching how to
_understand_ ANNs.

ANNs make more sense when we already have the training data, but we don't know
the underlying function that maps input to said outputs. In the trivial case
of the polynomial function, if someone were to hand us the training set, we
could use an ANN to figure out what the polynomial must be.

Except--for this particular example of a polynomial function--this isn't very
efficient. For a polynomial of N terms, you only need N+1 sets of IO to
trivially use algebra to determine the function. You can use any of the
readily available linear algebra libraries to do such a thing. In fact, I
wrote a project for a client that does just that: it uses a basic matrix
library to crunch a set of GPS data to create a quadratic formula estimation
of curves in roads, so that model can them be resampled, continuously, sans
noise.

And if that function is not just a simple polynomial--if, say, it includes
sines and cosines and square roots, etc.-- then the ANN is going to have to be
large enough to include in it ad-hoc, arithmetic estimations of sine and
cosine and square roots sufficient to give the right answers. It might even
include several different estimating functions just for sine just because our
mystery function requires more than one sine operation. It might even have
corner cases where it gets the answer wrong, because you didn't have a
sufficiently large data set for it to "figure out" things like the fact that
sin(x) is approximately x for small values of x. _If_ one knew the right
formula (and yes, that's a big if), it'd be significantly more efficient to
write a program that computed the values correctly.

All of this is not to poo-poo on ANNs. ANNs are great tools for when we don't
know the function and when the function is sufficiently non-trivial to
discover. The polynomial example is like trying to kill a fly on the wall with
a swarm of nanonmachines designed to evolve and learn how to construct a
flyswatter (which is part of the reason I dislike it as a learning tool). But
write traditional code to do Optical Character Recognition, I dare you. ANNs
are just highly specialized. Think of setting up your ANN like defining the
full width and depth of the space of all possible programs that you'd like to
search for the program that solves your problem. You then use feedback to
"walk" across that space until you find something that looks like your desired
program. We're entering an era where we have the memory and distributed
processing capabilities to crank out some rather large ANNs. For some
problems, we end up training a computer to write programs for us that we could
have written on our own. This can impact the number of requests you can handle
in a given amount of time.

Of course, that is not necessarily bad, either. "Throwing money at the
problem" is not the wrong solution when you have a lot more money than time.
Technology is supposed to serve us, not the other way around. Why spend a week
discovering a formula to map your data when you can train an ANN in a few
hours? And perhaps you don't have very high requirements for request handling.
Maybe you only need to process one image a minute on your particular system.
Have at it.

But you really, really need to know that is the case before you jump on the
ANN bandwagon. You have to know what you want out of the ANN. If you don't
have that ability to look at a set of inputs and express a desired set of
outputs, then ANN isn't magic pixie dust that will solve that for you. If you
have experts in your particular field telling you that your particular problem
cannot be easily modeled, then ANNs might be helpful for you. If you are new
to your field and you think "let's try an ANN", you're probably going to have
a bad time. If you end up with an ANN that is estimating a _relatively_
trivial program, and you're trying to provide a SaaS offering that is meant to
scale to thousands or millions of concurrent users, the ANN approach could
seriously harm your ability to scale.

~~~
strong_ai
ANNs have been proven to be universal approximators
([https://en.wikipedia.org/wiki/Universal_approximation_theore...](https://en.wikipedia.org/wiki/Universal_approximation_theorem))
which I think is what you meant when you said 'Turing Complete'.

~~~
moron4hire
There is also "Turing Computability With Neural Nets" (Seigelmann, Sontag,
1991
[http://www.sciencedirect.com/science/article/pii/08939659919...](http://www.sciencedirect.com/science/article/pii/089396599190080F))

    
    
        This paper shows the existence of a finite neural network, made up of sigmoidal 
        neurons, which simulates a universal Turing machine. It is composed of less than
        10^5 synchronously evolving processors, interconnected linearly. High-order
        connections are not required.

------
ramgorur
It takes forever to run a simple patternsearch(), fmincon() if a function gets
a bit complicated.

Their mcc compiler is even more crappier, it has so many memory leaks that
even valgrind gives up and gets freezed.

I do not want to run a MATALBBED-ANN over large datasets, no way.

MATLAB scwhag: "Do you speak MATLAB ?"

me: "No, I don't speak MATLAB, and I don't want to"

~~~
Kenji
I love matlab but specifically with neural networks, I made bad experiences.
Just generally subpar performance on convergence speed and results. It's
better to use caffe, which is the best neural network kit I know. Also, large
parts of caffe are being implemented for GPUs such that performance becomes
even better.

------
gansai
Just got 'Database Error'while trying to connect to this page. Error
establishing a database connection. So, the number of connections to this page
is limited? or what could be the issues throwing this kind of error?

~~~
meesles
Just refresh a few times, the sites probably getting hammered with HN users

------
fsqcds
I got this error for line "targetsd = dummyvar(targets);": Undefined function
'dummyvar' for input arguments of type 'double'.

------
curiousjorge
can someone give me some real world business need where I can apply RNN and
this type of knowledge? Obviously not looking for a hand out but open to
exploring problems in the enterprise or any other potential problems worth
solving which has a market.

I find that having a goal of what I want to solve or create motivates me to
learn. Whereas if I'm studying Statistics but don't have a clear goal that
motivates me (calculating sports betting odds) then it's that much harder to
master and appreciate it's applications.

I guess to me, knowing the application of something before I dive both feet
into learning it is actually the most important truth for beginners.

As a kid, did you want to make video games and then ended up learning
programming but ultimately not making video games? No 8 year old thinks I'm
going to implement lxml in javascript one day they just think of something
they like or curious about (ex. video games).

~~~
moron4hire
This is a pretty good list.

[https://en.wikipedia.org/wiki/Artificial_neural_network#Appl...](https://en.wikipedia.org/wiki/Artificial_neural_network#Applications)

