
Deep Learning: An MIT Press book in preparation - mutor
http://www.iro.umontreal.ca/~bengioy/dlbook/
======
nfoz
Why is "deep learning" so hype nowadays? They seem like just another AI tool
that will be prone to over-fitting datasets and provide an analysis that is
difficult to mathematically characterize and understand in a reliable way.

Not here to be cynical/negative -- they might be of great value, this is not
my expertise. Can someone explain why deep learning articles are receiving
attention rather than, say, Support Vector Machines / kernel-based methods of
pattern analysis? Or other nonlinear analysis? Are they related?

~~~
shock-value
I think you are right in your sentiment that deep learning is becoming over-
hyped. I've written before that it's mostly a method of brute forcing the
problem of AI that happens to work very efficiently on modern GPUs and other
parallel processing units. That being said, for the tasks it does well on
(mostly facial and other types of simple image recognition), it does extremely
well compared to everything else out there. (And it may very well end up as a
part of a future architecture that better simulates general intelligence.)

However, I don't think it is reasonable to assume that AI tools can only be
valuable if they can be rigorously mathematically characterized. The "holy
grail" of intelligence -- that is, actual human intelligence -- certainly
can't or at least hasn't been mathematically characterized, and I don't think
anyone will ever offer any "proof" (mathematical or otherwise) that the
biological brain is inherently wired to arrive at or at least tend to converge
to correct solutions to intelligence problems. And of course no one will argue
that human intelligence isn't valuable!

~~~
halfcat
>The "holy grail" of intelligence -- that is, actual human intelligence...

When people talk about human-level AI, are they referring to specific
beneficial subsets of human intelligence, like holding a conversation or
interacting with the world around them, or does it literally include all of
human cognition? In other words, would the "holy grail" include AI that acts
extremely irrational and makes poor decisions based on temporary chemical
imbalances in the body? Does the perfect human-level AI get depressed and
commit suicide some of the time? Are we trying to replicate all of the parts,
or only the good parts?

~~~
chmike
Good point. I would compare it with human speech. Human speech is a powerful
mean to communicate and for this reason it is thus very tempting to develop
artificial systems that can communicate in the same way. But human language is
far from optimal. In the same way, I believe that human brain is powerful but
far from optimal.

The strength of human brain is its ability to adapt to fast changing context.
The solution (function) it finds is however usually far from optimal.

I also think that the future of computing is the development of systems that
can efficiently adapt their rules of actions according to execution context
evolution. The programmers of today will then become trainers.

------
bentoner
See also Michael Nielsen's book Neural Networks and Deep Learning

[http://neuralnetworksanddeeplearning.com/](http://neuralnetworksanddeeplearning.com/)

~~~
ajays
Unfortunately, the chapter(s) on Deep Learning haven't even been written yet.

------
jbarrow
This looks like it has the potential to be a great resource! Not to mention
it's coming out of one of the big name schools of deep learning (along with
UToronto and Stanford).

I imagine, though, that anyone not well versed in college mathematics may have
issues with the explanations. If you want a good introductory resource, but
either haven't covered or have forgotten some of the math in this book, I
would recommend one of two resources:

[1] MetaCademy: [http://metacademy.org](http://metacademy.org) [2] Neural
Networks and Deep Learning (In Progress):
[http://neuralnetworksanddeeplearning.com](http://neuralnetworksanddeeplearning.com)

The first will take you through all the math first through some online courses
and textbooks, and the second is a good general purpose introduction that I
recommend to anybody interested in neural nets.

------
discardorama
Sigh. The authors have taken a PDF and generated HTML out of it. I can
understand the desire to maintain copyright, but this is ridiculous.

------
fizixer
Question: Does anyone have updated information about Hinton's 'trillion
parameter network' progress ([http://deeplearning.net/2013/05/20/g-hinton-
wants-to-train-n...](http://deeplearning.net/2013/05/20/g-hinton-wants-to-
train-neural-nets-with-trillions-of-parameters/)) from almost a year and a
half ago?

It's been quite a while and even Ng has demonstrated that a billion parameter
setup could be built for $20k using commodity hardware
([https://news.ycombinator.com/item?id=5896684](https://news.ycombinator.com/item?id=5896684)).

I wonder what's happening at Google labs as of August 2014.

------
shepardrtc
Here's a talk by Geoffrey Hinton back in 2007 about Deep Learning neural
networks:

[https://www.youtube.com/watch?v=AyzOUbkUf3M](https://www.youtube.com/watch?v=AyzOUbkUf3M)

------
justin66
Pages look ok in Firefox, borderline unreadable in Chrome. I wonder what
that's about.

~~~
thomasfoster96
They've converted a PDF to be shown in HTML. Possibly using PDF.js or
something.

~~~
felixr
Looks like they used pdf2htmlEX
([https://github.com/coolwanglu/pdf2htmlEX](https://github.com/coolwanglu/pdf2htmlEX))

------
iandanforth
Hrm ... there has got to be a better format for soliciting edits. Github,
Google Doc, ???

------
vonnik
I hear O'Reilly's coming out with a book soon as well.

------
plg
this is great, I hope the authors keep a .pdf version available for dl once
its done

------
gamesbrainiac
First it was data mining, then machine learning, and now deep learning. What's
the next name? Clairvoyant learning?

~~~
chestervonwinch
The latter is a subset of the former and refers to particular
architectures/strategies for doing machine learning/data mining. Think deep as
in hierarchies not deep as in "whoa, man". Some ideas in deep learning are
new; some are old. It's not entirely a rebranding.

