

Ask HN: What's the big deal about Deep Learning? - thewarrior


======
PaulHoule
It's a big deal because it works.

Back in the 1960's people only knew how to train one-layer neural networks. By
the late 1970's and 1980's people were training "multi-layer perceptrons" but
it wasn't practical to train networks with a lot of layers.

In the 1990's and early 2000's Neural Networks were out of favor for academic
work, not so much that they were bad but for almost any learning problem there
was some other kind of algorithm that did better. Neural Networks were still
used a lot in commercial applications that weren't so talked about since they
thrive given the kind of intensive attention that they sometimes get in such
environments.

Geoff Hinton then figured out a way to quickly train deep networks to learn
the statistical character of the input data. This works as an unsupervised
process. It's kindof like the first layer learns very simple features that
recur in the data and then the second layer learns relationships between theem
and so forth.

After you've done that, you can teach the network to do some specific task
using a supervised training algorithm such as back propagation. Because the
network is close to where it needs to be, you can now apply back propagation
to the whole network with excellent results.

Deep learning is such an advance that neural networks are now often the
leading algorithm for many learning problems.

------
agibsonccc
Simple: Data Scientists spend a great deal of time doing feature engineering.

That is, finding observations about their data that are indicative of a
prediction they are trying to make or a piece of data (say: a user or an
image, perhaps a document) they are trying to label.

Deep Learning handles this for us. Typically machine learning takes a lot of
domain knowledge. Deep Learning is capable of automatically finding
correlations in the data and allowing us to focus on solving problems rather
than worrying about what good indicators of data are.

Re: quick pretraining. Try again: deep learning pretraining takes forever to
train due to how many passes through the dataset you need to make in order for
it to make good approximations of the data.

That being said: deep learning is very receptive to map reduce type algorithms
and online learning (streaming of data in mini batches)

That's why bigger companies tend to use it at scale.

That being said: it's astonishingly accurate despite being hard to train.

I will be more than glad to answer specific questions if you'd like.

I'll give you a few use cases:

Google uses it in their voice recognition on android.

IBM uses it in their Watson technology.

Google uses it in their Google+ image search.

It's being used for sentiment analysis at stanford.

Source:

I'm the author of a newer deep learning framework called deeplearning4j [1].

My goal with this is to make it easier to train these neural networks by just
say: feeding it images and all you need to do is specify the labels: or text.

I'm also a collaborator/committer on another effort called metronome which is
deeplearning on hadoop [2].

If you'd like an example of what deep learning is capable of, please take a
look at [3]. It's a very different way of doing machine learning.

[1] [http://deeplearning4j.org](http://deeplearning4j.org)

[2]
[https://github.com/jpatanooga/Metronome](https://github.com/jpatanooga/Metronome)

[3]
[https://jpatanooga.github.io/Metronome/rbm20140306.html](https://jpatanooga.github.io/Metronome/rbm20140306.html)

------
sytelus
Current "machine learning" algorithms such as decision trees, SVM, regression
etc are in a way cheating. In all these algorithms you need to design features
and they contain much of the smarts that you see from "machine learning"
algorithms. In other words, an impressive machine learned model is typically a
result of an impressive features gymnastic manually programmed by humans after
lot of sweat and blood.

A true machine learning algorithms shouldn't require humans to encode their
smarts and knowledge. If these algorithms existed then they would just look at
stream of symbols or sensory inputs that occurs in the nature and extract
patterns from it. Deep learning gets closer to this by eliminating need for
developing features (to some extent so far). I think it is first of many steps
towards true machine learning.

------
ACow_Adonis
Are there any resources for someone wanting to get started looking into
coding/understanding deep learning algorithms?

By which i mean, i'm already a data scientist. I'm not looking for software
that runs it for me per se, i'm looking for resources that could let me
implement/research it myself.

I've seen a lot of hype. Some of the claims I've seen people make for deep
learning I know must be objectively false and sound like a lot of the
hype/marketing for other machine learning techniques, but the ideas also look
somewhat promising, so i'm really curious to get involved and at least be able
to make an educated decision...

~~~
sytelus
The [http://deeplearning.net/](http://deeplearning.net/) is a go to website
for beginners. You would find everything from datasets to experiment to
tutorials. This subject requires a lot of match and ML background though and
would need sometime to catch up to the latest research. Coding part will
involve GPU code if you want to go truely in with large data sets. If you want
to just apply the algorithms to specific problem without diving deep in to its
theory there are few kits available in Python and C++. There a tech talks and
other videos on tutorial page that might be good starting point.

