
Some of the most important deep learning papers - adeshpande
https://adeshpande3.github.io/adeshpande3.github.io/The-9-Deep-Learning-Papers-You-Need-To-Know-About.html
======
fizixer
With this field advancing so fast, I guess if we could do something like this,
that would be great:

Maintain a running list of:

\- 3-5 most important papers in the last 3 months

\- 3-5 in the last 1 year (not all in the 3 month list would make into this
list)

\- 3-5 in the last 5 years.

I guess it's difficult for a small number of people to rank the papers. Maybe
a hackernews or reddit style upvote/downvote system can be used, with a list
that essentially scrapes arxiv for papers.

~~~
j2kun
> I guess it's difficult for a small number of people to rank the papers.

This is what's done every year at the AI conferences. No need for a new voting
system.

\- 3-5 most important papers in the last 3 months: The "best paper award" deep
learning papers of the most recent 1-2 AI conferences.

\- 3-5 in the last 1 year: Top cited deep learning papers from AI conferences
this year.

\- 3-5 in the last 5 years: Top cited deep learning papers from AI conferences
in the last 5 years.

Since AI conferences don't happen every day (or even every month) this would
not be a hard list for a novice to maintain.

~~~
hueving
>This is what's done every year at the AI conferences. No need for a new
voting system.

Not really. Academia rarely ranks things important to the application (aka
industry) of a field of study. An efficiency improvement that completely
changes the economics of something would likely be rejected from an academic
conference for being 'incremental'. Similarly, 'novel' work is much more
highly rewarded in these conferences than improvements to existing techniques,
experiment replications, or negative results, all of which are more important
to industry.

Bibliometrics are garbage and represent little more than a popularity contest
(and what is controversial). It's basically like if you took up votes and down
votes on reddit and just had them both increment a single score. If they were
worth anything, there wouldn't be such a thing as review papers that highlight
important developments.

~~~
j2kun
> Academia rarely ranks things important to the application (aka industry) of
> a field of study.

"Rarely" is a strong word. I think the truth is closer to, "Academia has
screwed up as a whole in a very small number of cases, but usually works just
fine, and always self-corrects eventually."

~~~
hueving
I'm not talking about screwups. I'm talking about fundamentally unaligned
objectives between academia (advancing the edges of knowledge) and industry
(making knowledge useful in an economic sense).

It's not a problem, it's just something you have to recognize so you don't
have the wrong expectation.

------
sja
Good list! I think it's important to note that this article is (intentionally)
focused on modern CNN architectures, and not "deep learning" in general.

I'd also add in the following "technique" articles: Geoff Hinton et al.'s
dropout paper[0] and Loffe and Szegedy's Batch Normalization paper[1]. I don't
think there's been enough time for the dust to settle, but I'm excited about
the possibilities Stochastic Depth[2] could offer, too.

[0]: [http://arxiv.org/abs/1207.0580](http://arxiv.org/abs/1207.0580) [1]:
[http://arxiv.org/abs/1502.03167](http://arxiv.org/abs/1502.03167) [2]:
[http://arxiv.org/abs/1603.09382](http://arxiv.org/abs/1603.09382)

------
vonnik
We've compiled a fairly extensive list of deep learning papers here:
[http://deeplearning4j.org/deeplearningpapers](http://deeplearning4j.org/deeplearningpapers)

We also Tweet out new ones as they're published here:
[https://twitter.com/deeplearning4j](https://twitter.com/deeplearning4j)

~~~
morenoh149
If you had to pick one which would it be? the oldest one?

------
currywurst
I really enjoyed this write up .. would be so great if research papers in
general were commented on like this (what is interesting, what is the
significance of the result etc)

------
andrewtbham
Here is the same list but including another important paper, inception-v4, it
beat MS resnets on top 5 error.

[https://github.com/andrewt3000/MachineLearning/blob/master/c...](https://github.com/andrewt3000/MachineLearning/blob/master/cnn4Images.md)

------
epberry
Glad to see R-CNN and its follow-on work on the list. We've been using R-CNN
for a few weeks now and have seen great results on object detection and
localization. A few papers this year have played around with substituting
different convnets and different classification schemes and improving the
network in various ways. I'm excited to see where this specific architecture
goes in the next few years.

------
raverbashing
Yes, it's a lot of papers, this might help people keep on top of the most
important ones (though it works from the top of the queue, and not from the
bottom up) [http://www.arxiv-sanity.com/](http://www.arxiv-sanity.com/)

------
haddr
Only applied to image recognition.

~~~
rer
The "Generating Image Descriptions (2014)" paper also involves text.

