
ImageNet Large Scale Visual Recognition Competition - luu
http://image-net.org/challenges/LSVRC/2014/results
======
jbarrow
I love looking at the results and seeing the annual improvement and breakout
strategies. Deep learning has really come into its own during the last eight
years.

People interested in learning about some of the techniques employed (CNNs and
deep networks) are encouraged to take Geoff Hinton's Coursers course [1],
which does a really good job of covering modern techniques from one of their
pioneers.

[1]
[https://www.coursera.org/course/neuralnets](https://www.coursera.org/course/neuralnets)

~~~
rjtavares
How much are results improving? What's the general long term trend?

~~~
dave_sullivan
The trend is depreciating returns with current architectures (after truly
breakthrough results in the first place). It's a good time for companies to
update their ML pipelines to use deep learning. Performance does seem to scale
with hardware though, so as hardware improves (as well as the software), these
models will see some more gains without any real technology change.

But I should also point out deep learning doesn't work for everything. Random
forests or SVMs work about as well on certain (I would argue "more linear")
datasets. Deep learning is for when RFs and SVMs aren't giving the results you
need (or at all). It doesn't negate all things that came before, nor should
it.

One thing to consider: As far as I can tell, there's an order of magnitude
more people working on these types of problems with an order of magnitude more
money being thrown at it and an order of magnitude increase in "collaborative
efficiency" due to Internet and open source/research. I think breakthroughs
are thought of best as a probability distribution, but I think the probability
of further breakthrough successes has increased considerably in just the last
few years.

There are reasons for optimism and pessimism, as usual.

------
Houshalter
The winner got 6.7% in top-5 classification error. 2013 was 11%, 2012 was 15%.
This may be _better_ than exponential, since there is an upper limit of 0%
error, and every additional percentage point is harder than the last.

Exponential improvement in this area, and it's a pretty significant field that
will affect everyone. The results will likely help improve neural networks in
other areas as well, as NNs are extremely general purpose.

Here is an example from a system from 2012 in the same competition to give an
example of what this is: [http://mappingignorance.org/fx/media/2013/04/Deep-
learning-5...](http://mappingignorance.org/fx/media/2013/04/Deep-
learning-5.png) A bunch of images were scraped from Flickr and then given a
single label by a human. The computer gets 5 guesses to guess what label the
human used. As you can see, even when the computer fails, it's guesses are
usually semantically very good. This specific system got an error rate of 17%.

You can explore the images of ImageNet here: [http://image-
net.org/explore](http://image-net.org/explore).

Here are some image recognition systems online you can play with:

* Toronto Deep Learning Demo: [http://deeplearning.cs.toronto.edu/](http://deeplearning.cs.toronto.edu/) * clarifai: [http://www.clarifai.com/](http://www.clarifai.com/) * AlchemyVision: [http://www.alchemyapi.com/products/demo/alchemyvision/](http://www.alchemyapi.com/products/demo/alchemyvision/)

Note these aren't related to the winner of this competition and probably are
no where near as good. Just examples of this kind of system you can play with.

------
cr4zy
Looks like BVLC's Caffe and R-CNN are getting a lot of mindshare. Also
interesting to see that the creator of Caffe, Yangqing Jia, was on team
GoogLeNet and didn't appear to have used either.

Edit: Last years results with no Caffe (just Decaf, its predecessor)
[http://image-net.org/challenges/LSVRC/2013/results](http://image-
net.org/challenges/LSVRC/2013/results)

~~~
dave_sullivan
Cuda convnet 2 was recently released, it's faster than caffe it seems. Caffe
is very cool though, I've been playing with it a bit.

[https://code.google.com/p/cuda-convnet2/](https://code.google.com/p/cuda-
convnet2/)

~~~
cr4zy
Hey Dave! (Craig here) nice, achieved 6x speed up on an 8 gpu box
([http://arxiv.org/pdf/1404.5997v2.pdf](http://arxiv.org/pdf/1404.5997v2.pdf)).
Thanks for the pointer and hope to see you soon.

~~~
dave_sullivan
Hey Craig, nice to hear from you, I hope things are going well. That is a nice
bump! The data and model parallelism they added is very cool. We'll be hosting
a meetup soon, it's been a while, hope to see you there.

------
contingencies
_Science is but a perversion of itself unless it has as its ultimate goal the
betterment of humanity._ \- Nikola Tesla

Does this not nearly amount to "population-scale mass surveillance
algorithms"? Do people not feel this is accelerating negative social impacts
of technology?

Is it merely a coincidence that winning teams include many from countries
criticized for their totalitarian social contracts: Hong Kong University of
Science and Technology, National University of Singapore, Microsoft Research
China, Southeast University (China), Chinese Academy of Sciences? There's also
a presence from Holland.

Oh, and guess who won the category "with additional training data"? Google.

Come on people, we can do better than this! _SHAME SHAME SHAME._

~~~
Houshalter
Machine vision has _numerous_ applications, better surveillance being only one
of them. Mainly self driving cars, and significantly better AI for robotics
and some other automated systems.

The organizations you mention are just places that invest in or do a lot of
research in AI.

