additional flavor: http://arxiv.org/pdf/1412.0233.pdf
and slides 16-22 of https://docs.google.com/viewer?a=v&pid=sites&srcid=ZGVmYXVsd...
> Multiscale convolutional neural networks aren't that much different than the feature-based systems of the past. The first level neurons in deep learning systems learn to utilize gradients in a way that is similar to hand-crafted features such as SIFT and HOG. Objects used to be found in a sliding-window fashion, but now it is easier and sexier to think of this operation as convolving an image with a filter. Some of the best detection systems used to use multiple linear SVMs, combined in some ad-hoc way, and now we are essentially using even more of such linear decision boundaries. Deep learning systems can be thought of a multiple stages of applying linear operators and piping them through a non-linear activation function, but deep learning is more similar to a clever combination of linear SVMs than a memory-ish Kernel-based learning system.
> Features these days aren't engineered by hand. However, architectures of Deep systems are still being designed manually -- and it looks like the experts are the best at this task. The operations on the inside of both classic and modern recognition systems are still very much the same. You still need to be clever to play in the game, but now you need a big computer. -
It's worth saying that deep learning is very powerful at find features but isn't adaptive, requiring careful hand-tuning for a given feature-set.
My question is, at what point does this situation start to resemble the situation of computer-programs and their interfaces, where once a good-enough computer-based solution to a problem arise people are forced to adapt to the solution rather having the solution adapt to them (learning the irrational and counter-intuitive interfaces of X program and then having that knowledge codified as a skill, etc).
I already find myself speaking in a chirpy, robotic view when I am called by the chirpy robotic programs which may sometimes understand me.
When deep learning first started, it was actually very unsupervised focused with stacked autoencoders and DBMs, the most well known paper around that time would probably be the youtube cat recognizing paper from Google/ Stanford . A lot of the "early" works (from around 2006-2010) is unsupervised learning as well, just look up Geoff Hinton paper around that time.
There is going to be an AI engine that we use in the future in a similar way that we use Linux today. I mean that "future Linux-ish AI engine" and unless Linus has been busy ML-ing, it won't be called Linux.