Quick Summary - Not having enough data to access won't affect your results as much as you might think, and so shouldn't prevent you from trying out Deep Learning.
The thing about that is it might depend on the data set you work with.
I have the impression that machine learning kind of torn between accomplishing things that humans can do but machines can't and trying to get useful information from inherently uninformative data sets.
I can see that recommendation engines are things that a lot of companies want but when recommendations just come from prior history, perhaps there are hard limits to how much any pattern finder can deduce. No doubt, one can define the problem and get better but the reason that adding more data in this case doesn't get you that much improvement is that you hit diminishing returns on your data.
The article reminded me of a paper that described a NN that was able to learn with just a few examples. However I'm not able to find that paper on my notes.
This isn't deep learning (or a neural network at all). However, it is an extremely interesting approach.
Most of the previous "low data" deep learning approaches I've seen are broadly based around the approaches seen in "Zero-Shot Learning Through Cross-Modal Transfer"[1]
That's not really low data in the sense that it needs lots of data for initial training, but then is able to learn new things with very few examples.
We do not presume to come to this Thine output trusting in our own correctness, but in Thy manifold and great Processors. Print, we beseech Thee, the content of Thy variable X, according to Thy promises made unto mankind through Thy servant Alan, in whose name we ask. Amen.
Off the top of my head and a quick search online, I can't think of what you're referring to, but a search for papers on data augmentation methods might find it. It would be helpful to know what context you're referring to - visual data, text, etc.
Edit: Unless you're referring to transfer learning to domains with limited training data?
That's not entirely fair when sites are unavailable. And especially annoying is when sites are behind paywalls. Sometimes I use an old mobile device, and most news sites cause the browser to crash. So I just read the comments instead.
Though leaving a top level comment based just on the title is a little extreme. Even if you know the subject, you have no idea what the argument or information in the article is. I wouldn't mind replying to someone else's comment though.
You're main points are true, but I think you have one thing backwards. We're not finding a way around stupidity, we need to teach machines to perceive things the same way we do.
All of the Piaget conservation illustrations are critical towards understanding bottom up "raw" perception that is untempered by top down logic. I'll actually be more impressed when we can show machines functioning in the same way.
That was not the state of the art then. You can read the state of the art for yourself in the 2012 ImageNet competition (see the 2nd place team): http://image-net.org/challenges/LSVRC/2012/results.html. Fisher vectors + SIFT + GIST + some other hand-crafted features.
Basically just neural networks. The significant difference is that in the past neural networks were "shallow" and only had one layer (as were most other machine learning algorithms.) Now people are building neural networks with hundreds of layers and millions of parameters.
This is the usual definition, assuming NN, because it's a conveniently general model with a well-understood training method. However, to me, deep learning is a more general concept: several layers to learn features, with a final classification or regression layer; where "layer" is some statistical learning process. The key idea is that features can be learned in a hierarchical way, not so much that they are NN layers, which is an implementation detail.
For example, several layers of high-dimensional k-means unsupervised learning followed by a simple linear classification stage can perform very well.
