Here's a summary. Yann Le Cun discusses his research on deep learning. The basic problem with standard learning, SVMs, neural networks and the like is that they are limited to coming up with shallow templates for classification, with more or less fancy versions of linear and nonlinear combinations of weights. The number of such templates you need in a highly dimensional space like computer vision is exponential (think about how much data you need to represent one object at different combinations of distance, lighting, orientation, focus, etc.).
Instead, he suggests, we need to learn how to get past shallow template-matching and train, essentially, features of features or networks of networks. This gives us a shot at discovering highly abstract features of the data we have and doing real learning.
If you have any interest in the subject, I strongly suggest you carve out an hour for the video.
TD-Gammon is another example. It is a learning program that plays backgammon. Training from simple board features, it eventually derived what you might call first-order expert features by doing shallow learning. But, when similar first-order expert features (presence of an anchor, etc.) were added by hand to the initial state space, TD-Gammon derived deeper expert features to the point that it and similar programs became better than the best players in the world.
The point here is that there was no way to transition from the shallow features to the deep features without recoding the system from scratch. That obviously won't scale to the kinds of AI problems that are interesting.
Big finish: The advantage humans have over computers (at the moment) is that we work on those multiple levels of abstraction all the time. We do deep learning in every field of human endeavor. That's what the deeply connected neural networks in our brain are all about. In fields where such expertise is possible, humans have it all over the computers. In fields where brute-force calculation can win, the computers have it all over the humans.
What that implies to me is that we can't train computers to think as deeply about catastrophes as we do. It requires a new paradigm of learning to get the computer to that point.
It seems like the subject of this Google Tech Talk keeps coming up over and over again.
http://video.google.com/videoplay?docid=-2469649805161172416
Here's a summary. Yann Le Cun discusses his research on deep learning. The basic problem with standard learning, SVMs, neural networks and the like is that they are limited to coming up with shallow templates for classification, with more or less fancy versions of linear and nonlinear combinations of weights. The number of such templates you need in a highly dimensional space like computer vision is exponential (think about how much data you need to represent one object at different combinations of distance, lighting, orientation, focus, etc.).
Instead, he suggests, we need to learn how to get past shallow template-matching and train, essentially, features of features or networks of networks. This gives us a shot at discovering highly abstract features of the data we have and doing real learning.
If you have any interest in the subject, I strongly suggest you carve out an hour for the video.
TD-Gammon is another example. It is a learning program that plays backgammon. Training from simple board features, it eventually derived what you might call first-order expert features by doing shallow learning. But, when similar first-order expert features (presence of an anchor, etc.) were added by hand to the initial state space, TD-Gammon derived deeper expert features to the point that it and similar programs became better than the best players in the world.
The point here is that there was no way to transition from the shallow features to the deep features without recoding the system from scratch. That obviously won't scale to the kinds of AI problems that are interesting.
Big finish: The advantage humans have over computers (at the moment) is that we work on those multiple levels of abstraction all the time. We do deep learning in every field of human endeavor. That's what the deeply connected neural networks in our brain are all about. In fields where such expertise is possible, humans have it all over the computers. In fields where brute-force calculation can win, the computers have it all over the humans.
What that implies to me is that we can't train computers to think as deeply about catastrophes as we do. It requires a new paradigm of learning to get the computer to that point.