

A Personal Perspective on Machine Learning - jmount
http://www.win-vector.com/blog/2010/10/a-personal-perspective-on-machine-learning/

======
iandanforth
"I would temper this with we are likely in the age of unmarked data and
unstructured data. Less often are we asked to automate a known prediction and
more often we are asked to cluster, characterize and segment wild data. In my
opinion the hard problem in machine learning has moved from prediction to
characterization."

I would say instead we are in the age of Discovery. The hard problem is
discovering causes that give rise to coherency in large sets of real world
data.

As to whether we are in the age of large computers, the classification is
irrelevant. We are in the age of sufficiently large computers. There is
sufficient power and scale to accomplish everything that the brain does with
our current infrastructure. Is it highly inefficient on a watt/thought basis?
Yes. But that's a secondary metric.

To address the concerns the author outlines:

Dealing with lack of training data - when you move to discovery labeled data
is far less relevant, which should relieve research assistants who are
currently being asked to label images as cat/not-cat.

Dealing with large sets of unstructured data - this is a processing problem
for which the current infrastructure is sufficient.

IMO we have plenty of data, we don't need many more labels, and we have
sufficient processing power. That leaves algorithmic work for the discovery of
causes. (My bet is on systems similar to the kind of sparse distributed
hierarchical models that Numenta is working on.)

One thing that I don't think most people have realized is how far past the
quantity of data and processing power we need we already are.

If you were to take all the visual information the brain receives in the first
30 years of life (plenty of time to become intelligent) you'd arrive at a
number around 1.4 petabytes. This is hardly a large number compared to the
amount of information processed by YouTube on a monthly basis. Hearing, taste,
and smell all add in a fraction more, and touch (well I don't have any good
numbers there yet).

The hard part now is going to be convincing machine learning researchers to
actually expose their algorithms to a swath of data similar to the inputs we
humans get, over sufficient time, so as to allow for the development of a
robust hiearchy of stored causes (aka understanding.)

AFAICT - Experience _is_ understanding.

------
zemaj
"In my opinion the hard problem in machine learning has moved from prediction
to characterization."

Is this because prediction has been "solved" or because characterization is
currently more commercially useful?

~~~
jmount
I was hoping nobody would outthink me and ask that. I am going equivocate a
bit. Now that we have kernel methods (for interactions) and graphical models
(for dependencies) you could claim prediction is "solved" in that if you guess
the right feature design (no small task) current optimization methods are up
to the task of prediction. Of course some better method could come along and
we would be forced to say the current level of prediction turned out to be
inadequate. Clustering and characterization are indeed where a lot of the
current applications lie and can range from fascinating to unpleasant
(depending on details of the problem domain).

------
lkozma
"if a system is well understood or works then it is no longer called
artificial intelligence." This was also observed already by von Neumann.

