
Should I Machine-Learn? (2014) - luu
http://blog.nullspace.io/should_i_machine_learn.html
======
olooney
Andrew Ng once tweeted about a 1-second heuristic:

> Pretty much anything that a normal person can do in <1 sec, we can now
> automate with AI.

[https://twitter.com/andrewyng/status/788548053745569792?lang...](https://twitter.com/andrewyng/status/788548053745569792?lang=en)

And was summarily derided for it. Read the replies to his tweet to see some of
the suggested counter examples.

I sympathize with Ng here because he's making an effort to describe the
boundaries and capabilities of ML to non-technical people in a simple way, and
the 1-second heuristic is about as good at that as any I've seen. Yet it's
incredibly difficult to delinieate what ML is good and bad at because it's so
_different_ than human or animal intelligence!

On the one hand, Modern ML is reasonably good at what Daniel Kahneman calls
"System 1" thinking - fast, intuitive, stimulus-response stuff. However, for a
lot of this stuff, the AI can do it MUCH faster, sometimes millions of times
faster.

On the other, There are many tasks that modern ML can do that humans cannot
do, such as learn to see digits even after the pixels have been scrambled in a
deterministic way. To the human eye, it always looks like static, but (non-
CNN) neural nets don't care and still achieve 99% accuracy on MNIST. A very
broad category of things where ML is just plain better than humans is nuanced
probability estimates. While human intuition into probability appears to be a
pile of fallacies and broken rules of thumb, ML algorithms with appropriate
loss functions can achieve very high "calibration" in the sense that the
probability estimates they produce closely match the empirical conditional
probabilities in real-world data.

~~~
jasode
_> Andrew Ng once tweeted about a 1-second heuristic [..] And was summarily
derided for it. Read the replies to his tweet to see some of the suggested
counter examples._

Fyi to add some extra context to that "1 sec heuristic"...

I've seen several Andrew Ng presentations over the last several years and in
the live talks, he has more of a lead up to that "1 second" idea[0]. He was
trying to help project managers at Baidu _think_ about the possibilities of
_new_ AI products to build.

Unfortunately, the extreme brevity of that tweet removes all that surrounding
context that so it makes him look like an AI crackpot instead of an AI
realist. (E.g. The top tweet reply from Pedro Domingos seems to be based on
the limitations of _current_ AI but Andrew Ng was trying to convey the idea to
PMs of what _new_ AI to build.)

[0] deep link at 12m20s for context and 14m05s is the "1 sec heuristic":
[https://youtu.be/21EiKfQYZXc?t=740](https://youtu.be/21EiKfQYZXc?t=740)

~~~
olooney
Thank you, good clarification, sorry if I made it sound I was dog-piling on
Ng. Andrew Ng works harder at explaining ML in a non-technical way than anyone
else I can think of, and the 1-second rule came out of that genuine desire to
educate. I've been in the exact same boat: a senior executive who "used to
code" trying to get me to explain to him in simple terms what kind of problems
ML could solve; I sort of fumbled through it by mentioning the 1 second rule
and giving some concrete examples. It's just absurdly hard to put it in non-
technical terms.

------
CoffeePython
>In contrast, take music recommendations. If I give you a series of suggested
songs, can you detect a pattern?

It seems to me that there has been a lot of success in recommendation engines
(i.e. Spotify, the "Netflix Algorithm", etc). Maybe not so much on a smaller
scale, but large companies are using recommendation engines all the time to
keep user's engaged.

Other than that, I agree with the rest of the post. The eagerness to throw ML
at any and everything can be kind of exhausting. When used right, it can be
super powerful but sometimes a simpler method that achieves 80% of the results
may be enough for certain use cases.

~~~
rorykoehler
Spotify recommendation engine is half the reason I use Spotify.

~~~
CoffeePython
Yeah same here! The daily mixes that Spotify makes and groups based on my
different listening habits has been pretty great. I get a ton of value out of
it.

~~~
moccachino
They were great for me until we had a kid. Now it's unusable because they will
mix 'real' music with annoying kid stuff all over the place.

~~~
learc83
Netflix had the same problem before they added profiles.

