
Yann LeCun's comment on AlphaGo and true AI - brianchu
https://www.facebook.com/yann.lecun/posts/10153426023477143
======
Smerity
Preface: AlphaGo is an amazing achievement and does show an interesting
advancement in the field.

Yet ... it really doesn't mean almost anything that people are predicting it
to mean. Slashdot went so far as to say that "We know now that we don't need
any big new breakthroughs to get to true AI". The field of ML/AI is in a fight
where people want more science fiction than scientific reality. Science
fiction is sexy, sells well, and doesn't require the specifics.

Some of the limitations preventing AlphaGo from being general:

\+ Monte Carlo tree search (MCTS) is really effective at Go but not applicable
to many other domains we care about. If your problem is in terms of {state,
action} pairs and you're able to run simulations to predict outcomes, great,
but otherwise, not so much. Go also has the advantage of perfect information
(you know the full state of the board) and deterministic simulation (you know
with certainty what the state is after action A).

\+ The neural networks (NN) were bootstrapped by predicting the next moves in
more matches than any individual human has ever seen, let alone played. It
then played more against itself (cool!) to improve - but it didn't learn that
from scratch. They're aiming to learn this step without the human database but
it'll still be very different (read: inefficient) compared to the type of
learning a human does.

\+ The hardware requirements were stunning (280 GPUs and 1920 CPUs for the
largest variant) and were an integral part to how well AlphaGo performed - yet
adding hardware won't "solve" most other ML tasks. The computational power
primarily helped improve MCTS which roughly equates to "more simulations gets
a better solution" (though with NNs to guesstimate an end state instead of
having to simulate all the way to an end state themselves)

Again, amazing, interesting, stunning, but not an indication we've reached a
key AI milestone.

For a brilliant overview: [http://www.milesbrundage.com/blog-posts/alphago-
and-ai-progr...](http://www.milesbrundage.com/blog-posts/alphago-and-ai-
progress)

John Langford also put his opinion up at:
[http://hunch.net/?p=3692542](http://hunch.net/?p=3692542)

(note: copied from my Facebook mini-rant inspired by Langford, LeCun, and
discussions with ML colleagues in recent days)

~~~
gracenotes
I took a closer read through the AlphaGo paper today. There are some other
features that make it not general.

In particular, the initial input to the neural networks is a 19×19×48 grid,
and the layers of this grid include information like:

\- How many turns since a move was played

\- Number of liberties (empty adjacent points)

\- How many opponent stones would be captured

\- How many of own stones would be captured

\- Number of liberties after this move is played

\- Whether a move at this point is a successful ladder capture

\- Whether a move at this point is a successful ladder escape

\- Whether a move is legal and does not fill its own eyes

Again, before the neural nets even get involved. Some of these layers are
repeated 8 times for symmetry. I would say for some of these, AlphaGo got some
domain-specific help in a non-general way.

It is of course still groundbreaking academically. The architecture is a
state-of-the-art deep learning setup and we learned a ton about how Go and
games in general work. The interaction between supervised and reinforcement
learning was interesting, especially how the latter behaved worse in practice
in selecting most likely moves.

disclaim: Googler, not in anything AI.

~~~
bbsome
Note, that these features are for the RollOut fast policy. The reason is that
this needs to be fast, so rather than a net they have a linear policy. A
linear policy in order to work it requires good feature selection, which is
what this is. In some future, when we have better hardware, you can imagine
removing the roll out policy and having just one.

~~~
aprescott
I think you're confusing this list of attributes with a separate list used for
rollouts and tree search. The ones above are definitely used for the neutral
networks in the policy and value networks. See: "Extended Data Table 2: Input
features for neural networks. Feature planes used by the policy network (all
but last feature) and value network (all features)."

------
fallingfrog
I don't know if I'd agree that unsupervised learning is the "cake" here, to
paraphrase Yann LeCun. How do we know that the human brain is an unsupervised
learner? The supervisor in our brains comes in the form of the dopamine
feedback loop, and exactly what kinds of things it rewards aren't totally
mapped out but pleasure and novelty seem to be high on the list. That counts
as a "supervisor" from a machine learning point of view. It's not necessary to
anthropomorphize the supervisor into some kind of external boss figure; _any_
kind of value function will do the trick.

~~~
stormbuilder
Wouldn't that be a self-supervised learning? Yes, as the brain lears, it gets
rewarded or not, but not by any external organism - rather by itself (or a
part of itself). The "learning instruction manual" is in there somewhere.

So if you are able to code an AI so that it can run experiments over itself
and gradually build this "learning instruction manual" eventually it will
become able to do things you never though of in the first place.

~~~
fallingfrog
I think that's what the AlphaGo team did - they trained their agent against
itself, and it learned new moves not explicitly programmed in! With an
evaluation function just saying ahead / not ahead.

------
sago
Ah, the joys of arguing about artificial intelligence without ever defining
intelligence.

It is the perfect argument, everyone can forcefully make their points forever,
and we'll be none the wiser whether this AI is 'true AI' or not.

~~~
conceit
So then the discussion is all about defining intelligence, beginning with a
fuzzy conception of required qualities. It literally means ability to select,
read, choose between. The discussion can be a means to judge the AIs or, for
the sake of the argument, to judge and improve intelligence with AI as a
heavily simplified model, which is an old hat by now.

Do you think that's irrational? Do you expect neuroscience or are you rather
interested in mathematics? I thought that's how to learn in absence of
external inputs, by recombination of the old inputs, ie. the fuzzy notions, to
generate new ones. I'd think that's how recurrent networks work. What do you
know about it (honest question)?

 _Fuzzy_ doesn't mean wrong. Underspecification, as little as I know about it,
is a feature.

~~~
sago
I remember similar arguments in the early 90s when I was doing my PhD. They
got about as far then. The same arguments will be happening in another 25
years. And beyond, even when a computer is super-human in every conceivable
way, there'll be arguments over whether it is 'real AI'. And nobody will
define their terms then either. Ultimately the discussion will be as
irrelevant then as it is now, and, then as now, it will be mostly take place
between well-meaning undergrads, and non-specialist pundits.

(Philosophers of science have a discussion that _sounds_ similar, to someone
just perusing the literature to bolster their position, but the discussion is
rather different though also not particularly relevant, in my experience.)

> What do you know about it (honest question)?

About recurrent NNs? Not much beyond the overview kind of level. My research
was in evolutionary computation, though I did some work on evolving NN
topologies, and using ecological models to guide unsupervised learning.

~~~
conceit
> About recurrent NNs?

yes, obviously, as that's the topic, but also the the rest I of what I
mentioned, neuroscience, maths, leaning on logic and philosophy.

> I remember similar arguments in the early 90s

That's why I mention neuroscience, the ideas are much older.

> even when a computer is super-human in every conceivable way, there'll be
> arguments over whether it is 'real AI'

Of course. Just because it's superhuman, we humans wouldn't know what it is,
whether it is what we think it is and if that's all there could be.

Real (from res (matter (from rehis (good as in the goods))) + ~alis (adjective
suffix (from all?))) means worthy and obviously an AI is only as good as its
contestants are bad. It won't be real for long before it's thrown to the trash
once a better AI is found.

That'll stop when the AI can settle the argument convincingly. That's what's
going on, evangelizing. And we do need that, because if not for the sake of
the art itself, then as proof for the application of answers and insights
other fields.

> And nobody will define their terms then either. Ultimately the discussion
> will be as irrelevant then as it is now

LeCun sure went along a lot further since then, and he defines the terms in
software. As I said, the discussion is just about what to make of it. Of
course many come up basically empty, that's why the discussion is important,
and that's why I asked _what do you know about it_. I think it's a very basic
question and not easy to grow tired of. If you work that, maybe that's
different and specialized to computation.

There might not be much to say about it, all the easier then to summarize in a
short post. Or there's indeed more to it, then I'd appreciate a hint, to test
my own understanding and learn.

I don't really know, what LeCun talks about, or the techniques you studied, so
I'm saying _it_. Just for perspective. I'm just generally interested in
learning and computation is just one relevant and informative angle. Maybe
that's what bothers you, learning to learn, and that's why its freshman
bothering with it, but learning to learn is maybe really just learning, or
impossible. That's the kind of logical puzzle that's to be taken half joking.
Don't beat yourself up over it.

------
hacknat
I think we need more advances in neuroscience and, I know this will be
controversial, psychology before we really know what the cake even is.

Edit:

I actually think the major AI breakthrough will come from either of those two
fields, not computer science.

~~~
emcq
I disagree. In engineering we've shown great advancements not by exactly
reproducing biology from cases like transportation (wheel vs legs) or flight
(engine and wing vs flapping feathers). We can't even mass produce synthetic
muscles and instead use a geared motor.

The growth in building increasingly sophisticated AI is faster than our
efforts to reverse engineer biology. I could see that changing with improved
observational techniques like optogenetics or bacteria and viruses we can
"program" to explore.

Researchers are already focusing on concrete insights from cognitive science,
neuroscience, etc such as one shot learning or memory that we haven't yet
figured out in a cohesive machine learning framework. For the time being I'd
bet on more advaces coming without significant changes in biological
understanding.

~~~
sabertoothed
I feel that your analogy does not disprove OP's claim. You would first have to
prove that the analogy is actually applicable. Theoretically, one could argue
that motion is substantially different from intelligence such that the analogy
does not hold.

But I am with you in that engineering / CS / ... should not wait for
neuroscience to make further discoveries but continue the journey.

~~~
emcq
Interesting point, but which do you think is a more reasonable prior: that
intelligence is a deterministic process similar other's we've encountered or
that there is something uniquely different about intelligence than other
physical phenomena? I think the latter requires more assumptions so I would
argue philosophically it is the one that requires the proof or more evidence
:)

Regardless, my feeling is that there is a healthy dose of human hubris around
intelligence. If I train a dog to go fetch me a beer from the fridge, that
seems pretty smart. It learned how to understand a request, execute a complex
sequence of actions for motion and planning, reason around occluded objects,
understand depth and 3 dimensional space, differentiate between objects, and
more without me writing a sequence of rules to follow. I'd be happy to have a
robot that intelligent. Plants dont have brains or neurons yet react to
sensory stimulus such as light touch or sound, communicate, and even have
memory and learn. It's not at a scale to do something interesting within one
plant but communities of plants are arguably the most successful organisms on
the planet.

Andrew Ng likes to point to a Ferret experiment [0] where experimental
neuroscientists rewired the visual inputs to the auditory cortex and the
auditory cortex learned to "see" the visual signals! This suggests that there
may be some amount of unified "learning" rules to the brain. Biology is never
so clean but if humans have whatever this intelligence thing is that lesser
organisms do not, there is another angle to look at things. We have a lot of
neurons which suggests less per neuron specialization than say a C. elegans;
basically large populations of neurons perform tasks that in lesser creatures
single or few neurons may perform. While the trees are complex and important
to understand for biology and medicine, the forest may have some high level
rules.

Looking at something that appeared intelligent 50-100 years ago but seems
mechanical now, we have text to speech. NETtalk was a simplified computational
neuroscience model from the 80s that could synthesize human speech. Today we
have far better quality techniques that came out of R&D focused on things like
large high quality labeled datasets for training, better soundcards, more
processing power, and algorithmic improvements. Researchers didn't continue
trying to model the brain and instead threw an HMM and a couple other tricks
at it. Now we're going full circle back to neural networks but they aren't
using any advances from biology and certainly arent produced by computational
neuroscientists like Terry Sejnowski.

It's funny because at the time of NETtalk they thought that learning to read
would be an extremely hard problem because it incorporates so many components
of the human brain [1]. While it certainly wasn't a trivial problem, state of
the art OCR and object recognition came from similar artificial neural
networks a decade later with LeNet and MNIST * . And no, ANNs != biological
neuronal networks. The models of computational neuroscientists are different;
for example look at [2, 3] for high level models or [4] for a tool.

Now I'm even more convinced than before that understanding the brain is great
for humanity but wont be necessary for building intelligent systems that can
perform tasks similar to biological ones.

[0]
[http://www.nature.com/nature/journal/v404/n6780/full/404871a...](http://www.nature.com/nature/journal/v404/n6780/full/404871a0.html)

[1]
[https://en.wikipedia.org/wiki/NETtalk_(artificial_neural_net...](https://en.wikipedia.org/wiki/NETtalk_\(artificial_neural_network\))

[2]
[http://science.sciencemag.org/content/338/6111/1202](http://science.sciencemag.org/content/338/6111/1202)

[3] [http://ganguli-gang.stanford.edu/pdf/InvModelTheory.pdf](http://ganguli-
gang.stanford.edu/pdf/InvModelTheory.pdf)

[4]
[http://neuralensemble.org/docs/PyNN/index.html](http://neuralensemble.org/docs/PyNN/index.html)

* Perhaps you could argue that convolutions are loosely inspired by neuron structure, but that sort of knowledge had existed for quite some time, with inspiration arguably within Camillo Golgi's amazing neuronal physiology diagrams from the 1870s let alone the 1960-80s. It's telling that papers on CNNs have little to no neuroscience and a lot of applied math :)

~~~
sabertoothed
I did not mean to have implied that I think there is anything magical about
intelligence. Of course, it is based on physical phenomena. I am doing my PhD
right now and I try to incorporate as much AI/ML into it as possible.

What I meant to say is that our ANNs are so ridiculously simplified versions
of real neural networks that there might still be something to be learnt from
the real brain. This shall not imply that to achieve intelligence, the
solution necessarily has to mimic a biological brain.

(Thank you for your detailed response. I love to read about this stuff!)

edit: missing word

------
pavanky
Can someone more knowledgeable explain why biological systems are considered
unsupervised instead of reinforcement based systems?

While it seems intuitive that most individual "intelligent" systems in animals
can be seen as unsupervised, isn't life itself driven in a reinforced manner?

~~~
nabla9
There is no enough reinforcement volume to learn anything complex.

If child gets external reward for every waking moment until she is 12 years
old, it's just 4.2 million signals.

Reinforcement learning works for fine motor control and other tasks where the
feedback loop is tight and immediate. Reinforcement and conditioning can also
modulate high level cognition and behavior, but it's not the secret sauce of
learning.

~~~
sabertoothed
I could not follow your argument. Could you elaborate? (Honest question)

~~~
nabla9
Reinforcement learning is learning by interacting with an environment. RL
agent learns from the consequences of its actions.

Biological system don't live long enough to get enough feedback to learn
complex behavior trough consequences. Animal or human must be able to
generalize and categorize what they have learned correctly without external
feedback teaching it how to derive the function that's doing it.

For example, if you want to learn how to tie a complex knot and learn it
trough trial and error you might have try it million times if you improve your
behavior mainly trough consequences of your actions. In practice you probably
try only 5-10 times before you learn to do it and it involves pausing and
looking at the problem. There is some kind of unsupervised model building
happening that is not involving external input.

~~~
sabertoothed
Ok, I see what you mean now.

But it seems like (or one could misunderstand you in way that) you see those
concepts as mutually exclusive. I would assume a combination of reinforcement
learning and unsupervised (and supervised) learning.

Rats have been trained to detect landmines and then go back to their trainers
and show them the mine. This is complex behaviour that was taught using
reinforcement (at least on a top level). There will be some unsupervised
learning going on in the rat's brain on a lower level. But it is complex
behaviour and it's been reinforcement learnt.

~~~
nabla9
>see those concepts as mutually exclusive.

I certainly don't. Reinforcement or conditioning is part of it, but it's not
the cake.

------
deegles
I would like future competitions between AIs and humans to have a "power
budget" for training and during gameplay. For example, a chess grandmaster
that has played for 20 years would have spent X amount of energy training. The
AI should get an equivalent budget to train with. During gameplay, the AI
would get the same 20 watts [1] that a human has. This would drive the
development of more efficient hardware instead of throwing power at the
problem :)

[1] [http://www.popsci.com/technology/article/2009-11/neuron-
comp...](http://www.popsci.com/technology/article/2009-11/neuron-computer-
chips-could-overcome-power-limitations-digital)

------
kzhahou
I'm surprised he'd make such an optimistic statement. I think a better analogy
would be:

We figured out how to make icing, but we still don't really know what a cake
is.

~~~
emmab
We can describe Solomonoff-based agents like AIXI. None of them are fully
sufficient for true general AI, but you could probably accomplish quite a bit
with an AIXI-like agent.

~~~
sampo
As fas as I know, the only thing existing AIXI implementations have
demonstrated, is to learn to play Pac-man at a somewhat reasonable, but not in
any way stellar level.

~~~
emmab
Yes, it is not tractable. It serves as an example of a definition of an agent
though.

------
maxander
It sounds reversed to me- shouldn't the "cherry" be supervised learning and
the "icing" be reinforcement learning? At least insofar as reinforcement
learning is closer to the "cake" of unsupervised learning, as there is less
feedback required for a reinforcement learning system to work (a binary
correctness signal rather than an n-dimensional label signal.)

It might also be argued that most "unsupervised learning" in animals can be
broken down into a relatively simple unsupervised segment (e.g., an "am I
eating nice food" partition function) and a more complicated reinforcement
segment (e.g. a "what is the best next thing to do to obtain nice food?"
function.) I'm sure someone like Yann LeCun is familiar with such arguments,
though.

------
grumpy-buffalo
I wish the term "true AI" were replaced with "strong AI" or "artificial
general intelligence" or some such term. We already have true AI - it's a
vast, thriving industry. AlphaGo is obviously a true, legitimate, actual,
real, nonfictional example of artificial intelligence, as are Google Search,
the Facebook Newsfeed, Siri, the Amazon Echo, etc.

~~~
ktRolster
_We already have true AI - it 's a vast, thriving industry._

Or how about calling that vast, thriving industry "weak AI," or "clever
algorithms," which is what they really are. The original definition of AI was
what we now call strong AI, but after some lesser problems were solved without
actually creating strong AI, we had to come up with some name for those.

~~~
Natsu
I want to see an AI that can improve itself by developing new algorithms for
arbitrary tasks. I wonder how far off we are from that now?

~~~
SilasX
You know, if you're at the point where you can give a human-readable spec of
the problem and the AI can make a passable attempt at it, that's basically the
Turing Test -- hence why I think it deserves its status as holy grail.
Something that passes would really give the impression of "there's a ghost
inside here".

~~~
Natsu
Rather than a ghost, I wonder if we'll ever have the average person looking at
brains and thinking "there's a program inside here."

And then to reverse it, imagine that the world really is some kind of massive
simulation... and that there are backups of the save()-ed :)

------
Houshalter
No one is claiming that alphaGo is close to AGI. At least not anyone that
understands the methods it uses. What alphaGo is, is an example of AI
progress. There has been a rapid increase in progress in the field of AI. We
are still a ways away from AGI, but it's now in sight. Just outside the edge
of our vision. Almost surely within our lifetime, at this rate.

~~~
sampo
> We are still a ways away from AGI, but it's now in sight.

These people in 1956 also thought that AGI was in sight:

[https://en.wikipedia.org/wiki/Dartmouth_Conferences](https://en.wikipedia.org/wiki/Dartmouth_Conferences)

~~~
Houshalter
Not most of them, and not with any certainty. The Dartmouth conference was
just the very first investigation into whether it was even possible and what
it would take. Of course they didn't know when AGI would be developed. The
computers they had access to were smaller than my calculator!

Now we have massive computers and 60 years of progress. And lots of real world
successes of AI technology. Surely we are in a much better position to predict
than people of 60 years ago!

~~~
sampo
> Not most of them, and not with any certainty.

This probably holds true also today.

------
sytelus
If you look at how child learns, it's huge amount of supervised learning.
Parents spend lots of time in do and don't and giving specific instructions on
everything from how to use toilet to how to construct a correct sentence. Lots
of language development, object identification, pattern matching,
comprehension, math skills, motor skills, developing logic - these activities
has huge amount of supervised training that runs day after day and year after
year. There is sure unsupervised elements like ability to recognize phonemes
in speech, tracking objects, inference despite of occlusion, ability to stand
up and walk, make meaningful sounds, identify faces, construct sequence of
actions to achieve goal, avoiding safety risks from past experiences and so
on. However, typical child goes through unparalleled amount of supervised
learning. There was an incidence of a child who got locked up in a room for
over a decade and she didn't developed most of the language, speech or social
skills. It seems unsupervised learning can't be all of the cake.

~~~
gracenotes
Interesting. I have heard the opposite of this.

Supervised learning may be how it looks from the outside, but consider that
out of the >6,570,0000 waking seconds of a child's life up to age 5, there
maybe only a few dozen instances of supervised adult instruction per day.
Besides those, what do neurons do the remaining 99.99% of the time?

Part of the problem might be that comparing supervised and unsupervised
learning 'effectiveness' is a bit apples-and-oranges. Their effect together is
highly collaborative. Children have to develop abstractions on their own
before you can supervise them on those abstractions. It is probably fair to
say that a key part of human general intelligence is creating high-level
representations of low-level stimuli. It might also be fair to say that this
is what the brain is doing 100% of the time.

So if I may hand wave a little: while supervised learning can make a child
better maximize objectives on those high-level representations (objectives
they may be aware of through unsupervised observation), for the most part it
does not fundamentally change the structure of those things in the child's
brain. This makes unsupervised learning almost all of the cake to me.

My post has the caveat that children undergo a lot of other objective-based
learning besides explicit instruction from adults, and all of this maps only
fuzzily to supervised vs unsupervised learning in AI, which is the issue from
the submitted post.

~~~
yoodenvranx
> there maybe only a few dozen instances of supervised adult instruction per
> day

There might be only a few dozen instances but I think each instance has a
lasting effect which makes up for this.

If you scold a child for something stupid it did then it will remember this
for a long-ish time. Same for teaching him things or correcting stuff.

I guess you show the child some correct behaviour at a few instances and this
is then used internally as a guideline for selflearning.

~~~
gracenotes
That is a great point. Talking about supervised vs unsupervised vs
reinforcement learning is most straightforward with tasks like language
learning, audio processing, image processing, and playing discrete games. It
is possible to see broad similarities between deep learning approaches and
human cognition for several of these tasks. But when you start getting into
tasks like the formation of narrative identity, things get very complicated.

Maybe one major difference between playing a game and forming a personality is
that these early important interactions don't just adjust wirings in the
cerebral cortex, the part of the brain most responsible for general
intelligence. It goes straight to our emotional memory bank in the limbic
system, which is _all about_ learning an incredibly important objective
function: to survive. But very high level features formed by unsupervised
learning can do this, not just reptilian predator detection routines. Being
scolded or corrected can have a powerful effect on future motivation. Suffice
to say, artificial intelligences don't currently worry about this.

------
javajosh
Is anyone working on an embodied AI? Even a simulated body might help.
Ultimately intelligence is only useful insofar as it guides the body's motion.
We often tend to minimize the physical act of say, writing down a theorem or
actually applying paint to the canvas, but there are certain actions like
playing a musical instrument that certainly blur the distinction between
"physical" and "mental". Indeed, even 'purely mental' things like having an
"intuition" about physics is certainly guided by one's embodied experience.

~~~
wpietri
You might look at the work of Rodney Brooks. There was a wonderful documentary
called "Fast, Cheap, and Out of Control", and I'm pretty sure it was in there
that he explains his notion that true intelligence is embodied. That was years
ago and I haven't kept up with the field, but perhaps it's a useful starting
point for you.

~~~
javajosh
Cool, thanks. I think the idea is also important to Zoltan Torey, who believed
that "consciousness" is essentially a simulation of a kind of arm in our mind.
("The Conscious Mind"). But I've never heard of anyone attempting to train an
AI within a simulated world. The nice thing about doing that is that aren't
limited by physics of robotics. I suspect that for basic intelligence stuff,
relatively lo-fidelity simulation would be sufficient. (Although there are
some very important moral issues presented by such a program!)

~~~
wpietri
> But I've never heard of anyone attempting to train an AI within a simulated
> world.

Ah, then you might want to follow the trail from SHRLDU:

[http://hci.stanford.edu/winograd/shrdlu/](http://hci.stanford.edu/winograd/shrdlu/)

[https://en.wikipedia.org/wiki/SHRDLU](https://en.wikipedia.org/wiki/SHRDLU)

~~~
javajosh
Thanks for the link! That's pretty close to what I mean; but I was thinking
more like an embodied AI living in a simulated world. So, for example, you
might start from a Counter-Strike bot, input to the bot would be it's
'camera', and output would be it's position and actions.

------
Animats
What we need next are more systems which can predict "what is likely to happen
if this is done". Google's automatic driving systems actually do that. Google
tries hard to predict the possible and likely actions of other road users.
This is the beginning of "common sense".

------
scotty79
> As I've said in previous statements: most of human and animal learning is
> unsupervised learning.

I don't think that's true. When baby is learning to use muscles of its hands
to wave them around there's no teacher to tell it what should its goal be. But
physics and pain teaches it fairly efficiently which moves are bad idea.

It has built in face detection engine and the orienting and attempting to move
and reach towards it is clear goal. Reward circuit in the brain do the
supervision.

~~~
elementalest
The difference between supervised and unsupervised is that the inputs are
paired with known outputs for supervised. In unsupervised the agent has
(initially) no knowledge of what the outputs will be, given the inputs.

The baby does not know (initially) that something will cause pain, or the
extremities of its joints. It must learn this over time and experience. The
baby must also learn how to use the built in components, as it has no idea
what outputs will occur given the inputs.

As you allude to, there are built in mechanisms/configurations in the brain
which provide various forms of feedback, as well as built in behaviours and
responses. If there was no basic structure to the brain, I think it would be
almost impossible for an unsupervised agent to develop and learn to the
complexity and level of a human brain. These basic behaviours significantly
speed the initial development process up.

~~~
scotty79
> The difference between supervised and unsupervised is that the inputs are
> paired with known outputs for supervised. In unsupervised the agent has
> (initially) no knowledge of what the outputs will be, given the inputs.

I'd still call learning to move, supervised (or reinforced) then. You're
feeding the world some input (muscle contractions), and the world immediately
gives you the output in terms of pain. You are using it to adjust your
internal function. After a while you have pretty good function that maps you
muscle contractions to whether it valid move or not and you can generalize it
to when your position is different and get to some other stuff like trying
which moves can alter what you see and feel (apart from your hands that you
already know).

> If there was no basic structure to the brain, I think it would be almost
> impossible for an unsupervised agent to develop and learn to the complexity
> and level of a human brain.

I agree that there's some stuff built in, but I think it's surprisingly little
of it. How little I think we can see when we learn about people blind from
birth or with deformities. They still learn to operate their bodies as well as
it's physically possible.

Whatever person can relearn after physical brain damage I think can't be
built-in. I think the structure we see in the brain is result of built-ins +
various structural optimizations that make some stuff faster (or more energy
efficient) than if the structure was different.

For me the real trick in neural networks is to find out how exactly natural
neurons learn because it's not back-propagation and it's important. Do we know
that? In detail? How scratching yourself on the face as a baby translates to
chemical changes in synapses of neurons that fired recently?

------
flashman
If artificial intelligence is the cake, true AI is the ability to argue about
whether cake is a useful analogy.

~~~
StanislavPetrov
The ability to argue, or the capacity to reason that argument? If the AI can
convincingly argue that the analogy is apt, without actually reasoning or
"thinking", does that make it the holy grail? I'd suggest that self awareness,
and the ability to reason, would be true AI, and not just a glorified Turing
test in the form of an effective ability to pose an argument.

------
Sergej-Shegurin
(1) Adversarial learning is unsupervised and works great. Most of language
modeling is unsupervised (you predict next word, but it's not real supervision
because it's self-supervision). There're many works in computer vision which
are unsupervised and still give more or less reasonable performance. See f.e.
[http://arxiv.org/pdf/1511.05045v2.pdf](http://arxiv.org/pdf/1511.05045v2.pdf)
for unsupervised learning in action recognition, also
[http://arxiv.org/pdf/1511.06434v2.pdf](http://arxiv.org/pdf/1511.06434v2.pdf)
and [http://www.arxiv-sanity.com/search?q=unsupervised](http://www.arxiv-
sanity.com/search?q=unsupervised)

(2) ImageNet supervision gives you much information to solve other computer
vision tasks. So perhaps we don't need to learn everything in unsupervised
manner, we might learn most features relevant for most tasks using several
supervision tasks. It is kind of cheating but very reasonable one.

Moreover,

(3) We observe now just fantastic decrease of perplexity (btw, it's all
unsupervised = self-supervised). It's quite probable that in the very near
future neural chat bots write reasonable stories, answer intelligibly with
common sense, discuss things. All of this would be just a mere consequence of
low enough perplexity. If neural net says smth inconsistent it means that it
gives too much probability to some inappropriate words i.e, it's perplexity
isn't optimized yet.

(4) It's quite probable that it would open a finish line for human-level AI.
AI would be able to learn from textbooks, scientific articles, video lectures.
Btw,
[http://arxiv.org/pdf/1602.03218.pdf](http://arxiv.org/pdf/1602.03218.pdf)
gives a potential to synthesize IBM Watson with deep learning. May be, the
finish line to human level AI has been opened already.

------
fiatmoney
There's also a huge issue around problem-posing and degrees of freedom, that
doesn't necessarily get better as your AI tools improve. Go has a fairly large
state space, but limited potential moves per turn, well-defined decision
points, limited time constraints, and only one well-defined victory condition.
The complexity is minuscule compared to even something relatively well-
structured like "maximize risk-adjusted return via stock trades".

------
kailuowang
Can someone elaborate the difference between reinforcement learning and
unsupervised learning? It seems that I mistakenly think that human learns
through reinforcement learning, that we learn by the feedback from the outside
world. I mean without feedback from aldult can a baby even learn how to walk?

~~~
chrisfosterelli
Not an expert, but my understanding is that humans can learn many things
through reinforcement learning, but most of our intelligent decisions are a
result of unsupervised learning.

For instance, if the stove element was red hot and you touched it, you'd
receive a feeling of pain. Reinforcement learning would suggest that you
shouldn't do this again, and you might learn to not touch stove elements when
they are red hot anymore.

However, a human is likely to additionally realize that touching a red hot
marshmallow stick would burn them as well. And at the same time, a human can
also tell that a red ball is safe to touch. This sort of behaviour (internally
labeling things and deciding _what_ it is that made the stove element
dangerous so you can apply that to other things) would be unsupervised
learning.

~~~
markism
Interesting, so it's the ability to generalize attributes selectively? That
sounds like it would make for the difference between the specialized deep
learning we have and a more general intelligence.

Could that be accomplished if NN problems are broken into features, and those
features are individually tested against new information? Though you'd need a
layer for feature selection, and it still lacks the ability to pick features
without training.

~~~
chrisfosterelli
> Interesting, so it's the ability to generalize attributes selectively?

In technical terms it is the ability to generate its own features and
classifications for making decisions, where in supervised learning a human
provides the features and classifications.

> Could that be accomplished if NN problems are broken into features, and
> those features are individually tested against new information?

If I understood your example correctly, that would be an example of supervised
learning, as you pointed out it lacks the ability to pick features without
being told what they are. There are types of unsupervised learning that exist
and work quite well, for example cluster analysis [1].

[1]
[https://en.wikipedia.org/wiki/Cluster_analysis](https://en.wikipedia.org/wiki/Cluster_analysis)

------
Geekette
The statement that he's critiquing does reflect the wider-spread, overly
simplistic view of AI. Contrary to hype, recent events represent only partial
development/peeling of the top layer from the AI onion, which has more known
unknowns and unknown unknowns than known knowns.

------
LERobot
Totally agree, it's a bit like when some physicists were convinced that there
wouldn't be other great breakthroughs after Maxwell's theory of
electromagnetics. Maybe Yann LeCun is the Einstein of Machine Learning? haha

------
esfandia
It seems that AI does well when the problem and the performance metrics are
well defined: chess, Go, various scheduling problems, pattern recognition,
etc. At the very least we can track, quantitatively, how far off we are from a
satisfactory solution, and we know we can only ever get closer.

"True", or general-purpose AI, is harder to pin down, and thus harder to
define well. I'd argue that the moment we have define it formally (and thus
provided the relevant performance metrics) is the moment we have reduced it to
a specialized AI problem.

~~~
Houshalter
The Turing test is one such measure that is unbeatable by specialized AIs.
Language understanding in general requires some degree of intelligence.

I don't think the Turing test should be an actual goal of AI researchers.
Turing just proposed it as a hypothetical example.

------
megaman821
It seems to me one of the higher hurdles for creating a general purpose
intelligence, is human empathy. Without it you are left with creating a nearly
infinite-length rules engine.

When you ask your AI maid to vacuum your house, you would prefer it not to
plow through closet door to grab the vacuum, rip your battery out of your car
and hardwire to the vacuum, and then proceed to clean your carpets. If you
don't want to create a list of rules for every conceivable situation, the AI
will need to have some understanding human emotions and desires.

~~~
viraptor
I'm not sure why you connect that behaviour to empathy. There are two simple
rules here that apply to almost every possible situation, as well as the one
you presented. They aren't even connected to human emotions. That's pure
economy.

1\. Minimise work (plugging into socket has lower cost / effort than what you
described)

2\. Minimise irreversible changes (or cost of reversing them)

There are so many people with low empathy who are useful, I don't think this
is an issue until someone needs a personal companion rather than general
purpose AI.

~~~
megaman821
These rules cannot be applied in every situation. Then you back to writing a
bunch of rules. If you told a general AI to take care of a puppy for a few
days, it would end up putting a diaper on it and keeping it in a crate 24
hours a day. That would be the least amount of work to take care of the puppy,
and minimize the chance of the puppy hurting itself or damaging anything else.

~~~
viraptor
This is close to what happens when you leave your pet at a cheap pet hotel
though. General purpose humans do it the same way.

------
rdlecler1
We keep trying to engineer AI rather than reverse engineering it. The thing
with living organisms is that the neural network underlying the intelligence
of living organisms is a product of evolutionary design of an organism
situated in the real physical world with laws of physics and space and time.
This is where the bootstrapping comes in. Unsupervised learning is built on
top of this. Trying to sidestep this could prove difficult to get to General
AI.

------
jonbarker
To be fair AlphaGo never decided Go was a fun and worthy challenge; Lee Sedol,
however, did.

------
johanneskanybal
Click bait titles aside it's an amazing achievement.

------
kafkaesq
A beautifully concise statement of an incredibly common misconception as to
the current state of the field.

------
_snydly
I have Facebook blocked for the next week (because, you know, productivity).
Can someone post LeCun's comment here?

~~~
sound_of_basker
Statement from a Slashdot post about the AlphaGo victory: "We know now that we
don't need any big new breakthroughs to get to true AI" That is completely,
utterly, ridiculously wrong. As I've said in previous statements: most of
human and animal learning is unsupervised learning. If intelligence was a
cake, unsupervised learning would be the cake, supervised learning would be
the icing on the cake, and reinforcement learning would be the cherry on the
cake. We know how to make the icing and the cherry, but we don't know how to
make the cake. We need to solve the unsupervised learning problem before we
can even think of getting to true AI. And that's just an obstacle we know
about. What about all the ones we don't know about?

------
chriscappuccio
Like, duh.

------
juskrey
True life has no rules

------
ronilan
The cake is a lie. Obviously :)

~~~
mkoryak
reddit is over there ->

------
justsaysmthng
Stop looking at the red dot. Take a step back and look around you. "True" AI
is here and it's been here for some time. You're communicating with it right
now.

It's just that we find it so hard to comprehend it's form of "intelligence",
because we're expecting true AI to be a super-smart super-rational humanoid
being from sci-fi novels.

But what would a super-smart super rational being worth 1 billion minds
look/feel like to one human being ? How would you communicate with it ?

Many people childishly believe that "we" have control over "it". You don't. We
don't.

The more we get used to it being inside our minds, the harder it becomes to
shut it down without provoking total chaos in our society. Even with the
chaos, there is no one person (or group) who can shut it down.

But "we" make the machines ! Well... yes, a little bit..

Would we be able to build this advanced hardware without computers ? Doesn't
this look like machines reproducing themselves with a little bit of help from
"us" ?

Think about the human beings from the Internet's perspective - what are we for
it ? Nodes in a graph. In brain terms - we are neurons, while "it" is the
brain.

But it's not self-aware ! What does that even mean ?

Finally, consider that AlphaGo would have been impossible without the Internet
and the hardware of today.

And that "true" AI that everybody expects somewhere on the horizon will also
be impossible without the technology that we have today.

If so, then what we have _right now_ is the incipient version of what we'll
have tomorrow - that "true" AI won't come out of thin air, it will _evolve_
out of what we have right now.

Just another way of saying the same thing - it's here.

Is this good or bad ? Well, that's a totally different discussion.

------
daxfohl
Once an AI algorithm (even just one for Go) realizes that it can hijack the
bank accounts of all the world's other 9 dan players in order to demand an
analysis of its planned move, and figures out how to do that, _then_ we've
made the cake.

N.B. the genericity of the deepmind stuff that is the basis of AlphaGo makes
this seem not entirely far-fetched.

Yum, cake.

