
AlphaGo shows its true strength in 3rd victory against Lee Sedol - luu
https://gogameguru.com/alphago-shows-true-strength-3rd-victory-lee-sedol/
======
sdenton4
It would be interesting to know AlphaGo's estimated probability of winning as
the game progressed; presumably we can see directly how worried it was at any
given point in the game. And thus get another sense of whether it ever really
felt threatened by Sedol.

At this point, the match is won, but games 4 and 5 will commence. The question
shifts from whether AlphaGo is better than humanity's best, to whether
humanity can even have a chance of beating AlphaGo in a single game. And so
far, it sounds like the answer is likely to be a resounding no.

~~~
teraflop
This isn't exactly what you were asking for, but here are some graphs from a
_different_ Go AI (Facebook's "darkforest") showing its estimated win
probabilities:
[http://zhuanlan.zhihu.com/yuandong/20639694](http://zhuanlan.zhihu.com/yuandong/20639694)

~~~
EGreg
Can anyone with Go knowledge comment on this?

~~~
andrew-lucker
First game, AlphaGo looked ambiguously weak until end of mid-game when it
suddenly looked very strong.

Second game, AlphaGo looked obviously strong throughout.

So the probabilities are maybe not entirely wrong.

For reference I am 4k rating, somewhat below darkforest(1d amateur) and so
very very far from 9dan pros that AlphaGo is beating. Usually, pro matches are
hard to understand but the excellent commentary and abundant community
analysis helps greatly.

------
atrudeau
I felt that Lee thought global sometimes, especially when he had time on the
clock, but his mind could only take so much thinking, and time was catching
up, so for a great many number of moves he thought locally.

AlphaGo doesn't tire and thinks globally on every move.

We saw that time and time again. Lee playing locally and AlphaGo surprising
everyone with an unexpected move somewhere else on the board.

Obviously it is better to be globally optimal than locally optimal.

~~~
highwind
Doesn't principle of optimality state that if a solution is optimal than any
of its sub problem must be optimal as well?

~~~
wickawic
Instead of downvoting this comment, let me provide a counter example. Imagine
the knapsack problem, where you can lift 10kg and you have a 10kg gold bar and
5kg cinderblocks. In this case solving 2 5kg knapsack problems will give you
two cinderblocks, which is obviously not optimal.

I don't know a lot about the strategy of go, but it seems to me that any play
that doesn't take into account the entire state of the board is allowing for
the same class of sub-optimal behavior as the example above.

~~~
felixgallo
the guy you're responding to is correct:

[https://en.wikipedia.org/wiki/Bellman_equation#Bellman.27s_P...](https://en.wikipedia.org/wiki/Bellman_equation#Bellman.27s_Principle_of_Optimality)

and your attempted counterexample makes no sense (as the 'two cinderblocks'
solution is not a subproblem of the optimal solution).

~~~
lorenzhs
From the very section of the article you quoted: "In computer science, _a
problem that can be broken apart like this_ is said to have optimal
substructure." \- There is no reason whatsoever that _any_ problem could be
broken down to this. In fact there are many problems for which it is known to
be impossible. The article on optimal substructure lists a few.

~~~
daveguy
Yes. The principle of suboptimality only applies to problems which are
optimally solved by dynamic programming. This is a small subset of problems
and the knapsack problem is not one of those problems solved by dynamic
programming. It does not have optimal substructure.

------
partycoder
Skeptics have said that achieving this milestone would not happen within this
decade, our lifetimes or even ever. It happened yesterday.

It took Lee Sedol many decades of his life to train to achieve this level. And
his ability to pass on his skills is limited. Now that a computer has achieved
this level, the state of the neural network behind it can be serialized and
ran into an unlimited number of computers and have millions of systems that
are more proficient at Go than the best player in the world.

People have said that achieving the cognitive level of the human brain
requires to match its computational power. But if you take out all the
parasympathetic and motor boilerplate, what is actually left for mental tasks
is much less from that, most of that power is not even recruited for higher
level mental tasks. That lowers the bar for strong AI.

Then, strong AI can be immortal, and never physically deteriorate from aging.
Strong AI can multiply infinitely and communicate at a rate that would be
equivalent to writing millions of books in a second. It could transfer all its
knowledge in seconds. It can also recursively improve itself. This advantage
will lower the bar for strong AI even more.

~~~
return0
On the other hand i would like to read an assessment of the significance of
this advance. While AI may have perfect applicability to these perfect
information games, we may still be decades from useful real world generic
applications.

~~~
aerovistae
Exactly so. While this is a major advance, I roll my eyes at the sea of
comments hailing the arrival of a superior intelligence.

When the AI is cognizant of the fact that Go is a game, and knows what a game
is, and perceives that the game is taking place in a larger reality, where
there are other things happening while the game is going on....then I'll be
impressed.

~~~
danbmil99
> ...then I'll be impressed.

Or maybe not. The history of AI is one of humans always moving the goalposts
when AI advances.

Chess? just a computationally simple game. Driving? Well it's just physics.
Go? _now that requires intelligence_ oh wait, just some deep neural savant
thing, not _real_ AI

~~~
aerovistae
Well, I think what I'm describing (general awareness) is a little different
from your examples there, but I acknowledge my choice of words implied I'm not
impressed, which I completely am because this is obviously utterly amazing.
It's just not AS big a deal as a lot of people are making it out to be with
proclamations of apocalypse.

------
andrewflnr
What I found especially chilling was how the AI took its time when it thought
it was ahead, and played exactly as aggressively as needed to win. It seems
like we're seeing the full benefit of detaching logic from human emotional
drives like pride, loss of nerve, etc. Which at first sight looks awesome, but
in a way it's deeply terrifying, because if such a machine is ever given power
in the real world, we would really like it to have some regard for human
emotion. More than anything else I've seen, this makes me feel in my gut like
I'm seeing my replacement.

~~~
jtolmar
On the perhaps more reassuring side, it's possible for there to be an AI that
takes into account human emotion, but does not have any of its own. Given
power in the real world, it would do a better job of making people happy than
a human with all their biases. It wouldn't assume that other people must like
things it like, for example.

(Of course, making an AI that understands emotion isn't easy. And happiness by
itself is an insufficient goal if we're building ourselves a benevolent
overlord.)

~~~
armitron
I view all such arguments about "friendly AI", "emotion", having machine
intelligence "understand us" as wishful thinking at best and laughable
delusions otherwise.

I think the writing on the wall is clear, to anyone who cares to take a look.
The moment we create an artificial intelligence capable of self-improvement
and let it loose, we will have fulfilled our function (others say destiny) and
will therefore be obsolete in every sense of the word.

What will happen to humanity after that point is irrelevant.

You don't see humans trying to keep bacteria "in the loop", why expect
otherwise from our artificial progeny?

~~~
jtolmar
You're anthropomorphising the machine. An AI will do what it's programmed to
do, not develop free will and decide on its own destiny.

Correctly programming something that takes into account all of the nuances of
the human condition might be impossible, and the results of mistakes are
uniformly terrifying, but the result is entirely in the hands of humans (and
human mistakes).

~~~
tim333
Unless someone programs one to develop free will and decide on its own
destiny. Which someone will no doubt, when that becomes possible.

------
guelo
I don't understand how this is considered fair. AlphaGo has been trained on a
database that includes every recorded game Sedol has ever played while Sedol
is seeing AlphaGo's play style for the first time. Sedol should have been
allowed to play against AlphaGo for a few months before the match so he could
study its style.

~~~
plank
So what? I could watch every game Roger Federer played in any tennis
tournament and still lose all sets to love. It used to be that computers could
only do combinatorics better, but that where 'intuition' played a strong part,
there was still hope for us humans... Well, guess I will have to start playing
Calvinball...

~~~
fma
There's a difference between having skills and having knowledge.

In your case you have knowledge but lack skills.

When Federer was losing to Nadal he changed his practice and game to counter
Nadal's style. He took knowledge and applied it to his skills.

When Federer came out with SABR people were like wtf. Now they know of it and
put more umph on their second serve. See. Knowledge + skills

------
eagsalazar2
I'm curious if AlphaGo is simply winning by virtue of having more
computational capability (and therefore can never be defeated consistently by
humans) or if in its training it actually discovered and is now deploying
interesting new tactics that humans, studying these games, can uncover and
therefore use to defeat AlphaGo in the future.

~~~
hodwik
Sedol said that it was doing tactics he had never seen before -- like the now
infamous move 37 that caused Sedol to get up and walk out of the room.

~~~
cjbprime
Apocryphal, but romantic -- he left the room ten seconds earlier.

~~~
hodwik
Is that right? From the Google stream you couldn't actually see him leave,
that's just what the commentators said.

~~~
cjbprime
Yeah, reddit /r/baduk had a thread about it including video from a different
angle. The commentator only noticed that the move had been played and he was
gone, didn't get the ordering correct.

------
greenyoda
The main discussion of the third game can be found here:
[https://news.ycombinator.com/item?id=11271816](https://news.ycombinator.com/item?id=11271816)

However, this article contains much more analysis of the game than previously
posted articles, and is worth reading.

------
the_af
I understand very little of Go beyond its basic rules and having played only a
couple of matches with a friend who knows as little as I do...

I reviewed the match, and it seems that very quickly White (AlphaGo) went
aggressive, and Black was trying to contain it. I suppose a human player can
commit serious mistakes by playing too aggressively from the start, right? But
the summary of the game speculates move 31 might have been the losing move
(maybe implying the game until that moment wasn't going so bad for Lee
Sedol?), while to my untrained eyes it looks as if White was constantly on the
offensive from the start, and Black was playing almost exclusively to contain
it.

Or am I reading this wrong?

~~~
autarch
I think that most high-level Go players (including AlphaGo) play moves that
serve multiple purposes simultaneously. The ideal move reinforces your own
position while forcing your opponent to defend their own.

White certainly played plenty of moves that required black to respond, but
black also did the same. For example, the sequence beginning at 77 is black
getting a foothold inside an area that white might've hoped to claim later.
Then at move 115 black attempts an invasion into a very strong white area,
ultimately failing to escape or live, leading to resignation.

That all said, I wouldn't say that either player was particularly aggressive.
I've watched plenty of human games where one player or the other very
intentionally picks an all-out fight. If you're really good at reading ahead
locally and you think your opponent is not, this make sense. This is
especially true if your large-scale game play is not as good as your tactical
play.

This is also common in high handicap games. If white can't slowly eke out an
advantage early on, the only remaining tactic might be to go for a huge fight
and see what happens.

------
mchahn
It was fascinating when the author experienced a weird bit of nausea when
thinking about the implications of AI. I'm not sure how I feel about it yet.
It does scare me a bit.

~~~
agumonkey
Think about that we're constantly pushing toward our own replacement.

~~~
david-given
Don't we normally call that _having children_?

~~~
agumonkey
How much of humanity genes will skynet inherit ?

~~~
hodwik
Not genes, memes.

~~~
agumonkey
memetics

------
raverbashing
This is different from DeepBlue and Garry Kasparov

Chess may have a big search space, but it is "well behaved". Moves might be
out of the ordinary, but not too much (weird moves that still help to win the
match are rare)

AlphaGo might also have learned from past matches, but that doesn't give all
the answers

The black-box aspect, the fact that you can't understand what it is thinking
is curious, to say the least

~~~
asdfologist
I don't know if that's true. "Computer moves", referring to weird moves that
computers play that often turn out to be very good, tend to get mentioned a
lot in chess. And you might hear about them less and less, because many of
them have been absorbed into the human chess repertoire.

~~~
jdietrich
Also notable is the verb "spacebar", meaning "to play the same move an AI
would". The chess world provides an interesting microcosm of how humans adapt
to a world dominated by AI.

------
smitherfield
I just thought of an interesting (and somewhat more "real-world") task for AI
research: Can an AI outperform a human at play-calling in [American] football?

~~~
rcthompson
Considering that plays are often called or changed by quarterbacks on the
field right before the snap based on what the QB sees from the opposing team,
I think it would be difficult to provide all that input to a computer AI.

~~~
jonknee
A neural net would be able to read the opposing defense more quickly and
accurately than any human, but it would not be a fair fight (the human doesn't
get a bird's eye view of things!).

~~~
tolas
It just doesn't have as big of an impact though. Each team has a very limited
set of plays it's actually able to call and execute. Maybe 300 max (as a
guess). Many times much less than that. So there's really not much room for an
AI to see huge improvements via play calling.

It's really all about execution on the field and the "feel" of the game.

------
lilcarlyung
Well, at least we still got poker. Until AI knows how bluff or how to call out
bluffers.

~~~
VikingCoder
Hmmm...

[http://www.latimes.com/nation/great-reads/la-
na-c1-claudico-...](http://www.latimes.com/nation/great-reads/la-
na-c1-claudico-poker-20150521-story.html)

[http://spectrum.ieee.org/tech-
talk/computing/software/comput...](http://spectrum.ieee.org/tech-
talk/computing/software/computer-battles-top-human-poker-players)

[http://www.npr.org/sections/alltechconsidered/2015/01/08/375...](http://www.npr.org/sections/alltechconsidered/2015/01/08/375736513/look-
out-this-poker-playing-computer-is-unbeatable)

------
clickok
How strong is AlphaGo?

DeepMind doesn't generally comment on these sorts of things, but I think they
knew it would be strong, just not how strong.

It is hard to accurately gauge the playing strength of such a program for the
following reasons:

* Go is not solved, it doesn't really even admit a good heuristic for close games. In chess, given enough time, we can do a pretty good job of searching the possible outcomes from a given position, and even if we can't evaluate all the possible endgames, evaluation functions exist that give us an idea of which side is ahead when we terminate the search. Go has a much higher branching factor (more moves available at each turn, making the search more expensive) and short of a catastrophic blunder, it's hard to quantify who is ahead at each point in time[1]. So we can't (in general) quantify optimal play, and therefore cannot quantify how much AlphaGo (or anyone else) deviates from it.

* One known aspect of programs using Monte Carlo Tree Search is that they play to win, and are willing to sacrifice margin of victory to maintain or increase their odds of winning. According to some people I've talked to, this can be suboptimal, but there are methods of addressing this[2]. Note that you can't just change the objective function from "winning" to "win by as much as possible" without potentially reducing AlphaGo's strength.

* The value function learned by a deep net is hard to interpret, partly because it encodes information about the possible futures arising from a position, and partly because it involves a tremendous number of calculations to compute. We do not know what it is representing at the intermediate levels-- techniques that aim to visualize or cluster unit activations can provide a bit of insight, but there's always the possibility that we're interpreting the patterns incorrectly because we're trying to fit it into the framework of "what would a human think?". Further, the representation is somewhat monolithic-- we can't tweak the value of one thing without changing the values of others. In chess engines, we might modify the material value of, say, a knight, without affecting the value of a rook. In a convolutional net, if we adjust the value of one position, it will tend to affect the value of many others.

* We can attempt to quantify its strength by comparing it to other programs, but AlphaGo has already crushed other programs (99% win rate), so all that it tells us is that it is stronger than those programs.

Essentially, without the ability to perform a significant amount of searching,
or gauge strength via margin-of-victory, or to examine the program from other
angles, it's hard to gauge just how good the program is.

The only thing we can do is throw skilful opponents at it, and see if it fails
eventually[3]. In its current incarnation, though, it seems like its playing
strength is just going to be "stronger than you".

\---

1\. Hence the need for Deep Reinforcement Learning-- we learn how valuable
each position is based on the results of the positions that it can lead to.

2\. By modifying the search to maintain a given margin once it has been
attained-- but I do not work at DeepMind, nor am I an expert on MCTS like some
of my colleagues, so I don't know if this could adversely affect AlphaGo's
overall strength.

3\. We might be able to get a better idea by having extremely strong players
play against weaker versions of AlphaGo (ones with less computing power) and
then sort of telescope upwards once a good baseline has been set, but it
remains to be seen if the current level of invincibility is due to it not
having been around long enough for its weaknesses to become apparent[4].

4\. How would an AI researcher play against it? I've talked to a few people,
and the answers have been: (a) play the game to the conclusion, don't resign;
(b) attempt to take the game off-the-rails into parts of the state-space it
hasn't really explored before (although this is risky because the human player
is more likely to make mistakes in these cases as well), and (c) let it get
ahead somewhat so that it "relaxes" (see the points about margin), allowing
you to catch up and overtake it.

~~~
cottonseed
The way to find out how strong AlphaGo is is to start giving handicap and see
when the game becomes even. A difference in rank of 1 stone corresponds
roughly to a 2:1 advantage. It is traditional in long-running matches to
increase the handicap by 1 after one player wins three consecutive games. I
haven't heard any talk of that here -- I'm sure Lee Sedol wants the chance to
beat AlphaGo in an even game -- but it would be interesting. It would also be
interesting to see how AG plays when it thinks it is behind.

~~~
clickok
That might work; the concern is that this would take it too far off of the
task it was trained on. That is, if it doesn't have a lot of experience being
significantly down, then it won't play nearly as well when trying to catch
up-- but that doesn't matter in even games because it never gets that far
behind.

You're right that it would be interesting to see, though-- we need to get
better at understanding these sorts of systems, at least until they can start
optimizing themselves.

Alternatively, we might train a different agent (OmegaGo?) to try to win by
the largest margin possible-- if it works as well as AlphaGo, then that might
give us some more insight into how strong both programs are.

------
Sir_Cmpwn
What are some good resources to learn Go?

~~~
dyoo1979
There's a good interactive tutorial at
[http://playgo.to/iwtg/en/](http://playgo.to/iwtg/en/)

The community has also collected a lot of good Go resources at Sensei's
Library: [http://senseis.xmp.net/](http://senseis.xmp.net/)

Good luck!

~~~
Sir_Cmpwn
Thanks!

------
mingodad
Out of curiosity how much electric power is alphago burning while playing ?
Does someone know it ?

~~~
hutzlibu
Much more than then "stupid" human used, I guess. So for now, we are at least
much more energy-efficient ...

~~~
tim333
Apparently 1920 cpu's so maybe 200 kW?

------
botw
The actual match is largely effected by psycology at the time of game. The
match is somewhat unfair to human player. AlphaGo and its team know all about
each human player and preference. But human player knows little about AlphaGo.
Also it is AlphaGo team who decides whom to choose to play against AlphaGo,
not the other round. In the sense of psycology and , the game is greatly favor
to AlphaGo right now. This is the difference with the match between IBM
DeepBlue and the chess world champion 10 years ago. It should be that any
player(human or AI) can be allowed to enter the game, with or without a fee.

------
naveen99
Seems like just yesterday (actually seven months ago), I was saying dbn's
aren't good at go yet in an argument:
[https://news.ycombinator.com/item?id=10028908](https://news.ycombinator.com/item?id=10028908)

i guess it's time to play with dbn's again. At least I have accumulated a few
more Titan x cards in the mean time.

------
petegrif
Holy crap. It's a total beatdown.

~~~
jacquesm
So far, but yes, it looks bleak (or good, depending on which side you're
rooting for).

------
71817188
Did Lee Sedol have a reasonable idea of AlphaGo's recent strength? To put it
more harshly, did DeepMind misrepresent the true strength of the program?

I find it very hard to find information about this online.

~~~
lukaslalinsky
They challenged a top player for $1M. Doesn't that imply they think it's
stronger than any human player?

~~~
wolfgke
This can also imply that for Google it is worth $1M to know where AlphaGo
still has weaknesses.

~~~
iainmerrick
Don't be silly, a Go AI has very little direct value for Google's business.
But the publicity is great, easily worth $1M. More important is keeping the
DeepMind team busy and engaged -- the real value is in spinning that know-how
off into other projects.

~~~
wolfgke
> Don't be silly, a Go AI has very little direct value for Google's business.

On the other hand: Knowing what makes some problems much easier for humans
than for computers can have direct value on Google's business. Go was a hot
candidate for a game with this property.

------
hyperpallium
Huh, Go doesn't require intelligence after all.

------
131hn
Can alphago be defeated ?

~~~
iainmerrick
Deep Blue can be defeated at chess now.

------
bobbles
So what happens if we play AlphaGo vs AlphaGo?

~~~
tim333
They did loads of that in the cloud during training.

------
nzonbi
This is awesome, finally AlphaGo has beaten Lee Sedol. Congratulations to the
team at DeepMind and Google. But I will go straight to the really interesting
point: My completely baseless intuition tells me that we will have full AGI
(artificial general intelligence) in 5-10 years. Maybe not very powerful at
first, but a least working. Real general intelligence. I say this as a
programmer with superficial knowledge on ML (machine learning). All that is
needed is to figure a way, to make neural networks handle the flow of thoughts
of human consciousness. Reasoning, short/long term memory, basic controlling
"emotions", etc. Then to assemble this central thought unit, in a system with
various neural networks to handle more specialized tasks: Vision, natural
language processing, audition, motion,etc. The task is of course extremely
difficult, but I have the feeling that it will be possible to overcome.

Once this have been achieved, it is downhill from there. The singularity. In
my opinion, we must start to consider how this tremendous breakthrough will
affect the human race. Our lives, economy, culture, etc. Awesome things we
will be witnessing in the coming times. It would be great if it were possible
to make public, the progress made researching specifically AGI. I wonder how
big of an effort is currently being done. This would be possibly the biggest
discovery ever for humanity. It deserves a fantastic effort.

~~~
bgar
I don't know if you're serious, but how can you make such grand predictions,
considering you yourself admit to having only superficial knowledge of machine
learning?

~~~
nzonbi
It is just a guess. Human thought, although highly complex, is in my opinion
something actually feasible to be simulated. If I had a large research budget
under my control, I would be betting most of it on AGI.

