
AlphaGo beats Lee Sedol again in match 2 of 5 - pzs
https://gogameguru.com/alphago-2/
======
fhe
As someone who studied AI in college and am a reasonably good amateur player,
I have been following the matches between Lee and AlphaGo.

AlphaGo plays some unusual moves that go clearly against any classically
trained Go players. Moves that simply don't quite fit into the current
theories of Go playing, and the world's top players are struggling to explain
what's the purpose/strategy behind them.

I've been giving it some thought. When I was learning to play Go as a teenager
in China, I followed a fairly standard, classical learning path. First I
learned the rules, then progressively I learn the more abstract theories and
tactics. Many of these theories, as I see them now, draw analogies from the
physical world, and are used as tools to hide the underlying complexity
(chunking), and enable the players to think at a higher level.

For example, we're taught of considering connected stones as one unit, and
give this one unit attributes like dead, alive, strong, weak, projecting
influence in the surrounding areas. In other words, much like a standalone
army unit.

These abstractions all made a lot of sense, and feels natural, and certainly
helps game play -- no player can consider the dozens (sometimes over 100)
stones all as individuals and come up with a coherent game play. Chunking is
such a natural and useful way of thinking.

But watching AlphaGo, I am not sure that's how it thinks of the game. Maybe it
simply doesn't do chunking at all, or maybe it does chunking its own way, not
influenced by the physical world as we humans invariably do. AlphaGo's moves
are sometimes strange, and couldn't be explained by the way humans chunk the
game.

It's both exciting and eerie. It's like another intelligent species opening up
a new way of looking at the world (at least for this very specific domain).
and much to our surprise, it's a new way that's more powerful than ours.

~~~
forgotpwtomain
> It's both exciting and eerie. It's like another intelligent species opening
> up a new way of looking at the world (at least for this very specific
> domain). and much to our surprise, it's a new way that's more powerful than
> ours.

I have been watching Myungwan Kim's commentary for the games - and it seems
notable that a few moves he finds very peculiar immediately when they are
made, he will later point out to as achieving very good results some 20 moves
later. So it also seems quite possible that AlphaGo is actually reading this
far ahead, to find those peculiar moves achieve better results than from the
more standard approaches.

Whether these constitute a 'new way' or not I think depends highly on whether
these kind of moves can fit into some general heuristics useful for
considering positions, or whether the ability to make them is limited to
intelligence's with extremely high computational power for reading ahead.

~~~
lpage
> _he will later point out to as achieving very good results some 20 moves
> later_

This. It's a fairly common feature of any AI that uses some form of tree
search/minimax, and the effect is very pronounced in chess. Even the best
human players can only think 6-8 plies into the feature versus ~18 for a
computer. What we can (could?) do is apply smarter evaluation functions to the
board states resulting from candidate plays and stop considering moves that
look problematic earlier in the search (game tree pruning). AI tends to use
very simple evaluation functions that can be computed quickly. They do so
given that 1) it allows for deeper search, and a weak heuristic evaluated far
in the future often beats a strong one evaluated a few plies prior and 2) for
some games (like Go) it's really hard to codify the "intuitions" that human
players speak of.

Because search based AI considers board states __very__ far in the future, the
results are often completely counterintuitive in a game with an established
theory of play. Those theories are born of humans, for humans.

The introduction of MCTS some years back was the first leap towards a human
level Go AI (incidentally, MCTS is more human-like than exhaustive tree search
in that it prunes aggressively by making early judgement calls as to what
merits further consideration). AlphaGo's use of deep policy and evaluation
networks to score the board is very cool, and the next step in that journey.
What's interesting to me is that, unlike chess AI, AlphaGo might actually
advance the human theory of Go. It's possible that these "strange moves" will
lead to some very interesting insights if DeepMind traces them through the
eval and policy networks and manages to back out a more general theory of
play.

~~~
dragontamer
On the contrary.

I think that Chess machines play perfectly for the next 8 moves, but don't
necessarily sense the importance of a Knight Outpost (which may have relevance
20 moves ahead. A proper Knight Outpost will remain a fork threat for the rest
of the game).

It is far easier for a Human to beat a Chess Machine at positional play (ex: a
backwards pawn shape will probably be a problem at endgame, 30+ moves from
now) than to beat a Chess Machine at tactical play (3 moves from now, I can
force a fork between two minor pieces)

~~~
rwill128
This was true 10-15 years ago. It is no longer true. Chess engines have
positional evaluation algorithms that have been trained using many millions of
games, and the weighting parameters for different kinds of positional features
have been adjusted accordingly.

Do some reading on Stockfish for example if you doubt the veracity of my
statement.

~~~
dragontamer
Yes, I do realize that.

But its just as you say: its weighting parameters and heuristics. When
Stockfish recognizes a backwards pawn, it deducts a point value. When
Stockfish recognizes "pawn on 6th row", it adds a point value to that pawn.

But that's a heuristic. A trained heuristic using games, but still comes down
to what I understand to be a +/\- point value (like... +35 centipawns).

In contrast, a chess engine truly knows that if you do X move, it will force a
Rook / Minor piece exchange in 8 moves.

When you play positionally vs Stockfish, you're arguing with a heuristic (a
heuristic which has been refined over many cycles of machine learning, but a
heuristic nonetheless that comes down to "+/\- centipawns") . When you play
tactically vs Stockfish, it is evaluating positions more than a dozen moves
ahead of what is humanly possible.

When you play against Stockfish in endgame tablebase mode, it plays utterly,
and provably, perfectly.

Take a pick of what game you want to play against it. IMO, I'd bet on its
positional "weakness" (yes, it is still very strong at positional play, but it
is the most "heuristical" part of the engine)

------
Cookingboy
Someone somewhere asked why a lot of people in the Go community is taking this
in a somewhat hard way, here is my hypothesis:

Go, unlike Chess, has deep mytho attached to it. Throughout the history of
many Asian countries it's seen as the ultimate abstract strategy game that
deeply relies on players' intuition, personality, worldview. The best players
are not described as "smart", they are described as "wise". I think there is
even an ancient story about an entire diplomatic exchange being brokered over
a single Go game.

Throughout history, Go has become more than just a board game, it has become a
medium where the sagacious ones use to reflect their world views, discuss
their philosophy, and communicate their beliefs.

So instead of a logic game, it's almost seen and treated as an art form. And
now an AI without emotion, philosophy or personality just comes in and brushes
all of that aside and turns Go into a simple game of mathematics. It's a
little hard to accept for some people.

Now imagine the winning author of the next Hugo Award turns out to be an AI,
how unsettling would that be.

~~~
gsklee
On a more realistic side note... Professional Go players devote decades in
training ever since their youth, giving up normal educations and lots of other
more lucrative opportunities for their lives. It's very easy to imagine their
frustrations now that their life-time devotion actually means nothing in front
of the AI.

It's an upright denial to the way of life they so chose and devoted.

IMHO Google should donate the prize towards Go education and Go organizations
instead of some random charities.

~~~
VladKovac
Isn't this a good thing? Why are high IQ people devoting their entire lives to
a game? Maybe this will make them shift their priorities to solving problems
that only really smart humans (like them) can solve.

~~~
cli
At the root of it, they earn a living by being entertainment. This can be
applied to any of the arts or sports. Why are smart people making movies,
writing fiction, making music? I think these are the sorts of things that make
life worth living.

------
davelondon
Let's compare Go and Chess. We all know that Go is more complex that Chess,
but how much more?

There's 10^50 atoms in the planet Earth. That's a lot.

Let's put a chess board in each of them. We'll count each possible permutation
of each of the chess boards as a separate position. That's a lot, right?
There's 10^50 atoms, and 10^40 positions in each chess board so that gives us
10^90 total positions.

That's a lot of positions, but we're not quite there yet.

What we do now is we shrink this planet Earth full of chess board atoms down
to the size of an atom itself, and make a whole universe out of these atoms.

So each atom in the universe is a planet Earth, and each atom in this planet
Earth is a separate chess board. There's 10^80 atoms in the universe, and
10^90 positions in each of these atoms.

That makes 10^170 positions in total, which is the same as a single Go board.

Chess positions: 10^40
([https://en.wikipedia.org/wiki/Shannon_number](https://en.wikipedia.org/wiki/Shannon_number))
Go positions: 10^170
([https://en.wikipedia.org/wiki/Go_and_mathematics](https://en.wikipedia.org/wiki/Go_and_mathematics))
Atoms in the universe: 10^80
([https://en.wikipedia.org/wiki/Observable_universe#Matter_con...](https://en.wikipedia.org/wiki/Observable_universe#Matter_content))
Atoms in the world: 10^50
([http://education.jlab.org/qa/mathatom_05.html](http://education.jlab.org/qa/mathatom_05.html))

~~~
pavelrub
This doesn't seem to be the main reason why Go is harder than chess for
computers. It was noted that even in 9x9 Go, with a comparable branching
factor to Chess, traditional Go programs are still no stronger than on big
boards. The main difficulty for Go is that it is much harder to evaluate board
positions. So in Chess the depth of the search can be significantly reduced by
using a reasonable evaluation function, whereas in Go no such function seems
to exist.

~~~
liviu-
>It was noted that even in 9x9 Go, with a comparable branching factor to
Chess, traditional Go programs are still no stronger than on big boards.

Are they not? MoGo beat pros of 9 Dan on 9x9 in 2011:
[https://www.lri.fr/~teytaud/mogo.html](https://www.lri.fr/~teytaud/mogo.html)

~~~
pavelrub
Well, I guess it was more true before the advent of Monte Carlo Tree Search.
Even so, note that even in the case of MoGoTW in 2011, it played _blind_ Go
(this helps the computer), and out of 4 games, won two games against a 9p
player, and lost 1 game to a 5p player. Though it is perhaps better than
MoGo's performance on 19x19, it still isn't very good, doesn't seem much
better than MoGo on 13x13, and performs much worse than computer Chess,
despite a similar branching factor.

~~~
iopq
The branching factor is much larger, around 75 legal moves after the opening,
while chess has at most like 30.

Fuego beat a pro in 2008 using MCTS actually.

~~~
pavelrub
The branching factor of 9x9 Go isn't 75. 75 could be the factor in early game,
but the average factor is somewhere between 40 and 50, versus 35 in chess.
State-space complexity is also considerably higher in Chess than in 9x9 Go.

Not sure what you meant regarding MCTS, I never said anything about MCTS not
being able to beat pros.

------
mixedmath
This game was largely played extremely well by both sides. There were a a few
peculiar-seeming moves made by AlphaGo that the commentator found very
atypical. These moves ended up playing a very important role in the end game.

I should also say that it's somewhat clear that Sedol made one suboptimal
move, and AlphaGo capitalized on it. Interestingly, the English commentator
made the same mistake as he was predicting lines of play. This involved play
in the center of the board, in a very complicated position. Prior to this set
of moves, the game was almost a tie. Afterwards, it was very heavily in
AlphaGo's favor.

~~~
ljk
> _These moves ended up playing a very important role in the end game._

Does this mean AlphaGo was playing at a higher level or is it just a
coincident?

~~~
thomasahle
The 9p commentator at the AGA channel said you should judge genius versus
madness on whether alphago won. And it did.

~~~
zamalek
The "mad" moves could also be throwing the human off. I wonder how Sedol would
fare given a year or so of practice against AlphaGo.

~~~
piyush_soni
Just that in that time AlphaGo will have played millions of more matches
against itself, learning at a much faster rate than him. Not sure, he might
still beat the machine though. That needs to be seen.

~~~
scarmig
AlphaGo has already played millions of matches against itself. The obvious low
hanging fruit there has already been harvested.

------
skc
I find it very interesting that to a layperson, the idea of a computer being
able to beat a human at a logic game is pretty much expected and
uninteresting.

You try and share this story with a non-technical person and they will likely
say "Well, duh..it's a computer".

~~~
mmahemoff
Most people operate on a moving definition of intelligence as "whatever humans
can do and machines can't" (thus ruling out AI by definition).

If software started writing bestselling novels, it would soon become a "duh
that's just what computers do" matter of fact.

~~~
Certhas
That's why task based notions for general intelligence are rubbish.

Here is what humans can do: When presented with pretty much any task,
specified however poorly, they can design hardware and algorithms that can
beat themselve at that task.

That's a decent measure of intelligence. A decent measure of creativity is
coming up with tasks that make the intelligence part as interesting as
possible.

~~~
irascible
Optimizing compilers do this already and they aren't 'intelligent'. Iirc alpha
go is based on the same algo that could beat all atari games sight unseen..
satisfying the 'novel situation' requirement you set forth..

------
rurban
What I really liked about those games so far, and Michael Redmond commentary,
is that AlphaGo not only beat Lee Sedol twice, but also Redmond. He is playing
the same style as Sedol, he constantly predicts Sedol's moves and he is as
surprised and does the same miscalculations as Sedol. He really needs some
time to find out when he made a mistake, the same mistake Sedol was eventually
doing. This is high class commentary. Even if they have no Computer screen to
clear the screen after some variations. He remembers all the stones and
immediately clears his own moves, amazing. I'm not sure if a better device
would actually help.

~~~
masklinn
> He remembers all the stones and immediately clears his own moves

IIRC that's basic expectation of any Go player at a non-trivial level,
starting from the mid-high amateur ranks.

It's completely expected that both players and observers can record the game
on a kifu or replay and discuss it immediately after they finish.

------
IvyMike
I sense a change in the announcer's attitude towards AlphaGo. Yesterday there
were a few strange moves from AlphaGo that were called mistakes; today,
similar moves were called "interesting".

~~~
2bitencryption
fun fact: when TD-Gammon hit the backgammon scene in the 90's, it didn't just
defeat the top level human pros, it shattered the whole metagame and changed
how humans play the game. It could be that human vs human go will look very
different in the future due to what AlphaGo has learned and can teach us.

~~~
sjmulder
Can you recommend any reading about this that’s approachable for someone who
has only a shallow understanding of the game?

~~~
eru
For Backgammon, just search for the paper on TD-Gammon.

------
bradley_long
Human can become tired, emotional and nervous. However, a computer/ software
would not have these problems.

Especially for Lee, the whole world is looking at him. An "ordinary" human
like me won't be able to make the right decisions under this pressure.

A great respect to Lee and the Developers of AlphaGo. Good Game!

~~~
javajosh
To address this, has anyone considered pitting AlphaGo against a _team_ of 9p
players consulting with each other, and perhaps taking charge of different
conflicts on the board?

~~~
kqr
I can't find the source now, but I remember reading a while ago that a team of
go players is, perhaps contrary to intuition, not significantly better than
their strongest individual alone.

There's just no useful way to "pool" human thinking power and redistribute it
to where it's needed the most – all the players will simultaneously consider
the same branches. At best you reduce the risk of making a silly mistake, but
sitting pros are already pretty good at that.

~~~
derefr
> all the players will simultaneously consider the same branches

A computer-assisted pool of humans might work. Feed each human a board state
advanced by 1/2/3/N moves down the decision tree in some direction, and have
them evaluate that particular sub-tree. It's a map-reduce problem!

~~~
arethuza
For some reason this made me think of the Focused in _A Deepness in the Sky_
\- where real general AI isn't possible [at least where we are] so human minds
are harnessed to solve problems in a deeply unpleasant way.

~~~
JabavuAdams
Is there a non-dystopian way we can do this? The immediate problems seem to be
bandwidth of communication and ability to quickly generate shared culture and
jargon.

Are Bridgewater Capital's employees Ray Dalio's focused?

~~~
arethuza
Possibly something like the "priming mods" used by police while on duty in
Greg Egan's _Quarantine_ :

[https://en.wikipedia.org/wiki/Quarantine_%28Greg_Egan_novel%...](https://en.wikipedia.org/wiki/Quarantine_%28Greg_Egan_novel%29)

------
mark_l_watson
I wonder how this will affect future human play. About 30 years ago my brother
and I started playing a simple African stone game Kala. We each won about half
the games until I coded up a brute force search to play against. Given a game
tree to the end, the program made the weirdest looking opening move, when
playing first. I started making that move and forever after won.

The situation with Go is different. (I wrote the Go program Honninbo Warrior
in the 1970s, so I am a Go player and used to be a Go programmer.) Still, I
bet the AlphaGo, and future versions, will strongly impact human play.

Maybe it was my imagination, but it sometimes looked like Lee Sedol looked
happy + interested even late in the two games when he knew he was losing.

------
brianchu
I'm totally uninformed about Go, but by now it seems that unless you're
clearly in the lead by the end of the midgame, AlphaGo is going to win, simply
because at that point its Monte Carlo Tree Search is going to our-compute you
in examining all the tactical variations in the endgame. Lee Sedol really
seemed to be under a lot of time pressure at the end.

EDIT: clarified to what I originally meant: "end of midgame"

~~~
nandemo
I don't think so. Pros are already pretty good at the endgame. Yeah, they make
blunders now and then, so it's possible that AlphaGo would gain a few points
in the endgame in some games, but not enough to overcome a significant lead
and not every time.

In any case, I personally find it more interesting to see what we can learn
from AlphaGo about the opening and the midgame.

~~~
brianchu
Isn't that what I'm saying though? If you're tied/close going into the
endgame, AlphaGo will probably win.

~~~
nandemo
My writing was sloppy, let me rephrase it: in a pro match, whatever is the
estimated score when the game goes into the endgame, that will likely be the
final score.

Where I wrote that a pro will blunder "now and then" I should have written
"rarely" \-- I don't have data to back this up but I'd guess once in every n
games for some n > 30.

------
jknz
The next person that will beat alphaGo may not be a top go player.

In particular, I'm wondering if a computer scientist with access to the
alphaGo source code and all the weights of the network could trick alphaGo in
order to win games automatically (cf. the papers that show a neural net can be
tricked to classify a plane as any other class).

If a human with the knowledge of the source code and the weights can do this,
it is scary. Imagine a similar algorithm runs your car. An attacker that knows
the source code and the weights may trick the algorithm to send your car in a
wall!

~~~
Jach
I had this conversation earlier with a friend wondering if any of Sedol's
Korean pro buddies have noticed any systematic biases that could be exploited.
I think it would be possible to make the neural net relatively useless by
playing strange sequences that hack the weights, but you're still left with a
monte-carlo-tree-search bot which alone depending on its implementation is
between 2d and 6d amateur (so on the far weaker side of the professional
scale). Whether you could make those strange sequences to trick the neural net
while also not dying to MCTS, I'm skeptical.

I'm kind of hoping for an unconventional opening in game #3. Come on tengen or
5,5... Additionally I think that might be one way of weakening the bot in that
it will find its net for suggesting candidate moves less useful and lean more
on unguided MCTS, but that's just a wild guess.

~~~
kqr
I'm fairly certain that among the many thousands of high-level games from KGS
AlphaGo was trained on, several opened tengen and 5,5.

~~~
Jach
Sure, almost any HnG fan will play them at least once. ;) But there's so few
of them in general I would think any training data derived from them would be
low-value.

------
pushrax
If AlphaGo wins all 5 matches, what do you think DeepMind will do with it? My
intuition is that they won't continue development, and instead focus on other
applications.

Great game btw, a pleasure to watch.

~~~
albertzeyer
StarCraft is their next challenge. :)

~~~
trampi
Maybe related to your interests:
[http://cilab.sejong.ac.kr/sc_competition/](http://cilab.sejong.ac.kr/sc_competition/)
[http://wiki.teamliquid.net/starcraft/ICEBot](http://wiki.teamliquid.net/starcraft/ICEBot)

------
bronz
Who was the GO professional commentator? He was consistently predicting the
moves of both Sedol and alphago. I was extremely impressed.

~~~
sgw928
As the only 9p player in the western world, Michael Redmond is of course very
impressive.

Chris Garlock, on the other hand, doesn't add that much value to the
broadcast. Maybe somebody will start a "Left Commentator" meme, just like Left
Shark.

~~~
jules
It would be great if his co-commentator was a computer scientist who is
knowledgeable about AlphaGo's algorithm.

~~~
octatoan
Indeed, I wish someone could talk about how the value/policy thing works.

~~~
taneq
As I understand it, the value network takes the place of the heuristic for
scoring a given board layout, and the policy network takes the place of the
heuristic for ordering moves from most to least promising.

When searching the game tree, at each ply the most promising N moves are
examined (as determined by the policy network) and leaves of the game tree are
scored by the value network.

------
oneeyedpigeon
What I find fascinating - and I guess this really highlights that I have no
idea whatsoever how AlphaGo works - is that at the start of game 2, AlphaGo
plays P4, then Lee Sedol plays D16. To a layman, this looks like it would be a
very, very common opening. Moreover, it's symmetrical - I'm not sure how that
affects things, but my naive intuition is that it makes the game state less
complex.

Nonetheless, AlphaGo takes a minute and a half to play its next move. Can
anyone explain what on earth is going on during those 90 seconds?

~~~
edraferi
My guess is it's doing the same thing it does every move: look at the state of
the board and calculate the next move.

A person might memorize standard openings and play them by rote. The computer
_could_ do that, but it would just be a computational short cut. In a sense,
the machine is re-discovering the classic opening.

------
sams99
The thing I find amazing about this is how soon this has happened. We all were
expecting this to eventually happen but if you asked anyone who played go and
was across the computer go scene when it would happen, say a year ago, they
would say it was "10 years out". AlphaGo is one incredible feat of
engineering.

------
pmontra
Does anybody know how many CPUs and GPUs they're using this time? It was 1200+
and 700+ in October against Fan Hui. It would be interesting to know if
AlphaGo became better only because the extra learning or also because of extra
hardware. I googled for that and didn't find anything but I could have missed
the right source.

~~~
m741
Apparently 1920 CPUs and 280 GPUs according to The Economist.

~~~
pmontra
Thanks!

------
pavpanchekha
What's even more exciting is that there weren't direct mistakes by Lee Sedol
in this game, like there were in Game 1. So does that mean that AlphaGo is
just playing on a level beyond him?

~~~
rurban
Looks like so. He miscalculated the center, while it looked he got the
corners. But AlphaGo didn't care and continued pressure

~~~
rurban
That might be the only strategy to learn from AlphaGo: If she makes an
unexpected bold move, like she did in the center a couple of times, she's onto
something and you certainly miscalculated that area. Don't gamble, defend
that.

------
typon
I've had this thought watching this play out over the past few months. You
have this deeply mystical, zen-like game of ancient China which represents the
philosophy of the East and it's pitted against this pure capitalist, cold and
calculating (literally) machine.

You can hold out for a few thousand years, but eventually the uncontrollable
and amoral technological imperative will catch on and crush you.

It's kind of poetic and sad. It feels like technology will render everything
un-sacred eventually.

~~~
aaimnr
On the other hand one could say that it's a pretty shallow understanding of
Zen, given that Buddhism assumes that everything in the mind - being a part of
nature - is fully conditioned (deterministic), as opposed to Christian view of
a human as a holy, God-like creature, distinctly different than the rest of
nature.

~~~
typon
I agree. I think the same concept applies in Daoism. Sort of giving yourself
up to the flow of the universe. That's why I only said "zen-like".

------
ljk
Wow you're fast!

good to know they'll play all 5 games no matter what the result is though

People seem to think Lee knew he lost and was just playing to learn more. Hope
he learned enough to take the overlord down in the next three games

------
studentrob
That was entertaining and I don't even really know the game. Props to Google
for making this available live on a solid feed.

I wonder if Lee Sedol will have an interest in studying deep learning after
this =)

------
0x777
Lee Sedol seemed to be doing well before he went into extra time (as far as I
could follow from the commentators). How is it ensured that this is a fair
game given the time constraints? I'm guessing adding more computing power to
the AlphaGo program should definitely help it in this regard.

~~~
zodiac
> How is it ensured that this is a fair game given the time constraints?

Both players get the same time controls, seems fair to me

But if you're saying humans might fare better against computers in a game with
a longer time control, I suspect that's true

~~~
jacquesm
Time is not a good measure when competing with parallel hardware. Joules would
be a much better one.

~~~
hughperkins
This is very insightful. Id like to mod you up ten times if i could :)

------
Jach
I may be glad no one took my bet offer of me paying $19 if AlphaGo won 3/5 vs
them paying $1 otherwise... I had a prediction at 90% confidence that nothing
would show up before the end of this year that would be capable of beating the
top players (though since I first heard about MCTS's success the idea of
coupling it with deep learning seemed obvious, so I had an unfortunately non-
recorded prediction that if a company ever bothered to devote about 8-12
months of research and manpower into combining those two algorithms with a
very custom supercomputer or tons of GPUs then they would have something that
could beat the best), then AlphaGo was announced. But the top pros weren't too
impressed with its defeat of Fan Hui, and Ke Jie estimated something like
"less than 5%" chance of it beating Sedol so I updated to 5% for this match of
it winning 3/5...

Tonight's game was beautiful. Last night's was a fighting game way too high
level for me to really grasp (I have no idea how to play like that, all those
straight and thin groups would make me nervous). I'm expecting Sedol to win
Friday since I imagine he's going to have a great study session today, but I'm
no longer confident he'll win the last two.. Still rooting for him though. :)
(I also want to see AlphaGo play Ke Jie (ed: sounds like from the other
submission on Ke's thoughts that may happen if Sedol is soundly defeated), and
for kicks play Fan Hui again and see whether it now crushes weaker pros or is
strangely biased to adopt a style just slightly stronger than who it's
facing.)

~~~
Smaug123
Re last paragraph: "just slightly stronger" is expected. AlphaGo is designed
to maximise its probability of winning, not its margin of victory. You can
expect solid not-very-flashy plays that definitely maintain an advantage,
rather than even slightly risky plays that probably increase the advantage.

------
pkaye
Let's say AlphaGo can beat all the best human Go players. Then what will the
next more difficult game for computers to compete against humans and win?

~~~
throwaway3301
Poker remains unsolved (except for 2-player "Heads-up" limit hold'em[1]).
Poker has many different variations and a player's strategy can change
significantly based on many factors. The University of Alberta Computer Poker
Research Group has already used counterfactual regret minimization in their
Cepheus AI. I wonder if they could work with DeepMind to apply some of the
techniques used in AlphaGo to poker.

[1]:
[http://science.sciencemag.org/content/347/6218/145](http://science.sciencemag.org/content/347/6218/145)

------
kerkeslager
As a programmer and a go player, I knew this day would come, but I'm a bit
disappointed that this is how it happened, for two reasons:

1\. As the game of go progresses, the number of reasonable moves decreases, so
that as the game progresses, players on average play closer and closer to
optimally. By the end of the game, even weak amateurs can calculate the
optimal move. Logically, I would guess that stronger players are able to play
optimally earlier than weak ones. Lee Sedol is known for his strong middle and
endgame, often falling behind early on and making it up late in the game. He
is so strong at this that he has driven an entire generation of go players to
developing very strong endgame. But AlphaGo, running Monte Carlo simulations,
almost certainly can brute force the game earlier than Lee Sedol can. Lee
Sedol is playing AlphaGo on its own turf. A player known for their opening
prowess, such as Kobayashi Koichi in his heyday, might have had an advantage
that Lee Sedol doesn't. (Note: I'm not strong enough to analyze Lee Sedol or
Kobayashi Koichi's play styles; I'm repeating what I've heard from
professionals.)

2\. I hoped that when an AI beat a pro at go, it would be with a more adaptive
algorithm, one not specifically designed to play go. If my understanding of
AlphaGo is correct, it's basically just Monte Carlo: the advances made were
primarily in improving the scoring function to be more accurate earlier, and
the tree pruning function, both of which are go-specific. It's not really a
new way of thinking about go (at least, since Monte Carlo was first applied to
go). It's just an old way optimized. The AI can't, for example, explain its
moves, or apply what it learned from learning go to another game. It's
certainly a milestone in Go AI, and I don't want to downplay what an
achievement this is for the AlphaGo developers, but I also don't think this is
the progress toward a more generalized AI that I hoped would be the first to
beat a professional.

~~~
versteegen
> I hoped that when an AI beat a pro at go, it would be with a more adaptive
> algorithm, one not specifically designed to play go.

The particular algorithm used by AlphaGo is of course specific to Go (the
neural network inputs have a number of hand-crafted features), but the overall
structure of the algorithm - MCTS, deep neural nets, reinforcement learning -
is very general. So there's two ways to look at it. One is that what you
wanted has actually transpired.

The other is that what you asked for is completely unreasonable. I think it
highly unlikely that an algorithm not specialised to Go will _ever_ be able to
beat all specialist Go playing programs.

AlphaGo can't explain the outputs of its two NNs, but it can still explain its
moves by showing which variations it thinks are likely.

~~~
kerkeslager
> ...the overall structure of the algorithm - MCTS, deep neural nets,
> reinforcement learning - is very general.

It is general in the sense that humans can apply those algorithms to different
problems (and have been doing so for decades). It isn't general in the sense
that we can't apply AlphaGo to other problems unmodified. AlphaGo can't even
play chess badly. It is not really even a step toward strong AI. (Note that
"strong AI" is a term with a specific meaning. [1])

> The other is that what you asked for is completely unreasonable.

That's tantamount to saying strong AI is unreasonable.

[1]
[https://en.m.wikipedia.org/wiki/Strong_AI](https://en.m.wikipedia.org/wiki/Strong_AI)

~~~
versteegen
> That's tantamount to saying strong AI is unreasonable.

No, what I said is that it's unreasonable to expect that strong AI would play
better Go than whatever the contemporary state-of-the-art Go AI is. But stated
like that, I'm not sure I can agree with my statement. Strong AI could design
and implement its own specialised Go AI. How would you count that?!

~~~
kerkeslager
Yeah, that's an interesting case. My initial reaction is that I'd think of it
as a tool that the AI was using. If a human used such a tool I'd consider it
cheating at the game. But a self-modifying strong AI could integrate the
specialized go AI into itself. If that is not considered cheating, should it
be considered cheating for a human player to integrate tools into their
physiology? Today it's pacemakers, why not a specialized go chip with a neural
interface tomorrow? And this is assuming the strong AI even has a concept of a
self separate from the software it controls; that separation might not even
make sense.

I think we might not be able to answer these questions until a strong AI
emerges.

------
astrofinch
So given that this victory seems to be happening a decade or so before experts
predicted, how likely are we to see similar acceleration in reaching other AI
milestones? (Especially given that AlphaGo is using the same algorithm that
won the Atari games, so it has the potential to be very general in its
application)

~~~
Jach
You probably saw this, but linking anyway:
[https://news.ycombinator.com/item?id=10983539](https://news.ycombinator.com/item?id=10983539)
The general point is that it's more evidence improvement can come in
discontinuous leaps, it doesn't have to be some smooth (even if accelerating)
incremental process. So timeline predictions should probably be wide, with
closer-to-present lower bounds (especially when successful generalizable
techniques become public). I don't think this initial view would have changed
much this week unless Sedol just totally crushed the bot, perhaps suggesting
there's still something more.

Personally I think the approach of combining deep learning with MCTS to beat
Go was obvious to anyone sort of familiar with each thing, and with a good
funded team could be done in a year or less, but a lot of 'experts' were
ignorant of one or more of the areas. The uncertainty should have revolved
around when some group would get around to writing the software and scaling up
with powerful hardware. Implementation details, the theory work was already
known.

My own lesson from this is that if all that is needed is tough engineering
work (but not really new theory) that work could literally arrive in a week
instead of the x+delay time it might take from scratch, because some group
could already have been in the process for about x time that isn't public
knowledge. AlphaGo kind of came out of nowhere; there were early signs with
papers on using deep learning techniques, but I don't remember any public
commitments to much. That just indicates companies are still quite capable of
doing secret projects. If they had a secret new theory, too, it could be even
more amazing. I know of one startup in particular that's been securely at the
top of its niche because internally they have secret CS research unknown to
academia.

The OpenAI initiative may be useful from the perspective that if they've
shared what probably shouldn't be shared, at least we can suspect something is
imminent and try to plan a last minute stand of getting it right first, vs
someone like Google doing all the research in secret and then bam, unleashing
UFAI.

~~~
andreyk
Hugely agree with your response. Just to add slightly to that, I think the
fact that Facebook had a quite similar Go AI in the works (just without self
play reinforcement learning) in the works is an indicator of how clear this
research direction was. The complexity of Google's solution (2 neural nets and
a fast evaluation function, plus other small details?) really indicates to me
a lot of manual engineering and iteration went into this. So it is not really
an indicator of cool new theoretical breakthroughs, but an indication that
applied engineering to make use of known techniques can achieve great things.

------
Dawny33
Wow! Monte Carlo Search learning into play in this match.

Especially when AlphaGo capitalized on just one suboptimal move of Lee Sedol.

------
skarist
I predict 5-0 for DeepMind. Now, Lee has a broken self-confidence to battle
(crucial for a human player), something that will not and _can not_ trouble
the DeepMind team.

~~~
ChuckMcM
And today you lost your bet :-)

------
spdy
Must be amazing seeing how the program you helped to create beat the best
player in one of the most complex games on this planet.

This is a milestone in modern informatics.

------
bronz
I am so glad that I got to see this live. These matches will be historic.

------
drjesusphd
So is this it then as far as games go? Does anyone know of any efforts to
develop a more "human-friendly" complete information game than go?

~~~
dfan
The game of Arimaa was created in 2002 specifically to be a complete-
information game that computers couldn't beat humans at.

As of 2015, the best player in the world is a computer.

~~~
drjesusphd
Whoa. Well, I guess that's a losing battle.

------
EGreg
Someone here said an interesting thing. Perhaps the next AI challenge would be
to see whether AI running on weaker machines can beat AI of yesterday on
stronger machines. And this test can be automated to find even better
algorithms. Like can Rybka running on an iPhone today beat Fritz running on a
distributed supercomputer? Or thinking for 2 seconds rather than 2 minutes, on
the same computer?

There is something unnerving about a computer that can answer in 0.01 seconds
and still have the move be better than any human would come up with in an
hour. At that point a robot playing simulatenous bullet chess would wipe the
floor with a row of grandmasters, beating them all without exception.

------
TheArcane
I wonder how long until AI starts writing bestselling novels.

------
axelfreeman
I don't get the mystery of this. This algorithm is complex. SURE! But deep
learning is very fast training / repeatition of a game (or some other goal)
while saving the good or bad results. Predict user moves. Find good
positions/patterns. Or did i miss some here?

[https://web.archive.org/web/20160128151110/https://storage.g...](https://web.archive.org/web/20160128151110/https://storage.googleapis.com/deepmind-
data/assets/papers/deepmind-mastering-go.pdf)

~~~
fma
Yeah...AlphaGo has played more games in the last few months than a human will
in their life time.

I'd be interested in how strong it would be if given the same constraints as
human learning (playing thousands of games, rather than millions).

------
dvcrn
I think this is really fascinating but also scary. Imagine you are the best in
the world in something. That is your thing and no one else can do it better
than you.

Then suddenly a computer comes along and takes that title from you. But it
takes it in such a way that you are never in your life able to re-take it
because of how the AI works.

A game will likely just be the first field. My girlfriend is working in
translation and interpretation which is another area already in the crosshair
of neural networks. AIs will step by step become more efficient than people
and that is terrifying.

------
Jerry2
Does anyone know what DeepMind's software stack looks like? Just based on past
work of some of the people working there, I'm guessing most of the code is C++
with some Lua. Anyone know for sure?

~~~
kozikow
For the most significant part, deep learning, deep mind uses Torch
([http://torch.ch/](http://torch.ch/)), although they are slowly moving to
Tensorflow.

------
dynjo
"By the 4th game, AlphaGo apparently became self-aware and the fate of mankind
was sealed..."

~~~
PascalsMugger
Mankind was harvested... every man, woman, and child forced to play Go for the
rest of their lives.

------
blacktulip
Does anyone notice the lack of ko[1] in the games? In all 7 public games (5
with Fan and 2 with Lee) there isn't any ko. This is unusual. If we still
can't see ko fights in the following 3 games...I would suspect that AlphaGo
isn't able to handle ko well enough yet, and Google asked Lee and Fan to not
initialize ko fights in the games.

[1]
[https://en.wikipedia.org/wiki/Ko_fight](https://en.wikipedia.org/wiki/Ko_fight)

~~~
lukev
Isn't it more likely that the opposite is true, and Ko is the kind of thing
where it is possible for a computer to play flawlessly?

I am very very very far from a Go expert, but Ko did always seem to be "first
person to make a mistake looses". If that's true, I'm sure Sedol deliberately
stayed away from it as an area where the computer is likely to be particularly
strong.

------
grouma
Exciting match with top notch commentary. I'm rooting for a sweep of the
series.

------
danielrm26
What I find fascinating about this is that the system was programmed by people
who were presumably not as good at Go as Lee Sedol.

So if the first comment in this thread (about how it's a completely non-human
approach) is true, it's really interesting that humans can enable computers to
come up with non-human ways of solving complex problems.

Seems like a big part of this story, if I'm not being completely dumb here.

------
blahblah12
[http://alphagochat.herokuapp.com](http://alphagochat.herokuapp.com)

Slack channel for discussion if anyone's interested. We're using it for
commentary while the games go on. Was created by AGA people.

------
toolslive
These matches are not really fair: the AI team can "prepare" and examine the
human's previous games, find weaknesses, aso, while the human doesn't really
have anything to guide his/her preparations.

~~~
sushirain
I don't think that AlphaGo was trained more on Lee Sedol's games, than on
others' games. The team said that they can't find computer weaknesses until
AlphaGo plays against top caliber.

------
vancan1ty
Did Lee Sedol have access to a dataset of AlphaGo games in preparation for
this match series? I wonder if it would help him if he could study the
computers moves and strategies in other matches.

------
dineshp2
Most people other than the researchers and hackers, really did not understand
what AI was capable of doing. The very idea of AI seemed too abstract to
comprehend(I consider myself guilty).

But AlphaGo showed us what AI is really capable of doing in an eerie sort of
way and I think interest in AI will soon become mainstream which is a good
thing for the development of AI.

Now it's at least easier to comprehend the context of all those doomsday
warnings about AI destroying humanity which I never took seriously.

------
jonbarker
AI enthusiast and amateur player here: Michael Redmond made a great point
yesterday, if the algorithm is only interested in maximizing probability of
win and ignoring margin of victory, shouldn't there be some override for weak
moves played when the lead is sufficient? AlphaGo played some weak moves when
it perceived it was sufficiently ahead yesterday in the end game. A truly
intelligent opponent will play strong moves even when sufficiently ahead, no?

~~~
patejam
I think the idea is that AlphaGo decided that the "strong" moves, while it may
have increased its lead, would have been more risky than moves that just
fortified its current lead.

------
i_don_t_know
Is there a complete recording of the commentary? They had one for game 1. The
current live stream only goes back two hours and doesn't include the beginning
of the game.

I'm looking at the DeepMind channel on Youtube:
[https://www.youtube.com/channel/UCP7jMXSY2xbc3KCAE0MHQ-A](https://www.youtube.com/channel/UCP7jMXSY2xbc3KCAE0MHQ-A)

~~~
forgotpwtomain
I tried to watch commentary on the DeepMind channel for game #1 and found it
to be very un-informative: I recommend the AGA channel (although it's been
linked a few times):
[https://www.youtube.com/watch?v=6ZugVil2v4w](https://www.youtube.com/watch?v=6ZugVil2v4w),
still waiting on the game 2 static video.

~~~
igravious
I disagree.

Last night I found that on the DeepMind channel they were explaining the game
a lot for beginners. This was a bit tedious for me but then this is a historic
event so there'll be noobs tuning in so I can understand that, and I
appreciate that the TV people are thinking of them.

Also, the AGA channel you just seem to have a split screen showing two people
with headphones on talking over Skype? (Granted, one of them is Myungwan Kim,
9p, a super likeable guy - but Michael Redmond, 9p, is also super likeable.)
The DeepMind channel switches between the game board and the large commentary
board, with occasional shots of Lee Sedol and occasional shots of the DeepMind
terminal.

Michael Redmond was reading out lots of variations and repeatedly trying to
count the board and evaluate the position. I think he was trying to be as
informative as he could. Occasionally he would get lost in thought as he
calculated owing to the complexity. You could call it many things but
uninformative is not one of them.

edit: added pro commentators names...

~~~
nathan_f77
I was wondering why they didn't keep a running total of the score after each
move, and show it somewhere on the screen. Isn't that pretty easy to automate?

~~~
forgotpwtomain
> I was wondering why they didn't keep a running total of the score after each
> move, and show it somewhere on the screen. Isn't that pretty easy to
> automate?

Until you are close to late-game the scoring of many of the positions is quite
flexible (depending on sente, and how certain local situations are resolved)
so although you would may have a semi-reasonable approximations for total
points, the games have been close enough that a +/\- 5 point error in
estimating will make it impossible to tell who is actually ahead.

That said, it would be very curious if we could get a virtual-board with
AlphaGo's evaluation function used to score each position.

------
mhagiwara
I always wanted to learn to play Go and one of the reasons was because it was
the only game where computers hadn't defeated human - well, it is no longer
the case and I kind of lost motivation to learn it.

I wonder what would be interesting games (intellectual sports) where computers
have yet to defeat humans that you would probably be interested in learning?

------
hutzlibu
Does anyone know of a site/video, where I can just see the game moves without
commentary and thinking pauses?

~~~
jpmattia
Game 1: [https://online-go.com/demo/114754](https://online-go.com/demo/114754)

Game 2: [https://online-go.com/demo/114161](https://online-go.com/demo/114161)

~~~
fudged71
You have those two links backwards...

~~~
jpmattia
You're right. Too quick with the copy and paste, but now the comment is
locked. :(

------
jasonjei
Isn't it kind of interesting that Google is pushing the lead for these
projects? It reminds me when IBM took on the gusto of developing Chess AI when
they had strong technical superiority. It's almost as if Google is taking the
mantle from IBM to develop these renaissance projects.

------
karussell
Wasn't reading the whole thread, but was it possible for Lee Sedol to play
against the final AlphaGo before? Although AlphaGo seems to be a huge
achievement I would find the lack of training before a bit unfair as AlphaGo
was probably able to play _lots_ of Games from Sedol before.

------
github-cat
Should we be worried about the win of AlphaGo?
[http://www.pixelstech.net/topic/141-Should-we-be-worried-
abo...](http://www.pixelstech.net/topic/141-Should-we-be-worried-about-the-
win-of-AlphaGo)

------
chm
This will be buried by now but:

What happens if the Go master tries to deceive the oponent? As in purposefully
play a counter-intuitive position, or even "try to lose"? Will the AI's
response be confused as it is expecting rational moves from its oponent?

~~~
spyckie2
No. There is a scoring heuristic that AI programs use to determine if they are
winning or not - ie: +2 stone ahead, -2 stones behind, etc. (Note: I don't
actually know what the real heuristic is, I'm making it up). A computer will
have no real reaction to a "try to lose" or counter intuitive move; it would
merely recalculate the heuristic score and doing exactly what it has been
doing all along - finding the next move that maximizes that score.

------
naveen99
Well, the nice thing with go is the handicap system. I wonder how many stone
handicap the human champion needs to beat alpha go, and watch that number
increase over time. I wonder if chess could use a handicap system to keep
things interesting.

------
myohan
I would like to see another experiment where Lee is aided by a computer and
plays against AlphaGo and see who wins...some believe that human intuition
working with a mediocre computer is much more powerful than a supercomputer by
itself.

------
vedaprodarte
A question: Should we take it as "a computer beating a human" or "developers
beating a Go player"? I had this discussion with my friends and we have
opposite opinions.

~~~
hueving
Developers beat a Go player in the same way civil engineers carry cars across
the San Francisco bay.

------
conanbatt
What is interesting to me is that the computer makes clear mistakes when its
on the lead. Since it might find the chances to win equally among different
scoring results, it often picks a weaker one.

This has a powerful consequence: we have not seen AlphaGo pushed to the limit,
he is lowering the distances as if it were playing a teaching game.

Lee Sedol I think came to this conclusion, and the only human strategy left is
to take a lead big enough to maintain the rest of the game. And that might be
the last strategy to play to show the computer is already unbeatable, because
it will be pushed to its limits to win a game and it might overcome humans.

~~~
taneq
I don't see how you can term them as "clear mistakes" when the game is playing
at a higher level than any of us meatbags. In this case (according to
DeepMind) it's no different to a racing driver backing off the pace in the
last few laps if they have a big lead - it's better to guarantee a win than to
win by a large margin.

~~~
conanbatt
You might be confusing correctness with relevance. You can pass at the last
move and lose 1 point, but if your lead is 5, its the same win-result but not
the same count-result. Well, that would be a mistake by all accounts, even if
AlphaGo made it.

Some of the moves AlphaGo played both in this game and the previous one are
very definite mistakes, but they were irrelevant to the difference in the
game.

~~~
cjbprime
> Well, that would be a mistake by all accounts, even if AlphaGo made it.

It is only a mistake to your human biases. AlphaGo literally has no conception
of the margin of a win. It doesn't care either way how many points it wins by,
as long as the win percentage is maximized.

~~~
taneq
Exactly. It's actually a pretty common behaviour for tree-searching game
playing algorithms. Unless they're set up to explicitly give weight to faster
wins or higher margin wins, they get into an unassailable position and then
just kind of dick around. If every move's a win then it doesn't matter which
move you make, right?

------
pgodzin
Is it best of 5 or are they definitely playing 5 matches?

~~~
yulunli
They will play 5 games regardless of the result. I guess it's a great learning
opportunity for both sides.

------
bennyg
I wonder how "smart" the AI can become once Lee Sedol starts pattern matching
and playing against its moves better.

------
awl130
do you think lee sedol should change his goal from trying to win all remaining
three games to winning just one? in other words, sacrifice the next two games
to learn about alphago and then try to win the final game.

------
andreyk
Very impressive. Since there is a ton of hype about this and many media
stories (at least NYTimes, with no citation at all) saying that this came 'a
decade early', I think its worth looking over Yann LeCun retrospective on
research in this area
([https://www.facebook.com/yann.lecun/posts/10153340479982143](https://www.facebook.com/yann.lecun/posts/10153340479982143)).
Clearly he was saying all this to preface the results of Facebook research in
comparison to Google's, but I still think it is a very good overview of the
history and shows the ideas did not come about suddenly. Quoting a few key
things since the whole things is very long:

"The idea of using ConvNet for Go playing goes back a long time. Back in 1994,
Nicol Schraudolph and his collaborators published a paper at NIPS that
combined ConvNets with reinforcement learning to play Go. But the techniques
weren't as well understood as they are now, and the computers of the time
limited the size and complexity of the ConvNet that could be trained. More
recently Chris Maddison, a PhD student at the University of Toronto, published
a paper with researchers at Google and DeepMind at ICLR 2015 showing that a
large ConvNet trained with a database of recorded games could do a pretty good
job at predicting moves. The work published at ICML from Amos Storkey's group
at University of Edinburgh also shows similar results. Many researchers
started to believe that perhaps deep learning and ConvNets could really make
an impact on computer Go.

...

Clearly, the quality of the tactics could be improved by combining a ConvNet
with the kind of tree search methods that had made the success of the best
current Go bots. Over the last 5 years, computer Go made a lot of progress
through Monte Carlo Tree Search. MCTS is a kind of “randomized” version of the
tree search methods that are used in computer chess programs. MCTS was first
proposed by a team of French researchers from INRIA. It was soon picked up by
many of the best computer Go teams and quickly became the standard method
around which the top Go bots were built. But building an MCTS-based Go bots
requires quite a bit of input from expert Go players. That's where deep
learning comes in.

...

A good next step is to combine ConvNets and MCTS with reinforcement learning,
as pioneered by Nicol Schraudolph's work. The advantage of using reinforcement
learning is that the machine can train itself by playing many games against
copies of itself. This idea goes back to Gerry Tesauro's “NeuroGammon,” a
computer backgammon player that combined neural nets and reinforcement
learning that beat the backgammon world champion in the early 1990s. We know
that several teams across the world are actively working on such systems. Ours
is still in development.

...

This is an exciting time to be working on AI."

------
Tistel
does anyone know anything about the implementation (language etc)?

------
lottin
From what I gather, if you have a computer powerful enough, you can solve any
game by simply applying Game Theory, as long as you can assign a numerical
value to the possible outcomes.

------
w8rbt
Would it be possible to play in a random/unpredictable fashion and win a game
of go? If so, that may be one approach to beating the computer.

------
LaFolle
This is superb awesome!!!

In future, it will be interesting to see AlphaGo playing against itself!

------
openaccount
Meanwhile 'Google Translate' translates texts terribly bad. Why don't they
work on important tasks?

~~~
ceejayoz
The AI learning techniques they're developing here are likely directly
applicable to stuff like Google Translate in the long run.

------
eruditely
Oh come on Lee Seedol we believe in you man, you might crack under pressure,
it's cool. Bring it home for us meatbags will you? HK-47 why T_T.

