
AlphaGo beats the world champion Lee Sedol in first of five matches - atupem
https://twitter.com/mustafasuleymn/status/707469083458068480
======
sethbannon
I was at the 2003 match of Garry Kasparov vs Deep Junior -- the strongest
chess player of all time vs what was at that point the strongest chess playing
computer in history. Kasparov drew that match, but it was clear it was the
last stand of homo sapiens in the man vs machine chess battle. Back then,
people took solace in the game of Go. Many boldly and confidently predicted we
wouldn't see a computer beat the Go world champion in our lifetimes.

Tonight, that happened. Google's DeepMind AlphaGo defeated the world Go
champion Lee Sedol. An amazing testament to humanity's ability to continuously
innovate at a continuously surprising pace. It's important to remember, this
isn't really man vs machine, as we humans programmed the algorithms and built
the computers they run on. It's really all just circuitous man vs man.

Excited for the next "impossible" things we'll see in our lifetimes.

~~~
esturk
I've never felt playing against what is suppose to be an entire room of
machines (wether Deep Blue or Watson) to be fair. What would be fair is to
limit the total mass of the computer to say 200kg and leave it at that. What
is effectively happening is AlphaGo is running on a distributed system of
many, many machines. Even Watson took an entire room. Google is paying a
premium to push AlphaGo to win.

~~~
taneq
It's a proof-of-concept. What they've proved is that the same kind of
intelligence required to play Go can be implemented with computer hardware.
Before now, software couldn't beat a ranked human player at Go _no matter how
much computing power we threw at it_. Now we can. Give it ten years and,
between algorithmic optimizations and advances in processing, you'll have an
unbeatable Go app on your phone.

~~~
exDM69
> Give it ten years and, between algorithmic optimizations and advances in
> processing, you'll have an unbeatable Go app on your phone.

I find this overly optimistic because of the huge amount of power required to
run the Go application. Remember, we're getting closer and closer to the
theoretical lower limit in the size of silicon chips, which is around 4nm
(that's about a dozen silicon atoms). That's a 3-4x improvement over the
current state of the art.

The computer to run AlphaGo requires thousands of watts of power. A smartphone
can do about one watt. A 3-4x increase in perf per watt isn't going to cut it.

If there will be a smartphone capable of beating the best human Go players, my
guess is that it won't be based on general purpose silicon chips running on
lithium ion batteries.

On the other hand, a desktop computer with a ~1000 watt power supply (ie. a
gaming pc) might be able to do this in a matter of years or a few decades.

~~~
sergiosgc
As solid as your argument may be, everyone saw arguments like this over and
over. Every single time they were solid. For a time, it was the high frequency
noise that would not be manageable (80s), then heat dissipation (90s), then
limits on pipeline optimization (00s) and now size constraints on transistors.
They were all hard barriers, deemed impossible and all were overcome.

I already know that your answer will be: "but this time it is a fundamental
physics limit". Whatever. I'm jaded by previous doomsday predictions. We'll go
clockless, or 3D, or tri-state or quantum. It'll be something that is fringe,
treated as idiotic by current standards and an obvious choice in hindsight.

~~~
afc
This looks like a good example of the Normalcy bias logical fallacy:
[https://en.wikipedia.org/wiki/Normalcy_bias](https://en.wikipedia.org/wiki/Normalcy_bias)

That previous constraints have been beaten in no way supports the argument
that we will beat the laws of physics this time.

~~~
bryanlarsen
The previous problems were solved because people were willing to spend
hundreds of billions of dollars to solve them. And they are still spending
that kinds of money.

If the normalcy bias was in effect, they wouldn't be spending that money.

~~~
zaphar
Actually Normalcy Bias may in fact feed that kind of money spending until such
time as reality hits. Assuming that people will automatically act more
logically when large amounts of money is in play flies in the face of recent
history. Just look at the recent housing loan crisis. Normalcy Bias played a
part there.

It's certainly possible that we'll break more barriers with clever engineering
and new scientific breakthroughs. But that doesn't mean the Normalcy Bias
isn't in play here.

~~~
bryanlarsen
Normalcy bias may have people spending lots of money on fabs assuming that the
problems would be solved by the time the fabs are built.

However, I'm talking hundreds of billions spent on R&D to specifically to
solve problems associated with chip manufacture. It took on the order of 25
years to solve each of the problems listed in the grandparent's post. Nobody
would spend that kind of money or time on something that they think somebody
else would solve.

------
dwaltrip
This is my generation's Gary Kasparov vs. Deep Blue. In many ways, it is more
significant.

Several top commentators were saying how AlphaGo has improved noticeably since
October. AlphaGo's victory tonight marks the moment that go is no longer a
human dominated contest.

It was a very exciting game, incredible level of play. I really enjoyed
watching it live with the expert commentary. I recommend the AGA youtube
channel for those who know how to play. They had a 9p commenting at a higher
level than the deepmind channel (which seemed geared towards those who aren't
as familiar).

~~~
awwducks
Yep, terrific commentary by Myungwan Kim 9p on the AGA channel.

For the folks who aren't as familiar with the game, how did you find the
commentary (for any channel)? What would you be interested in hearing for
events like these?

~~~
vessenes
I really enjoyed Myungwan's down-and-dirty commentary, and watching him get
lost in some variations, and it was just incredibly exciting to see him get
won over to AlphaGo during the game. From about move 50, I was just viscerally
excited to see where things went, and the game did not disappoint in any way.

I've read a few different reviews and watched Michael Redmond's live
commentary as well, who obviously has a slower Japanese style of play than
Myungwan, and his variations all exhibited a very thorough style and
sensibility, but I think he missed the key moment, and Myungwan called it --
the bottom right just killed Lee Sedol, and it was totally unexpected.

And, Sedol was thinking about it too, because right after he resigned, he
wanted to clear out that bottom right corner and rework some variations. I
presume that's one frustration playing with a computer -- they'll have to
instrument AlphaGo to do a little kibbitzing and talking after a game. That
would be just awesome.

If you are very, very inspired by AlphaGo's side of this, it's really
incredible to imagine, just for a moment, that building that white wall down
to the right was in preparation for the white bottom right corner variation.
The outcome of that corner play was to just massively destroy black territory,
on a very painful scale, and it made perfect use of the white wall in place
from _much_ earlier in the game.

If AlphaGo was in fact aiming at those variations while the wall was being
built, I would think at a fundamental level, Go professionals are in the
position that chess grandmasters were ten years ago -- acknowledging they will
never see as deeply as a computerized opponent. It's both incredibly exciting,
and a blow to an admirable and very unusual group of worldwide game masters.

I loved every minute!!

~~~
NhanH
Building the wall down to attack the bottom right corner isn't something
outrageous, not to those at the level of Sedol. AlphaGo definitely played
amazing, as the game was very technical in term of fighting. But the "flow"
(chase out weak group then invade a corner) is a fairly common situation. I
don't think it's a matter of AlphaGo seeing strategy further than Sedol. It
might have had much deeper calculation and reading than Sedol - as showed in
deflecting the attachment in lower right - but that's a bit of a different
story.

------
cgearhart
I was really hoping to see a more technical discussion than what I found here
in the comments. It's too bad that such a cool accomplishment gets reduced to
arguments about the implications for an AI apocalypse and "moving the
goalposts". This isn't strong AI, and it was at least believed to be possible
(albeit incredibly difficult), but it is still a remarkable achievement.

To my mind, this is a really significant achievement not because a computer
was able to beat a person at Go, but because the DeepMind team was able to
show that deep learning could be used successfully on a complex task that
requires more than an effective feature detector, and that it could be done
without having all of the training data in advance. Learning how to search the
board as part of the training is brilliant.

The next step is extending the technique to domains that are not easily
searchable (fortunately for DeepMind, Google might know a thing or two about
that), and to extend it to problems where the domain of optimal solutions is
less continuous.

~~~
j2kun
> without having all of the training data in advance

What? They certainly trained the algorithm on a huge database of professional
go games. It's even in the abstract. [1]

[1]:
[http://www.nature.com/nature/journal/v529/n7587/full/nature1...](http://www.nature.com/nature/journal/v529/n7587/full/nature16961.html)

~~~
cgearhart
> What?

 _Exactly_

They used the game database to learn the value network, then reinforcement
learning of the policy network was performed on self-play games. I.e., the
machine learned to play from existing data, then played against itself to
learn the search heuristics (the policy network) without the need for expert
data.

~~~
j2kun
Your claim still doesn't make sense. They either used expert data or they
didn't. If the algorithm would lose when they remove the expert data, then
they really do _need_ expert data.

The tree search wasn't even the novel part of the algorithm... the authors
even cite others who had used the identical technique in previous Go
algorithms.

~~~
cgearhart
It seems that my original comment is unclear. My apologies for the ambiguity.
I did not mean that they do not need _any_ expert data, but that part of the
training did not require a training data set.

They definitely need training data to learn the value function, but training
the policy network is based on self-play. While MCTS is not new, I believe
bootstrapping reinforcement learning with self-play to train a policy network
that guides the MCTS is novel.

------
clickok
I posted in the earlier thread because this one wasn't up yet[1].

Some quick observations

1\. AlphaGo underwent a substantial amount of improvement since October,
apparently. The idea that it could go from mid-level professional to world
class in a matter of months is kinda shocking. Once you find an approach that
works, progress is fairly rapid.

2\. I don't play Go, and so it was perhaps unsurprising that I didn't really
appreciate the intricacies of the match, but even being familiar with deep
reinforcement learning didn't help either. You can write a program that will
crush humans at chess with tree-search + position evaluation in a weekend, and
maybe build some intuition for how your agent "thinks" from that, plus maybe
playing a few games. Can you get that same level of insight into how AlphaGo
makes its decisions? Even evaluating the forward prop of the value network for
a single move is likely to require a substantial amount of time if you did it
by hand.

3\. These sorts of results are amazing, but expect more of the same, more
often, over the coming years. More people are getting into machine learning,
better algorithms are being developed, and now that "deep learning research"
constitutes a market segment for GPU manufacturers, the complexity of the
networks we can implement and the datasets we can tackle will expand
significantly.

4\. It's still early in the series, but I can imagine it's an amazing feeling
for David Silver of DeepMind. I read Hamid Maei's thesis from 2009 a while
back, and some of the results presented mentioned Silver's implementation of
the algorithms for use in Go[2]. Seven years between trying some things and
seeing how well they work and beating one of the best human Go players.
Surreal stuff.

\---

1\.
[https://news.ycombinator.com/reply?id=11251526&goto=item%3Fi...](https://news.ycombinator.com/reply?id=11251526&goto=item%3Fid%3D11250748)

2\. [https://webdocs.cs.ualberta.ca/~sutton/papers/maei-
thesis-20...](https://webdocs.cs.ualberta.ca/~sutton/papers/maei-
thesis-2011.pdf) (pages 49-51 or so)

3\. Since I'm linking papers, why not peruse the one in Nature that describes
AlphaGo?
[http://www.nature.com/nature/journal/v529/n7587/full/nature1...](http://www.nature.com/nature/journal/v529/n7587/full/nature16961.html)

~~~
Mangalor
Since the European match went 5-0, how do we know the bot wasn't just as good
months ago?

~~~
z0r
[https://gogameguru.com/go-commentary-deepmind-alphago-vs-
fan...](https://gogameguru.com/go-commentary-deepmind-alphago-vs-fan-hui-
game-5/)

------
Aissen
Just for context, this is the first of a five-game match. Next one tomorrow at
the same time! (6am CEST, 8pm PT).

~~~
moonshinefe
Thank you. The title on HN here didn't imply it was a 5 game series at all,
nor did the tweet it linked to.

It's a cool win but despite the way the titles are being presented, this isn't
over yet.

------
rybosome
What an incredible moment - I'm so happy to have experienced this live. As
noted in the Nature paper, the most incredible thing about this is that the AI
was not built specifically to play Go as Deep Blue was. Vast quantities of
labelled Go data were provided, but the architecture was very general and
could be applied to other tasks. I absolutely cannot wait to see advancements
in practical, applied AI that come from this research.

~~~
ktRolster
Here's the Nature article: [http://www.nature.com/news/google-ai-algorithm-
masters-ancie...](http://www.nature.com/news/google-ai-algorithm-masters-
ancient-game-of-go-1.19234) (it has a link to the free paper, as well)

The position evaluation heuristic was developed using machine learning, but it
was also combined with more 'traditional' algorithms (meaning the monte-carlo
algorithm). So it was built specifically to play go (in the same way deep blue
used tree searching specifically to play chess.....though tree searching is
applicable in other domains).

------
mark_l_watson
I just wrote a blogg about this. I was up to 1am this morning watching the
game live. I became interested in AI in the 1970s and the game of Go was
considered to be a benchmark for AI systems. I wrote a commercial Go playing
program for the Apple II that did not play a very good game by human standards
but did play legally and understood some common patterns. At about the same
time I was fortunate enough to get to play both the woman's world Go champion
and the national champion of South Korea in exhibition games.

I am a Go enthusiast!

The game played last night was a real fight in three areas of the board and in
Go local fights affect the global position. AlphaGo played really well and
world champion (sort of) Lee Sedol resigned near the end of the game.

I used to work with Shane Legg, a cofounder off DeepMind. Congratulations to
everyone involved.

------
tunesmith
I watched the commentary that Michael Redmond gave (9-dan-professional) and he
didn't point out one obvious mistake that Lee Sedol made the entire match.
Just really high quality play by AlphaGo.

Really amazing moment to see Lee Sedol resign by putting one of his opponent's
stones on the board.

~~~
mathgenius
Yeah according to Redmond, it seemed that AlphaGo made a few "mistakes"
whereas Sedol made none. And yet AlphaGo came out substantially ahead. So I'm
not sure what that means. Perhaps we need to see more in-depth analysis of the
moves, but it seems that AlphaGo just out-calculated Sedol.

~~~
nkurz
I wonder if their move selection algorithm takes into account the "surprise"
factor: given two moves that are almost equal in strength when analyzed to a
depth of N, chose the one that looks worst at N-1. That is, if all else is
equal, assume that you can search deeper than your human opponent, and lay
traps accordingly.

~~~
kqr
Besides trap-laying, there's also a second useful "surprise" factor: your
opponent is likely to have spent time on your clock to read out follow-ups to
your most likely move. By throwing in an unlikely (but still good!) move
you're forcing them to expend time on their clock to re-think their follow-
ups.

------
bencoder
I was really expecting Lee Sedol to win here. I'm very excited, and
congratulations to the DeepMind team, but I'm a bit sad about the result, as a
go player and as a human.

~~~
studentrob
If it's any consolation, there are still tons of things humans are far better
at than machines.

~~~
atemerev
The only remaining are language-related. Natural languages are the next focal
point of AI research.

~~~
sago
Skill related. I'd be interesting to see how quickly driving AIs take to beat
the best human drivers, in a weight-equal vehicle. An algorithmic competitor
in formula one, would be interesting.

~~~
genericpseudo
Would be a stomp for the computer; they don't have to respect G-forces.

~~~
sago
Racing cars aren't limited by the driver's G tolerance, I don't think. They
generate 4G-ish, from what I hear on the F1 coverage. Well within driver
capabilities. Their G is limited by tyres.

------
jonbaer
"AlphaGos Elo when it beat Fan Hui was 3140 using 1202 CPUs and 176 GPUs. Lee
Sedol has an equivalent Elo to 3515 on the same scale (Elos on different
scales aren't directly comparable). For each doubling of computer resources
AlphaGo gains about 60 points of Elo."

~~~
taneq
So has AlphaGo raised its level so far just by continuing with the games
against itself? Or did they just throw their entire server farm at it? (Or
both, probably.)

~~~
Teodolfo
Demis said it used roughly the same hardware resources as against Fan Hui?

~~~
eru
When playing, yes. Training is a different kettle of fish.

------
geebee
Terrific accomplishment.

Just a question to throw out there - does anyone feel like statements like
this one "But the game [go] is far more complex than chess, and playing it
requires a high level of feeling and intuition about an opponent’s next
moves."

… seem to show a lack of understanding of both go and chess?

I understand there may be some cross-sports trash talking, but chess, played
at a high level _by humans_ , relies on these things as well. The more
structured nature of chess means that it is (or at least was) more amenable to
analysis by brute force computer algorithm, but no human evaluates and scores
hundreds of millions of positions while playing chess or go.

Eh, the mainstream media is going to say this regardless, and I suppose it's
just unrealistic to expect them to draw a distinction between _complex for
humans_ and _amenable to brute force computation_ but statements like this
always seemed to show a remarkable lack of awareness of how people actually
play these games (though I am not an especially skilled chess or go player).

~~~
cjbprime
No, I think the statement's approximately correct. Chess has an average
branching factor of 35, Go has an average branching factor of 250, intuition
is required to prune candidate moves in Go in a way that it is only extremely
minimally required in Chess.

~~~
geebee
But is this true as humans play it? I'm not good enough at either to really
know for sure, but my impression is that while the branching factor makes a
big difference for computers, it is essentially impossible for a human to
manage the branching in either game (massive numbers of branches vs
exceptionally massive numbers of branches). As a result, humans play both
games at a high level by relying on intuition.

For instance, I read a while back (approximating and paraphrasing to
follow...) that top chess players can think up to 10 moves ahead along a very
few branches. So let's say that in chess, there are 30 million possible
positions to evaluate, and in go, there are 300 trillion. They're both such an
order of magnitude different for humans that it makes really no difference in
terms of how _we_ play the game, so intuition takes over. For computers, it's
a different story.

~~~
cjbprime
I think humans usually have ten or less plausibly-good moves to consider per
turn in Chess, and simply consider all of them, compared to tens to hundreds
in Go.

------
narrator
The funny thing about AI at this scale is we don't really know why the
computer does what it does. It's more of a inductive extrapolation that we can
verify that a technique works for a small problem, so we'll throw a whole
bunch of GPU power and data at it and it SHOULD work for a big problem. How it
actually works is fuzzy though as there's just a couple of gigabytes of floats
representing weights in neural networks. No human can look at that and say:
"Oh! I see why it made that move". It's so much data that it becomes kind of
nebulous what the AI is doing.

~~~
gearhart
It certainly seems incomprehensible now, but that isn't necessarily true in
neural networks - the amazing thing about some of the experiments using the
results of intermediate layers in image recognition is that they seem to be
building up higher and higher order "understanding" as you get to deeper and
deeper layers which correlates directly with how a human might explain their
strategy.

You can imagine a means of interpreting intermediate layers of alphago's
weighting function similar to the second image in [1] (not the best example, I
apologise) that would produce images or other abstract representations of the
strategy that layer was encoding, similar to how a human might classify moves
or patterns into categories.

[1] [http://cs231n.github.io/convolutional-
networks/](http://cs231n.github.io/convolutional-networks/)

~~~
narrator
There's a name for this phenomenon: "Subsystem inscrutablity". See this
presentation on Alpha Go linked to from here:
[http://nextbigfuture.com/2016/03/what-is-different-about-
alp...](http://nextbigfuture.com/2016/03/what-is-different-about-alphago-
versus.html?m=1)

------
cm2012
After Go, the next AI challenge they're looking at is Starcraft:
[https://twitter.com/deeplearning4j/status/706541229543071745](https://twitter.com/deeplearning4j/status/706541229543071745)

~~~
sago
The obvious problem is that speed of tactical execution can make up for a lot
of strategic thought. The famous example: you can rush a line of siege tanks
with zerglings if you can micro them fast enough[0].

[0]:[https://www.youtube.com/watch?v=IKVFZ28ybQs](https://www.youtube.com/watch?v=IKVFZ28ybQs)

~~~
LockeWatts
I hope that in the interest of fair play they'll limit their AI to 300 APM or
so. Make it win not on mechanical execution, but on decision making.

~~~
cm2012
Even with that though, They say Starcraft is still 5-10 years out for AI to
beat pros: [http://www.newyorker.com/tech/elements/deepmind-
artificial-i...](http://www.newyorker.com/tech/elements/deepmind-artificial-
intelligence-video-games) (ctrl+f for Starcraft at the bottom of this article)
\-----

------
tarvaina
The YouTube video:
[https://www.youtube.com/watch?v=vFr3K2DORc8](https://www.youtube.com/watch?v=vFr3K2DORc8)

~~~
acid__
Is this video laggy and constantly showing "The match will start in -0-
seconds" for anyone else?

~~~
mijoharas
Yes, apparently it gets better towards the end, but I have had to give up
watching the start because it is too annoying.

------
hrnnnnnn
We still have Arimaa. It's designed specifically to make it difficult for
computers to play.

[http://arimaa.com/arimaa/](http://arimaa.com/arimaa/)

~~~
simonbw
A computer won the Arimaa challenge last year.

[https://en.wikipedia.org/wiki/Arimaa](https://en.wikipedia.org/wiki/Arimaa)

~~~
faizshah
I guess the only thing left is to design a game where you can change the rules
of the game as a turn.

~~~
ucho
Might not work for long. There are already contests for best generic
solutions[0] and it seems like quite popular topic in machine learning.

[0]
[https://en.wikipedia.org/wiki/General_game_playing](https://en.wikipedia.org/wiki/General_game_playing)

------
codecamper
A human was beaten with some thousands of CPUS & GPUS. On a calorie level, the
human is still more efficient.

On a time to learn these skills... going from zero (computer rolls off
assembly line) to mastery, the computer wins.

Actually maybe the computer wins even on the caloric level, if you consider
all the energy that was required to get the human to that point (and all the
humans that didn't get to that point, but tried).

~~~
ragebol
But the computer certainly does not win on the amount of training samples
required. The human is at the same level as the computer now for Go, but the
computer has had much more training samples as Lee Sedol could process in his
lifetime.

The next step is to reduce the training time/samples for the computer to get
the same performance.

~~~
jules
That's silly. Why would you want to put human limitations on the computer? We
don't artificially put computer limitations on the human.

~~~
ragebol
I don't want to put a limit on the computer, not at all. But I do think humans
have an edge on computer because at least for now, humans can learn the same
skills from less samples (at least in this example: Go)

Of course, if there are many samples, the computer can go through those
faster, but if there are no samples already and the computer has to learn
example by example as humans do as well, humans may still have an advantage.

Of course, this advantage will diminish as well as AI advances.

~~~
jules
How do you count examples? The computer can generate its own examples by
playing against itself. So in theory it needs 0 examples. This is not a useful
metric at all.

~~~
ragebol
I would count every played game as an example.

What I mean is that I am more impressed by anyone of anything that can do a
task (go, golf, chess, learning a foreign language, doing the dishes even)
well with just a single example, or e.g. an hour of training.

Being able to train in solitude is an advantage indeed. You need two humans to
do this, but you also need two AlphaGo-instances as well.

~~~
jules
Are you going to count all the games that the human played in their head too?
What about the learning done in the human brain when sleeping? Do you count
that too?

------
Radim
Beating humans in Go is, in itself, not all that exciting. _Go bots have been
beating strong humans for quite some time now_ (just not the very top humans).

There are other implications that make this AlphaGo progress super exciting
though. Go captures strategic elements that go well beyond the microcosm of
one nerdy board game.

That's the real reason Go has been around for >2,000 years, and why this AI
progress is relevant, despite its limited "game domain".

I wrote about it here, from my perspective of an avid Go player & machine
learning professional [1].

[1] [http://rare-technologies.com/go_games_life/](http://rare-
technologies.com/go_games_life/)

~~~
habosa
I disagree with this due to the rate of AlphaGo's progress. Consider
CrazyStone which was the previous state of the art in Go computers. That
program reached 5dan after many years of development and has not shown any
signs of being able to reach Lee Sedol level (9dan).

In October of this year AlphaGo beat a 5dan player, bringing it into the range
of CrazyStone. Only ~6 months later it beats a 9dan player which means it is
now ~400 Elo higher. This means the new version would be predicted to beat the
old version ~99% of the time.

Such incredible consistent progress of a problem considered somewhat
intractable is notable and exciting. Imagine where this machine will be in 6
more months.

~~~
habosa
Edit: Fan Hui was only 2dan so this is even more insane.

~~~
Radim
Yes, AlphaGo's progress is amazing. I don't think there's any disagreement
there :)

But I don't think you know much about Go, if you can say Fan Hui is "just" 2
dan professional. What do you reckon the strength difference is between 2p and
9p?

Nitpick: while AlphaGo today is certainly stronger than AlphaGo last October,
it doesn't follow in any way from the fact that both programs beat their
respective opponents. _A > B_, _C > D_, _D > B_, therefore _C > A_? By "400
ELO", no less?

~~~
habosa
[https://en.wikipedia.org/wiki/Go_ranks_and_ratings#Elo-
like_...](https://en.wikipedia.org/wiki/Go_ranks_and_ratings#Elo-
like_rating_systems_as_used_in_Go)

You can use that table to calculate the win probability for a 9dan player
versus a 2dan player.

~~~
Radim
I rest my case.

For your info: professional ranks do not reflect strength. They are honorary
and based (typically) on achievement and seniority.

------
moonshinefe
Can someone explain why this is more impressive than a computer beating top
chess players over a decade ago? I'm not very familiar with Go, and while
there were far more squares on a Go board, it seems less sophisticated than
chess to me.

Maybe Go has way more moves possible and emergent strategies or something I'm
not taking into account.

~~~
gjm11
Here's a way to measure the sophistication of a game of skill. Consider two
players A and Z. A is a ten-year-old who has just been told the rules; Z is
God. Now, in between them, put a series of other players B, ..., Y, where B
beats A 2/3 of the time, C beats B 2/3 of the time, ..., Z beats Y 2/3 of the
time. (We assume God doesn't use his magical divine powers to cheat by, e.g.,
making Y play bad moves.)

Unfortunately, God is not readily available for comparison, so we'll use the
best human players instead.

How many links are there in that chain? The more there are, the more there is
to learn about the game, and hence the deeper and more sophisticated the game
is. (So you might think, anyway.)

If you rate players using the Elo system, beating someone 2/3 of the time
corresponds to being about 150 points stronger. A complete beginner at chess
might have an Elo rating of 500, compared with the world champion somewhere
around 2900, giving 16 links in the chain.

In go, beating someone 2/3 of the time corresponds to being about one kyu/dan
rank stronger. A complete beginner might be 30 kyu; the best players are
stronger than 9 amateur dan, so that's at least 40 links in the chain. (Lower-
numbered kyu ranks are stronger; after 1 kyu comes 1 dan, and then higher-
numbered dan ranks are stronger.)

So by this measure -- which you may or may not find convincing -- go is a more
sophisticated game than chess.

Here is the best argument I know against this definition. Define the game of
"tenchess" as follows. To play a game of tenchess, you play ten games of chess
and the winner is whoever wins more games (a draw if the same number). Then
it's easy to see that tenchess has a longer chain, as defined above, than
chess; if I win 2/3 of my chess games then I win 79% of my tenchess games, so
I can win 2/3 of my tenchess games with a smaller advantage. (I am ignoring
the existence of draws for this calculation, just for simplicity.) But surely
tenchess isn't a deeper game than chess; it's just _longer_. Perhaps go's
longer chain is just the result of its being a longer game.

~~~
stromgo
> In go, beating someone 2/3 of the time corresponds to being about one
> kyu/dan rank stronger

This isn't true. One kyu/dan rank stronger means being 1 stone stronger (so
winning 50% of the time when playing White with reverse komi). In practice
this may correspond to winning 2/3 of the time with normal komi for high dan
players, but that doesn't hold for low kyu players. A 29k has maybe a 51%
chance of winning against a 30k because both will make huge mistakes. So
although the 29k can score on average 13-15 more points than the 30k in a
given game, this advantage is swamped by the large standard deviation of
scores in beginner games, turning the win/loss outcome into essentially a coin
flip.

~~~
gjm11
Fair comment about very weak players. My impression is that the element of
chance goes way down well before you get to high dan level, though. How sure
are you that I'm wrong?

~~~
stromgo
Quite sure. 17k players still make many huge mistakes, and at the other
extreme, God (say 13d) would win 100% of the time against a 12d player. Given
that the win ratio for a one rank difference starts at 50% for extreme
beginners and ends at 100% for God, maybe you should be the one explaining why
you'd expect a significant plateau at 2/3 instead of a smooth increase.

~~~
gjm11
I agree. In particular, I wasn't arguing for anything special to happen at
p=2/3\. I was hopeful, though, that p=2/3 might be a tolerable approximation
over a reasonable range of skill levels.

Having played a bit with some toy models, I've changed my mind a bit; my guess
is that p=2/3 is a reasonable approximation for few-dan and few-kyu amateurs,
but that outside, say, the 5k-5d range it's far enough off to make a
substantial difference.

So, what does this do to those (anyway fairly bogus) "depth" figures? My
crappy toy model suggests that for a 2/3 win probability you need a 3-rank
difference around 24k, a 2-rank difference around 12k, a 1-rank difference
around 2d, a 0.5-rank difference around 8d. And I estimate God at 15 amateur
dan (if Cho Chikun is 9p and needs 4 stones from God then God is 21p; if,
handwavily, 9d=3p and one p-step is 1/3 the size of one d-step, then God is
21p = (3+18)p = (9+6)d = 15d). So we need maybe 20 steps from God to 5d, then
maybe 10 from there to 5k, then maybe 5 from there to 15k, then maybe 5 from
there to 30k. That's 40 steps -- not so very different from what we get just
by pretending one rank = one "2/3 win probability" step, as it happens.

------
agentultra
Isn't this jumping the shark a bit? It's a 5-game match. The first was really,
really close.

~~~
Matetricks
It's worth mentioning that Lee Sedol mentioned in an interview that even if he
loses a single game against AlphaGo, he will have lost the match. He was
expecting to win all 5 games.

~~~
agentultra
I hope that doesn't shake his determination and ability to concentrate. He
could still win.

------
kowdermeister
I'm truly amazed also, I'm not surprised or shocked. Once I knew that the
previous master was beaten, I knew it's just a matter of time to see the #1
player topped.

What would be shocking is to find out that a famous writer, musician or
scientist is in fact, just an alias for an advanced AI system :) It needs a
little trick, because people should be tricked into believing that there's a
real person behind the name.

Oh wait, I just remembered that there's a (mediocre) movie made on the
subject: S1m0ne (
[http://www.imdb.com/title/tt0258153/](http://www.imdb.com/title/tt0258153/) )

Are you saying it won't happen? Think of the guys saying the same of go :)

~~~
pgeorgi
> What would be shocking is to find out that a famous writer, musician or
> scientist is in fact, just an alias for an advanced AI system :)

so, Milli VanAIlli?

([https://en.wikipedia.org/wiki/Milli_Vanilli](https://en.wikipedia.org/wiki/Milli_Vanilli))

~~~
kowdermeister
Nice catch, that would be the perfect working title for the project :)

------
ankurdhama
What this actually means is that "the approach" AlphaGo team developed to
"computationally" play Go, which is an computationally intractable problem,
will be very useful in other computationally intractable problems. The media
is going to get crazy without understanding what actually happened. If you are
going very hysteric over this and thinking that robots are going to take over
then please try this:- Before the start of the game add/remove/update any
rules of the game and tell both the players - the human and computer - at the
start of the game about new rules and lets see who wins.

------
conanbatt
This not only shows the insane advances in computer AI, but an incredible
advancement between the Fan Hui games and this one. Im still going through the
kifu to get a sense of how could it have improved so much in only 6 months.

~~~
ktRolster
I feel like it started being more aggressive, playing more fighting
moves....whereas in the last match it was playing mostly a defensive game (I'm
not an expert by any stretch of the imagination, though).

~~~
apetresc
It's exactly the opposite – this last game was much more aggressive on both
sides than the Fan Hui match.

------
imh
I want to scratch my itch and play some go. I suck, and playing against other
players online I get destroyed so quickly I feel like I'm ruining their fun.
Where can I find a fun bot with variable difficulty?

------
terryf
Extremely interesting news and kind of sad as a human being :)

I don't really know that much about AI, but hopefully some experts can tell me
- how different are the networks that play go vs chess for example? Or
recognise images vs play go?

What I mean is - if you train a network to play go and recognise images at the
same time, will the current techniques of reinforcement learning/deep learning
work or are the techniques not sufficient at the moment?

If that works, then it really does seem like a big step towards AGI.

~~~
GolDDranks
This is basically a combination. A "traditional" chess program would use a
tree search, but trees get quickly ot of hand since they grow exponentially.
The trick is to prune them, and they trained a network to do that. It selects
just the moves that look good to it. (It has some level of randomness to it,
too) After reaching deep enough in the search tree, they use another network
to evaluate who's winning. Usually this is hard to do in Go, and that's why
the second network is quite novel and helpful.

So, they use a combination of techniques. And they're doing well at it.

~~~
terryf
right, yes, but my question was meant to be a bit more general - this and
various other results have shown that it is possible to train a deep net to do
a specific task very successfully - my question was if it's possible to train
it to do two or more tasks as successfully or will the network then have to be
exponentially larger. I suppose there is no known way to "combine" trained
networks together.

~~~
andreyk
The standard is to have each neural net be trained just for one task, as you
say; there may be research into multi-skill neural nets but I have yet to see
any. AlphaGo in particular is extremely specialized to Go, even in terms of
how the algorithm is implemented.

------
devy
I had a feeling that AlphaGo would beat Lee Sedol yesterday after watching Fan
Hui's interview [1].

According to Hui's recall, the defeat all came down to these things: the state
of the mind, confidence and human error. The gaming psychology is a big part
of the game, without the feelings of fear of being defeated and almost never
making mistakes like humans do, machine intelligence beating human at the
highest level of competitive sports/games is inevitable. However, to truly
master to game of Go, which in ancient Chinese society, it's more of an
philosophy or art form than a competitive sport, there is still a long way to
go.

There were a ton of details Hui cannot speak of due to the non-disclosure
agreement he signed with DeepMind, but those were the gist of the interview.

In the end, AlphaGo match is 'a win for humanity', as ​Eric Schmidt put it.
[2]

[1]
[http://synchuman.baijia.baidu.com/article/344562](http://synchuman.baijia.baidu.com/article/344562)
(In Chinese)

Google Translate: [https://translate.google.com/translate?hl=en&sl=zh-
CN&tl=en&...](https://translate.google.com/translate?hl=en&sl=zh-
CN&tl=en&u=http%3A%2F%2Fsynchuman.baijia.baidu.com%2Farticle%2F344562)

[2] [http://www.zdnet.com/article/alphago-match-a-win-for-
humanit...](http://www.zdnet.com/article/alphago-match-a-win-for-humanity-
eric-schmidt/)

------
pushrax
That sequence on the right side was excellent, I am so impressed with the
level of play.

------
ccvannorman
reference: SGF file on OGS: [https://online-
go.com/demo/114161](https://online-go.com/demo/114161)

To my untrained eye, AlphaGo was already way ahead by move 29 in the match
tonight with black having a weak group in the upper side, while black wasted a
lot of moves on the right side as white kept pushing (Q13, Q12), which white
erased later because those pushes were 4th line for black and the area was too
big too control. Black never had a chance to recover this bad fight. After
those reductions and invasion on right side white came back to the 3-3 at C17
which feels like solidified the win.

Some people are asking what was the losing move for Lee Sedol? I wanted to
joke and say "the first one.." but maybe R8 was too conservative being away
from the urgent upper side where white started all the damage.

------
GraffitiTim
A historic moment here, folks.

Incredible, and in my opinion a little terrifying.

~~~
nbaksalyar
What's more terrifying is that AlphaGo can learn even further, from these and
other matches. Just hard to imagine what lies ahead.

~~~
taneq
The Singolarity is here!

------
ausjke
No surprise at all, human brain is an organ with limited neurons, and computer
doubles its performance very 18 months. In fact not just the chess, I would
say that AI will beat human all around at unlimited ratio in the future, when
they learned how to improve themselves especially.

------
bwang29
I was just thinking, does AlphaGo's game strategy also emulate some sort of
psychological strategies used by real human, such as bullying, confusing or
making fun of its opponent when it sees fit.

------
nefitty
What do you guys think of the future progress on the game Go? Will our only
chance against AI be to team up with an AI to beat the lone AI? Like in this
article about centaur chess players:
[http://www.wired.co.uk/magazine/archive/2014/12/features/bra...](http://www.wired.co.uk/magazine/archive/2014/12/features/brain-
power/page/2) (2014) It all sounds very Gundam Wing to me.

------
nopinsight
_Deep Blue:_

Massive search +

Hand-coded search heuristics +

Hand-coded board position evaluation heuristics [1]

 _AlphaGo:_

Search via simulations (Monte Carlo Tree Search) +

Learned search heuristics (policy networks) +

Learned patterns (value networks) [2]

Human strongholds seem to be our ability to learn search heuristics and
complex patterns. We can perform some simulations but not nearly as
extensively as what machines are capable of.

The reason Kasparov could hold himself against Deep Blue 200,000,000-per-
second search performance during their first match was probably due to his
much superior search heuristics to drastically focus on better paths and
better evaluation of complex positions. The patterns in chess, however, may
not be complex enough that better evaluation function gives very much
benefits. More importantly, its branching factor after using heuristics is low
enough such that massive search will yield substantial advantage.

In Go, patterns are much more complex than chess with many simultaneous
battlegrounds that can potentially be connected. Go’s Branching factor is also
multiple-times higher than Chess’, rendering massive search without good
guidance powerless. These in turn raise the value of learned patterns. Google
stated that its learned policy networks is so strong “that raw neural networks
(immediately, without any tree search at all) can defeat state-of-the-art Go
programs that build enormous search trees”. This is equivalent to Kasparov
using learned patterns to hold himself against massive search in Deep Blue (in
their first match) and a key reason Go professionals can still beat other Go
programs.

AlphaGo demonstrates that combining algorithms that mimic human abilities with
powerful machines can surpass expert humans in very complex tasks.

The big questions we should strive to answer before it is too late are:

1) What trump cards humans still hold against computer algorithms and
massively parallel machines?

2) What to do when a few more breakthroughs have enabled machines to surpass
us in all relevant tasks?

Note: It is not entirely clear from the IBM article that the search heuristics
is hand-coded, but it seems likely from the prevalent AI technique at the
time.

[1]
[https://www.research.ibm.com/deepblue/meet/html/d.3.2.html](https://www.research.ibm.com/deepblue/meet/html/d.3.2.html)
[2] [http://googleresearch.blogspot.com/2016/01/alphago-
mastering...](http://googleresearch.blogspot.com/2016/01/alphago-mastering-
ancient-game-of-go.html)

~~~
_xander
Strong AI is not necessarily a bad thing. Instead of worrying about questions
1 & 2, we could be thinking less about constraint and competition with AI and
more about cooperation and goal-orientation: e.g. the work of Yudkowsky
([https://intelligence.org/files/CFAI.pdf](https://intelligence.org/files/CFAI.pdf))
or some of the thoughts provided by Nick Bostrom
[http://nickbostrom.com/](http://nickbostrom.com/). Goal-orientation is
preferable to capability constraint because the potential benefits are far
larger.

Tl;dr: I, for one, welcome our robot overlords (so long as they don't behave
like our robot overlords).

~~~
nopinsight
I agree that superintelligence could bring enormous benefits to humanity but
the risks are very high as well. They are in fact existential risks, as
detailed in the book Superintelligence by Bostrom.

That is why we need to invest much more research efforts on Friendly AI and
trustworthy intelligent systems. People should consider contribute to MIRI
([https://intelligence.org/](https://intelligence.org/)) where Yudkowsky, who
helped pioneer this line of research, works as a senior fellow.

------
scott_hardy
What an amazing game to watch. Congratulations to the AlphaGo team, and good
luck to both players in the next four games!

------
randomgyatwork
AI is good for rules based systems, but most of the worlds problems that need
to be solved don't have rules in the same way a board game does. Sure it's
cool that a computer beat a human at a board game, but thats like celebrating
a penguin being better at fishing than a person with bare hand

------
mrdrozdov
How much did this match cost the AlphaGo team? (From a computing resources
perspective)

------
bane
It's almost kind of bad timing in the U.S., what with one of the most insane
primary seasons in our history -- this will probably not make the news at all
let alone the front page like Kasparov's and Magnus's games did.

------
joe563323
Learning from experience goes both to the program and to the champion. Does
this mean if the champion keeps playing with the machine several times, he has
a chance of winning?

------
dropdatabase
I don't think a computer could ever beat me at Calvinball

------
panic
It'll be interesting to see what new things we learn about Go itself from
DeepMind. The game is very deep, and apparently we haven't found the bottom
yet!

~~~
visarga
Instead of the prize money, if I were Lee Sedol I'd request unlimited play
time against the latest AlphaGo.

------
socrates2016
I think it will be very interesting if Lee Sedol can win one. Humans have
different blueprints and environments. Who is to say a human can't become
better?

------
couchand
When there's a computer that can beat the world champion at both go and chess
with no modifications, then I'll be scared.

~~~
picozeta
You just take your top-notch Go/Chess engines and detect the game in the
initial step.

------
georgehaake
I have read a fair amount about how it was written without much detail. Anyone
know what it was written in?

------
chimtim
AlphaGo can be beaten. It uses reinforcement learning so it will perform the
set of moves that in the past led to its win. So predictable. Sedol just needs
to take control and make it play in a predictable fashion. Also, perhaps play
obscure moves that AlphaGo wouldn't have trained on. Perhaps next year's Go
winner will have a PhD in computer science.

------
kul
Here's one: how long until a computer can beat a human assisted by a computer?

~~~
visarga
Will humans be able to keep up with the depth of analysis these AIs will have,
or will it become a problem for the AI to dumb down its thinking in order for
us to grasp it?

More generally, scientists using AI for research will probably have to do
research on the research, to understand what the AI discoveries mean. Maybe
they mean something we can't grasp at all, in which case they go completely
over our heads, like ants trying to learn about the finer points of financial
markets. We will probably have to learn new concepts and even new languages
designed by the AI to convey the meaning.

~~~
d0m
I wouldn't say "dumb down" but it definitely needs to explain why it took some
lines of reasoning. With deep learning, you need to rebuild the whole system
with different test-cases to change a minor behavior.. but imagine if we could
just say "Why did you do that? XYZ. And adjust it: "Oh, gotcha. You can't
because of ABC", and then the AI has that problem solved. I guess that would
be the next step in AI. I think it's called symbolic reasoning.

Here's a very good article: [http://dustycloud.org/blog/sussman-on-
ai/](http://dustycloud.org/blog/sussman-on-ai/) (A conversation with Sussman
on AI and asynchronous programming)

------
vancan1ty
Does Lee Sedol have access to AlphaGo training games and/or matches?

------
pvinis
i would like to see the same match, but switched placed. alphago plays itself,
this time as black, to kind of see the choices it would make, and if they
would align with lee's.

------
tvvocold
Poll:
[https://news.ycombinator.com/item?id=11250806](https://news.ycombinator.com/item?id=11250806)

------
EGreg
Does this mean in the next few decades, computers will make better sex
partners and companions than any human?

~~~
Eliezer
Worrying about the effect of strong AI on sexual relationships is like
worrying about the effect on US-Chinese trade patterns if the Moon crashes
into the Earth.

~~~
ue_
Who says? I mean, computers can be used for multiple purposes, and although
some applications of AI seem more "noble" or "intellectual" than others, the
pursuit of knowledge and sexual relationships are both sense pleasures that we
indulge in to make ourselves feel good. On a very large scale, one is hardly
more noble than the other.

------
supergirl
after so much press about this, it would be funny if overall the human wins

------
21
The thing that was supposed to take at least 10 years happened. Only last
month people were still saying that no way AlphaGo will beat the champion and
that it will be crushed. Today everybody will have seen it coming and say that
it was normal.

Yet people will still tell that worrying about AI taking over is like worrying
about overpopulation on Mars, and that this is a problem at least 50 years
out.

~~~
simonh
Highly optimised single-function algorithms like this are impressive stuff and
can lead to useful tools, but that's it. This gets us no closer to strong AI
than a tic tac toe program. Until we have systems that can tackle a wide range
of fundamentally different problems and independently adapt strategies for
dealing with one class of problems to deal with other classes of problems,
systems like Alphago will remain one trick wonders with little relevance to
'true' AI.

Edit: I do understand that the techniques used to implement Alphago can be
used to implement other single-function solvers. That doesn't make it a
general purpose strong AI.

~~~
Houshalter
Welcome to the AI effect! Every time AI makes an accomplishment, it is
disregarded. The goalposts are perpetually moved. "AI is whatever computers
can't do yet."

People said for years that Go would never be beaten in our lifetime. They said
this because Go has a massive search space. It can't be beaten by brute force
search. It requires intelligence, the ability to learn and recognize patterns.

And it requires doing that at the level of a human. A brute force algorithm
can beat humans by doing a stupid thing far faster than a human can. But a
pattern recognition based system has to beat us by playing the same way we do.
If humans can learn to recognize a specific board pattern, it also has to be
able to learn that pattern. If humans can learn a certain strategy, it also
has to be able to learn that strategy. All on it's own, through pattern
recognition.

And this leads to a far more general algorithm. The same basic algorithm that
can play Go, can also do machine vision, it can compose music, it can
translate languages, or it could drive cars. Unlike the brute force method
that only works one one specific task, the general method is, well, general.
We are building artificial brains that are already learning to do complex
tasks faster and better than humans. If that's not progress towards AGI, I
don't know what is.

~~~
fma
As far as I know the goal post of Turing test has never moved.

~~~
Mangalor
Chatbots can already beat the turing test.

~~~
simonh
Chatbots ate trivially easy to beat. Just try teaching it a simple game and
ask it to play it with you. Basically any questions that require it to form a
mental model of something and mutate or interrogate the model state.

Many of the chatbot Turing test competitions have heavily rigged rules
restricting the kinds of questions you're allowed to ask in order to give the
bots a chance.

------
bitmapbrother
Some people were downplaying the victory of AlphaGo over the European champion
because he was only a 2p player. I wonder what they have to say now.

~~~
z0r
The last victory was significant, but this victory was far more significant.
The professional dan scale isn't exactly linear and the ranks can't simply be
compared numerically even when they are granted by the same organization - and
Korea, China and Japan all have at least one organization of professional go
players that each maintain their own rankings. Sedol is a current top player
who has won many, _many_ titles and Fan Hui is 10 years out of regular
professional play and doesn't have a title to his name. What people were
saying before is still true today. All of the reporting has suffered from the
usual problems of describing something specialized to the general public, and
all the typical inaccuracies of such journalism (compounded by Google's PR
department being the source of some of it).

Congratulations to the team at Deepmind, and I'm wishing good luck for Sedol
in the remaining matches - if he wins we would certainly get to see a second
series rematch some months down the line, and that would be very exciting for
go fans everywhere.

------
jorgecurio
man I am fired up to watch tonights game...like I am fired up for UFC

there should be like a North American Go Nationals or something like that
televised on twitch

Anyone putting money down on Sedol? He said it will be either 5-0 or 4-1 in
his favor.

------
arao
Lee is not the best player NOW.

------
thomasahle
Giant spoiler! Does Hacker News have any policy against these things?

------
typeformer
Lee Sedol should have played that top left 3,3 move earlier (at least before
white covered it) WTF. Humanity is not longer at the top of the intelligence
pyramid...

------
andrepd
He has lost 1 game of a 5 game match, on a handicap. Hardly a defeat.

~~~
fogleman
What handicap?

~~~
andrepd
I misread it, he had only the usual first move advantage handicap.

