
Google reveals secret test of AI bot to beat top Go players - cpeterso
http://www.nature.com/news/google-reveals-secret-test-of-ai-bot-to-beat-top-go-players-1.21253
======
dfan
One thing that isn't made clear in this writeup is that Master plays in a very
nonhuman style, as opposed to the version of AlphaGo that beat Lee Sedol,
which mostly played like a strong human except for a few surprising moves. My
first guess when I saw Master's games was that it was a program like AlphaGo
that had its policy network trained from scratch rather than being
bootstrapped by being given the goal of imitating the moves of strong humans.
I'm eager to find out whether that was the case or whether it just moved away
from human-like play during a very long self-play phase.

It really is a bit scary to see. I would not have guessed that human strategy
was that deficient. Computer chess programs still tend to play with human-like
strategy (partially because humans have coded their evaluation functions!) but
godlike tactics. Master is not really playing like a human at all.

~~~
stouset
As computers are able to evaluate positions faster (and therefore deeper), the
"godlike" tactics are dominating over human-style strategy. It used to be that
computers played "computer-like" moves because they didn't understand the
position. Now, they play computer-like moves because "understanding" the
position isn't as important as just being able to see 25+ moves ahead.

In a nutshell, positional play in chess is simply heuristics we humans use to
be able to evaluate a position in lieu of being able to calculate deep non-
forced lines. Computers do use this to an extent (as you point out, we coded
their evaluation functions) but positional play matters less when you see all
the outcomes of every possible tactic with 100% accuracy. So computers tend to
play reasonably human-like in the openings, but by the time you reach the
middle game they'll happily enter lines where their pawn structures are
shattered, pieces appear superficially to have little coordination, and where
their king safety appears compromised (all things humans rarely intentionally
do), all because they've seen that it works out 25+ moves in advance.

~~~
1024core
> Now, they play computer-like moves because "understanding" the position
> isn't as important as just being able to see 25+ moves ahead.

I don't think you can "see 25+ moves ahead" in Go. The branchout factor is
just too big.

~~~
dfan
Well, stouset is talking about chess, but in Go, Monte Carlo Tree Search plays
out all the way to the end of the game; it just does so in a much less
exhaustive way, for exactly the reason you mention.

------
jonmc12
Interesting reddit discussion before reveal. Assesses bot identity and
gameplay:
[https://www.reddit.com/r/baduk/comments/5l3l7e/chinese_ai_cr...](https://www.reddit.com/r/baduk/comments/5l3l7e/chinese_ai_crushing_pros_on_fox_server/)

~~~
dwaltrip
The reddit thread also links to more interesting discussion on the board
lifein19x19:

[http://lifein19x19.com/forum/viewtopic.php?f=10&p=215117#p21...](http://lifein19x19.com/forum/viewtopic.php?f=10&p=215117#p215117)

------
btilly
We know that Master can figure out what it would play. We also know that its
predecessor had a model for what moves a human professional would be likely to
play.

What I would find truly fascinating is if Master could divide moves that it
plays, which professionals wouldn't, into groups based on a similar internal
categorization of the moves. And then see if human minds can look at any of
groups and come up with a human understandable principle that humans had been
missing about the game.

The point here is not so much to improve human play (though it presumably
would do so), but as a step towards having an AI that can break down its
internal model into principles that can be used to train another AI to learn
those principles. Just like how a human expert can learn to turn expertise
into something that can be taught to other humans.

This has several potential benefits. The first is that human experience
suggests that this type of introspection tends to improve our own competency.
The second is that we could have a single AI trained by multiple specialist
AIs to get a compact "generalist". And the third is that this is a path
towards having AIs that can discover things then teach them to humans.

The whole idea might fail horribly. But I'd like to see it given a shot.

~~~
cLeEOGPw
Your described method of understanding his moves might work for some moves,
but there will inevitably be moves that are just a sum of so many different
probabilities that are so far down the road that even looking at the result
nobody would be able to actually recognize such a move on slightly different
board.

At some point, if this method would be used, it would probably require a
second AI that would help understand the main AI, because the primary
"explanation" would still be too complex and/or subtle for us to comprehend.

~~~
btilly
Obviously "automatically extract the principles in an understandable format"
is a long term and probably unachievable goal.

However there is hope that cluster analysis on data about the internal
reasoning process can successfully identify groups of positions that "seem to
share a common principle". Success in that is a first step towards lots of
interesting things.

------
koolba
This seems like the beginnings of the plot of an anime I'd want to watch.
Season two would probably start with the developer's getting complacent and a
Chinese AI entering the scene.

~~~
jaxondu
Really hope to see one machine against another. In future we'd probably have
game tournament of AI machines. Example my tensorflow-build machine against
your openai-build machine in a StarCraft game.

~~~
kefka
Corewars, where the program is written by a neural net.

------
MR4D
This is impressive. According to the article, the bot has not yet lost.

I think at the end of 2017, we may have to say that computers have conquered
Go.

Interesting that the timing is 20 years after mastering chess.

~~~
pmontra
The series is over with a score of 60-0.

You can find all the games at
[http://tieba.baidu.com/p/4922688212?pn=1](http://tieba.baidu.com/p/4922688212?pn=1)

It's chinese but the pictures are universal and the comments don't really
matter much.

By the way, Google Translate makes a pretty horrible work at translating those
pages. I think they need to add some more DeepMind to it :-)

To be fair, they probably don't have much training data for the jargon of the
game of go or of any other game.

~~~
guelo
How to know if Master is white or black in each game?

~~~
pmontra
After the diagram there is the Sgf file with the moves. PB is player black and
PW is player white. AlphaGo is Magister in the first 20 games and Master in
the last 40. Two accounts on two different servers.

------
partycoder
It would be good to start experimenting with handicap stones to understand how
many stones stronger the bot is.

[http://senseis.xmp.net/?Handicap](http://senseis.xmp.net/?Handicap)

Also experimenting with a new ranking system beyond 9 dan.

~~~
nathan_f77
I would also like to see how the strength is affected by computing resources.
For example, one server, versus 10 servers, versus a room full of servers,
versus an entire datacenter. I wonder how close this is to playing a perfect
game.

I would also like to know if two perfect players would always end up with a
draw, or if they would each win 50% of the games.

~~~
SonOfLilit
Either the white player would win in 100% of matches, or the black player
would win 100% of matches.

Go rules prevent draws by giving Black a non-integer score bonus (this is
called Komi, [http://senseis.xmp.net/?Komi);](http://senseis.xmp.net/?Komi\);)
By definition of "perfect play", a Black perfect player either always wins
games, or a White perfect player always beats him.

~~~
pmontra
Your reasoning is correct but the bonus score (komi) is given to white, which
is second to move and is behind in the race at building territory (the thing
that gets scored).

------
WestCoastJustin
Wonder when we will see AI tutors appear? Seems like there is a massive
potential to train an AI super player then have it teach you. Does anyone know
of such tools yet? Guess there are probably many AI augmented things we use
all the time (Google Search) but just don't know it. Fascinating to think.

~~~
burkaman
There was a recent post here about a new LiChess feature that identifies
mistakes in your (or someone else's) games, prompts you to look for a better
move, and tells you the best move if you can't figure it out. I think this is
a form of AI tutoring.

[https://news.ycombinator.com/item?id=13241669](https://news.ycombinator.com/item?id=13241669)

~~~
mormegil
The same feature has been in GNU Backgammon (and other programs, I guess) for
a long time. Also, it allows you to request you or the AI get good or bad dice
rolls, which is an interesting training option.

------
nullnilvoid
It seems that AlphaGo has got even better since when it defeated Lee Sedol.
This time, AlphaGo is undefeated against other top players. However, it also
gives other players a chance to practice against AlphaGo and finds weaknesses
in AlphaGo, which is super important for any competition.

~~~
21
It would be super-weird if AlphaGo became worse that the previous version.

~~~
daveguy
Neural networks have a bad habit of regressing. When training continues they
tend to replace earlier skill with newer skill.

~~~
21
The team would surely detect that. In the original paper they even mention
this issue, and as a safe-guard they also test later networks against previous
versions.

~~~
daveguy
True, they are aware and monitor for it. Just pointing out that it's not a
super weird thing to happen with a NN. In fact it's a super normal thing that
has to be continuously monitored.

------
oconnor0
> Go is regarded as the most complex board game ever invented

I'm not sure that's an accurate statement.

~~~
btilly
How about the following statement?

"There is no widely played game for which an Elo rating system shows a larger
measurable range of skill than Go."

This is a precise and measurable statement. The standard deviation of the Elo
system used in Chess is 200, and the range from the best humans to rank
amateurs is 14 times this standard deviation. When the Elo system was adapted
to Go, the standard deviation was set to 100, and the range is 29 times the
standard deviation.

The possible skill range, of course, measures something how complex the
possible strategies are. And Go trounces every other game by this measure.

I think that gives Go a pretty defensible claim to "most complex strategy of
any popular board game".

~~~
fryguy
Complex is being used in a different context here though. You are meaning
complex as in strategic complexity. The parent (I think) is using it in terms
of the games rules. Go is pretty simple in principle: place stones on the
board one at a time, and remove enemy pieces completely surrounded by your
own. But it's fairly complex to know what exactly constitutes being
surrounded. Chess has 6 different kinds of pieces that have different
movements, and different kinds of rules depending on the state, and promotion
at the end, castling, and en passant. Checkers is a much simpler game because
there's only 2 kinds of pieces, and 2 kinds of moves. Strategically, Chess
isn't leaps and bounds more complex than Checkers since Carlsen has an Elo
rating of 2800 in Chess, and Tinsley had an Elo rating of 2700 in Checkers and
I believe it's the same deviation of 200.

~~~
btilly
Checkers has adjusted its Elo system over time. I believe that it now matches
chess except with a floor of a 1000 rating. From
[http://icheckers.net/ratings/](http://icheckers.net/ratings/), the current
ratings range goes from 1000 to 2297 among active players, and up to 2510
among inactive players.

Even if you use the larger range of inactive players, that's a range of 7.5
times the standard deviation. Which is considerably less than chess.

~~~
fryguy
Tinsley isn't on that list, and I saw it claimed that his rating was 2700:
[https://en.wikipedia.org/wiki/Marion_Tinsley](https://en.wikipedia.org/wiki/Marion_Tinsley)

Chess has a floor of 1200 for novices. So I'm not sure your point.

~~~
btilly
The Elo system for Checkers has changed over time. Tinsley had a rating around
2700, but he would not get that rating today.

Chess ratings from the USCF have a floor of 100. What you are thinking of is
that they start people at a provisional rating of 1200 and then let them drift
to where they belong. But a person who just knows how the pieces move will
quickly head down towards that floor.

------
emj
The Master bot was discussed a bit in on Google Deepmind 2016 round up:
[https://news.ycombinator.com/item?id=13312219#13314592](https://news.ycombinator.com/item?id=13312219#13314592)

~60 wins atm.

------
rdlecler1
I'd like to see if a small team of the worlds best go players could beat AG.

~~~
codehotter
How much stronger is a team of top players than its strongest member?

I wonder what the best way of coordination would be. Perhaps they can identify
several promising lines and each player chooses one variation to calculate
more deeply.

From my own experience playing (as a weak amateur), I feel I'm rarely able to
think so systematically -- often ideas I discover while contemplating one line
are tried in entirely different variations, so it would perhaps take some
getting used to before a team can function optimally.

~~~
tel
There's a very interesting book called The Go Consultants
([http://www.slateandshell.com/SSJF003.html](http://www.slateandshell.com/SSJF003.html))
which describes how professions have worked together during an extended time
consultation game (2 v 2).

------
natosaichek
I'm pretty sure this[1] is the archives of the bot's play. You can download
the SGF files of the matches from there and view them online at EidoGo[2]

[1]
[http://www.gokgs.com/gameArchives.jsp?user=Master](http://www.gokgs.com/gameArchives.jsp?user=Master)

[2] [http://eidogo.com/upload](http://eidogo.com/upload)

~~~
igravious
Marcel Grünauer over on LifeIn19x19 has made a compilation of all 60 of
Master's games. Found it just then! (A zip archive of 60 sgf[1] files) I
_think_ that's all of them as of now.

[http://lifein19x19.com/forum/viewtopic.php?f=18&t=13935](http://lifein19x19.com/forum/viewtopic.php?f=18&t=13935)

[1]
[https://en.wikipedia.org/wiki/Smart_Game_Format](https://en.wikipedia.org/wiki/Smart_Game_Format)

ps: The Japanese term for a game record for the game of Go is kifu:
[https://en.wikipedia.org/wiki/Kifu](https://en.wikipedia.org/wiki/Kifu)

Hence [http://gokifu.com/](http://gokifu.com/) which tracks all pro games (and
then some, I think)

~~~
natosaichek
Ah! Thanks for your evidence in support of cunningham's law[1]. I'm delighted
to have access to the real deal.

[1]
[https://meta.wikimedia.org/wiki/Cunningham%27s_Law](https://meta.wikimedia.org/wiki/Cunningham%27s_Law)

------
amelius
Did any team reproduce the AlphaGo technology yet?

~~~
osti
Here is the facebook version:
[https://github.com/facebookresearch/darkforestGo](https://github.com/facebookresearch/darkforestGo)

And here's another public attempt, not sure about the progress.
[https://github.com/Rochester-NRT/RocAlphaGo](https://github.com/Rochester-
NRT/RocAlphaGo)

------
sweetjesus
if their goal was to reveal what they were doing to the world, this is a fine
way to go about it and build some anticipation and get a PR splash.

if their goal was to train the best go-bot, they could have had it play not to
win but to go down to the wire with very strong players but frequently lose.*
Experienced players might have identified it as a bot, but would have
dismissed it as "not good enough yet"

*especially as per the recent story that children don't learn when they win, so trains yourself without training your opponents :)

------
known
Can AI beat Fed?

------
chrisp_dc
Anyone know the origin of the name they chose? Did they name their bot after
the rapper Percy Miller?

~~~
21
I seriously doubt the Percy Miller connection.

AlphaGo is the best player in the world, so a name like Master fits it :)

They also probably didn't want to choose a human looking name which could
mislead people, and instead chose one which could presumably be either of a
human or an AI.

------
overhere2000
We will need a Kobayashi Maru exam for this.

------
onetimeacc33
..wonder when they're gonna "reveal" their secret orwellian its-all-for-the-
good-of-humanity retina-scan-ml-trainingcamp(yes, reads almost like german)
project from that UK-ish part of the world..

