
Watch an AI bot play Go [video] - r-u-serious
https://www.facebook.com/Engineering/videos/10153621562717200/
======
mark_l_watson
Very cool. I wrote a Go playing game for the Apple II which I sold.

I have recently become interested, again, in Go programming. I have started
setting up a framework for using GnuGo to generate training data for a deep
learning network. I am starting with a tiny 7x7 board just to see if the
general approach works.

If it works (at all) then I will post the entire project to github.

~~~
r-u-serious
Cool. I recently started working on a command line Go program. Mainly for
learning Haskell and without any AI, yet.
[https://github.com/tsujigiri/gosh](https://github.com/tsujigiri/gosh)

~~~
tinco
Fun! I have a Haskell Go web app. It has a super experimental AI that can't
actually win that I only spent a couple of hours on:
[https://github.com/tinco/ergo](https://github.com/tinco/ergo)

~~~
r-u-serious
I'll check that out!

------
vessenes
What's interesting to me about this is the visual pattern recognition --
rather than just scaling up a giant Monte-Carlo search, playing like
(presumably) humans do, searching for patterns, is very cool.

I was just this weekend going through my go library, and most of the books I
have focus on precisely this sort of pattern training. In fact, there's a
whole branch of study in go called in Japanese "Tesuji", essentially brilliant
moves that come from bad-looking patterns.

The idea is that you lull your opponent into complacency with your bad-looking
pattern, then "BAM", a surprise comes out. My guess is that an RNN could be
discovering new tesuji, which would be super cool. If the RNN can look ahead
deep enough, then I would think that's a path forward against strong players.

There's also the reverse, Anti-suji, essentially things that look brilliant,
but are really terrible.

------
fitzwatermellow
There is something captivating about watching two master-AI's battling it out
on the Go board, it's almost Art in a way ;)

Facebook's AI lab will probably reveal more technical details at NIPS 2015.
Here's the workshop on Reasoning, Attention, Memory (RAM) they are organizing
that will seek to address some of the challenges around advancing from
classification and prediction tasks to simple cognition:

[http://www.thespermwhale.com/jaseweston/ram/](http://www.thespermwhale.com/jaseweston/ram/)

------
samvsuel
Related article: [https://code.facebook.com/posts/1478523512478471/teaching-
ma...](https://code.facebook.com/posts/1478523512478471/teaching-machines-to-
see-and-understand-advances-in-ai-research/)

------
RBerenguel
From the short description, it looks like a project from Google about training
a deep network to play the game. The resulting bot was quite strong (or at
least, quite strong for not using Monte Carlo tree search, at least). Without
any more details though, it's hard to tell.

~~~
xianshou
Here is the paper for Google's DNN-based move evaluation in Go:
[http://www.cs.toronto.edu/~cmaddis/pubs/deepgo.pdf](http://www.cs.toronto.edu/~cmaddis/pubs/deepgo.pdf)

~~~
RBerenguel
Thanks, I wasn't sure where I had the reference.

------
weatherlight
"strong human player." Like 1 Dan strong or like 6 kyu strong? It's a
meaningless statement.

~~~
tel
If it's comparable to other computer go ais then it's low dan.

------
donatj
Go should just be labeled Gogame in contrast to Golang.

~~~
cjbprime
> Go should just be labeled Gogame in contrast to Golang.

At the same time, there's something amusing about the idea that a six year-old
programming language should cause a 2000 year old game to change its name.

~~~
jordigh
It's also kind of weird we call it by its (newer) Japanese name instead of
weiqi.

~~~
NhanH
There was a reason for that though. Before the latter half of the 20th century
Japanese go scene kind of dominated for several centuries (thanks to official
shogun sponsor).

~~~
hyperpape
More importantly, until the past few decades, Japan did the majority of the go
outreach in the West. That's changed a bit, and thanks to Korean influence,
it's more common to hear people saying "baduk".

------
jgalt212
He used Monte Carlo methods, not DNN, but a nice talk on Computer Go by Zach
Tellman

[https://www.youtube.com/watch?v=v5dYE0CMmHQ](https://www.youtube.com/watch?v=v5dYE0CMmHQ)

------
jordigh
Btw, why is go-the-game capitalised? We don't capitalise Chess, do we?

~~~
samvsuel
[http://english.stackexchange.com/questions/8126/is-the-
game-...](http://english.stackexchange.com/questions/8126/is-the-game-go-a-
proper-noun-what-about-checkers-or-chess)

~~~
ant6n
that's a good answer: capitalize to disambiguate

------
AndrewKemendo
_After a few months of playing ... it 's as good as a very strong human
player._

This is potentially a really big breakthrough if true, as Go is much harder
than Chess computationally. Nobody knows about Go in the West though so don't
expect any headlines.

~~~
hyperpape
This is hard to evaluate. Explicit metrics would be good, but there's also a
definitional point. Depending on who you ask, existing AI programs are as good
as a very strong human player. Others will scoff at that characterization. The
problem is what "very strong" means. Precisely, current programs can beat many
professionals with a 4 stone handicap.

Making good analogies is hard, but here's mine: current go programs are
probably somewhere in the range of runners running a 2:45 marathon. They're
beyond what most people can hope to reach, perhaps even with intensive
training, but they're not world-class. In fact, they're not just below world-
class: there are thousands of people better than them who are themselves not
yet world class. (Context: I play go and run at a level far below the ones
quoted above, but know people who play/run at that level. And I've seen those
people get their butts kicked by stronger players/runners).

~~~
RobertoG
But, surely, they are going to improve the system, and fast.

I would bet that in a few years is going to be clear who is superior, the
machine or the human.

~~~
tel
That's been happening for a while now. My understanding is that there was a
major wall many go AIs were stuck on prior to the invention of monte carlo
tree search (essentially, a way of evaluating a better play objective
function) and they're improving now again. There appears to be reticence to
evaluate the strength of go programs, though.

Here's a chart of current ratings of popular engines on online forums. The
rating system is one of "kyu" and "dan" going from 25k to 1k to 1d to 9d.

[http://senseis.xmp.net/?KGSBotRatings](http://senseis.xmp.net/?KGSBotRatings)

Informally there's another rating system the "pro" rating system which is a
bit ceremonial but also broadly assumed to be often higher than the amateur
dan rankings.

------
hellbanner
Does "To help teach systems how to plan, Facebook's AI research team has
created an AI bot to play the board game Go. " actually teach AI how to "plan"
at anything beyond board games?

------
deepnet
This Facebook Video suggests it has had months of either training or
development, which sounds might be it learning by playing, reinforcement
learning in this space could be very interesting - more details would be good.

The two breakthrough papers using Deep Conv Nets to play Go - trained on
Expert Games to predict the Experts Next Move.

This is made possible by the huge archives of expert play from online Go
servers that have reached Big Data sizes in the last few years.

Teaching Deep Convolutional Neural Networks to Play Go - Clark & Storkey
Edinburgh Informatics 2015
[http://arxiv.org/abs/1412.3409](http://arxiv.org/abs/1412.3409)

Move Evaluation in Go Using Deep Convolutional Neural Networks Maddison,
Huang, Silver, Sutskever Deepmind 2015
[http://arxiv.org/abs/1412.6564](http://arxiv.org/abs/1412.6564)

David Silver has work on Reinforcement Learning and Go: Reinforcement Learning
of Local Shape in the Game of Go Silver et al 2007.
[https://scholar.google.co.uk/citations?view_op=view_citation...](https://scholar.google.co.uk/citations?view_op=view_citation&hl=en&user=-8DNE4UAAAAJ&citation_for_view=-8DNE4UAAAAJ:9yKSN-
GCB0IC)

Go is scored by occupying and surrounding territory and the opponents pieces
taken.

The current State of the Art programs use Monte Carlo Tree Search with each
move being evaluated using random playouts - this is well suited to Go as it
reasnoble estimate for comparison of the territory that will result from a
move.

As the Go search tree branches a lot (~200 branches per play) tree search
takes some time and cannot be exhaustive.

The Expert Move Predicting Convnets provide very good moves very quickly, and
can provide and provide probabilities the expert would move for each square in
a single forward pass.

Most of the current development now centers around using the Convnets to prune
the search trees for the random playout MCTS engines.

Go is useful from an AI perspective because it is a huge state space and the
patterns are subtle with long term implications. No computer can yet beat the
best humans - so it is an area where natural intelligence beats artificial.

Yet the convnets are not doing planning, only predicting what an expert human
would do next, the planning is implicit in the dataset but not a part of the
convnet itself (though the internals of deepnets are mysterious so who knows).

The Computer Go mailing list is very lively and full of the best program
creators with regular computer tournaments against each other.
[http://computer-go.org/mailman/listinfo/computer-go](http://computer-
go.org/mailman/listinfo/computer-go)

The Facebook video shows GnuGo is the opposing player, this is not the best
computer player and suggests Facebooks player is not as strong as the Convnets
as yet - it also does not detail how deep the playouts of the tree search are,
i.e. what level GnuGo is playing at, so this cannot really be compared to
recent work.

Hopefully Facebook will publish more details soon - if they are learning
through play then their results may well be interesting.

Remi Coulom's excellent slides detail the Monte Carlo revolution in Go
programs. [http://www.remi-coulom.fr/JFFoS/JFFoS.pdf](http://www.remi-
coulom.fr/JFFoS/JFFoS.pdf)

AFAIK MCTS with Random playout and hand coded heuristics are still SOTA.

When I visited in the summer Edinburgh Uni Informatics Department convnet Go
was in active development.

