

Teaching Deep Convolutional Neural Networks to Play Go - urish
http://arxiv.org/abs/1412.3409

======
gamegoblin
I would be _extremely_ interested in seeing this used as a pruning function
for a state-of-the-art MCTS (Monte Carlo Tree Search) Go engine.

As it stands, your generic MCTS algorithm expands a game tree of nodes, and
gives more attention to more promising branches, but it still must give
attention to other branches to find out if they are promising or not
(exploration vs. exploitation).

In the paper, they get the right move (right as defined by what an expert
human would do) 44% of the time, but they also say the right move, if not the
#1 choice, is often in the top few choices. According to their graph it's in
the top 10 choices about 80% of the time, and in the top 30 choices about 98%
of the time.

If in MCTS you could prune the branching factor of your tree search down from
300+ to ~30, that could be huge.

=====

I'd also be interested in seeing it used as the playout function of an MCTS
engine.

As it stands, most playout functions use random, or random+quick heuristic to
playout thousands (or millions) of random games to rate a position. I imagine
if you used this, which can output an entire probability distribution of
moves, you could do significantly better than random with a fewer number of
games.

~~~
jpfr
Integration in MCTS should be straightforward. And I would be surprised if the
author's hadn't done it already. Normally, no actions are pruned away per-se.
Instead, the available actions are initialised with a utility value [1] from
an external oracle, i.e. the neural net. From then on, the normal MCTS
procedure continues until the time or memory runs out.

[1] ..and a faked visit-count. So the algorithm beliefs that the action was
already evaluated n times, which resulted in the given mean utility.

~~~
gamegoblin
I don't think they've done it yet -- in the conclusion they say:

    
    
        The most obvious next step is to integrate a 
        DCNN into a full fledged Go playing system. For
        example, a DCNN could be run on a GPU in parallel with
        a MCTS Go program and be used to provide highly quality
        priors for what the strongest moves to consider are. Such
        a system would both be the first to bring sophisticated pat-
        tern recognitions abilities to playing Go, and have a strong
        potential ability to surpass current computer Go programs.
    

I agree that the integration should be exceedingly straightforward. I've
written MCTS implementations (though not a Go implementation -- I used it on
Connect 4 and other easy-to-code games), and it seems like you'd just plug it
into your already-existing bias function.

The authors didn't mention my idea about using the probability distribution
output from the network to guide the random playouts, which I would also be
interested in.

------
hyperpape
There's a thread on this paper on the computer go mailing list. It's a quiet
list, but features authors of several of the best programs.

[http://computer-go.org/pipermail/computer-
go/2014-December/0...](http://computer-go.org/pipermail/computer-
go/2014-December/007010.html)

------
sanxiyn
In the thread discussing this work at computer-go list, Aja Huang wrote:
"Chris Maddison also produced very good (in fact much better) results using a
deep convolutional network during his internship at Google. Currently waiting
for publication approval, I will post the paper once it is passed."

[http://computer-go.org/pipermail/computer-
go/2014-December/0...](http://computer-go.org/pipermail/computer-
go/2014-December/007017.html)

------
romaniv
I wish there was something using DNNs that users could train and use without
writing code or connecting to some centralized service. It would help to get
the "feel" for how good these algorithms are. Something moderately general-
purpose, I mean. However, it seems that mapping them to the problem domain is
an art in itself.

~~~
modeless
Mapping DNNs to a problem domain is an art, but it is easier than the "feature
engineering" required to use other forms of machine learning.

------
hurin
For those (like me) wondering how strong Feugo is, I found this:
[http://www.gokgs.com/graphPage.jsp?user=fuego19](http://www.gokgs.com/graphPage.jsp?user=fuego19)

------
w1ntermute
An interesting Wired article on the topic: [http://www.wired.com/2014/05/the-
world-of-computer-go/](http://www.wired.com/2014/05/the-world-of-computer-go/)

------
gnidan
It should be pretty neat to see where this goes. I'm particularly curious what
it would take to bridge the gap from 4-5 kyu play to dan-level rankings. I
believe there are computer go programs that play around 5-dan level (e.g.
Zen19D, CrazyStone), using Monte Carlo, of course. Will the techniques in this
article be able to scale up to that?

------
rixoff
Happy to see that this paper is freely available for all to download and read!

