
Why AlphaGo Is Such a Big Deal - retupmoc01
https://www.quantamagazine.org/20160329-why-alphago-is-really-such-a-big-deal/
======
pmichaud
I don't think this is really why AlphaGo is a big deal. He's not wrong about
the ways that it works vs. DeepBlue, but there's a much simpler way to
understand why AlphaGo is a big deal.

(I am not an AI researcher, but I work with AI researchers, like by putting on
training programs and recruiting for them, and working out of the same office,
and this is a summary of [my understanding of] their discussions about this
topic.)

AGI is a Hard Problem (TM), and no one really knows how hard it is. General
intelligence is made of algorithms that are somewhere on a spectrum between
dizzyingly complex math that we have decades+ more work to develop, and simple
components that someone in their basement could stumble upon at any moment if
only we had a couple key insights. (Sorry for the gross oversimplification.)

Before AlphaGo there was a whole area of probability space that seemed
plausible--a whole area of complexity that had yet to be worked out in detail.
AlphaGo proved that more or less current AI tech was sufficient for that
problem (current meaning one incremental step forward, as opposed to some
revolutionary insight). That's why they "moved the field forward by 10
years"\--by proving that the work that AI researchers basically expected to
have to do over the next 10 years is not necessary.

Said another way, we're closer to the "simple components" end of the spectrum
than we collectively thought. AlphaGo proved that, and that's why it's a big
deal.

~~~
partycoder
Have you played go? When you play Go you start to appreciate how complex it
is, how strong Lee Sedol is and what this achievement means.

I could play my entire life and never achieve the level of play Lee Sedol has.
AlphaGo surpassed that level in a matter of _months_.

As a matter of fact, many professionals that play the game for a living as
their main occupation, since they're kids, will never achieve the level that
Lee Sedol has.

Also from here, they can add more infrastructure and feed the bot with more
training data and let it train for a longer time, and it will be distancing it
more and more from human level play.

Then, this task specifically is not just any random task... it's one of the
most complex games, one the oldest games, and is also professionally played
and studied.

Then, AGI is not a requirement for disrupting our world. You can have ANI for
one specific task that is performed by millions of humans and you have
disrupted the world enough to make it a problem.

~~~
devishard
> I could play my entire life and never achieve the level of play Lee Sedol
> has. AlphaGo surpassed that level in a matter of months.

In one, very tiny area, sure. But step outside the very specific task of
playing winning moves in go, and Lee Sedol wins every time. AlphaGo can't
explain its moves to someone learning go, let alone cognitive tasks we take
for granted like moving feet in a way that produces transportation. AlphaGo
can't even do things that other programs can do, like play chess.

Sure, you can "disrupt" with Monte Carlo algorithms _like_ AlphaGo in the San
Francisco startup sense of the word "disrupt", but that's also not new. AIs
have been doing that for years in finance, genetics, chemistry. We've been
able to write code that does one thing well for decades--what we lack is code
with breadth of application, and AlphaGo doesn't even make a dent in that.

~~~
Joeri
> AlphaGo can't explain its moves to someone learning go, let alone cognitive
> tasks we take for granted like moving feet in a way that produces
> transportation.

I'm pretty sure lee sedol cannot explain his own moves either. Yes, he can
rationalize why that move is a good move, but he won't be able to explain the
complete reasoning process that led to that move, because it relies on
intuition. He cannot train a random person to play at the level of lee sedol.

If you look at the human brain, every part learns to do just one thing.
Alphago is like having just the part of the brain that plays go. Moving from
there to something more general is a question of assembling the parts.
Building a biped robot that walks and plays go is not a hard problem.

The fundamental question is whether us humans are more than the sum of our
parts. If you add task-specific AI bits together, will you at some point have
something that becomes a person, or will it always be a collection of
features?

~~~
gcp
_I 'm pretty sure lee sedol cannot explain his own moves either. Yes, he can
rationalize why that move is a good move, but he won't be able to explain the
complete reasoning process that led to that move, because it relies on
intuition. He cannot train a random person to play at the level of lee sedol._

I think you're wrong here. I don't see why that wouldn't be the case?

The "intuition" just means he arrives at the move quickly without consciously
going through the thought process, but why wouldn't he be able to reconstruct
it?

~~~
mateuszf
He is right. It is a typical thing that masters can't explain why their
actions are that good. That's because their brains are just neural networks
which are trained by so many iterations of learning.

------
colllectorof
Like most articles about AlphaGo this one does not mention two (uncomfortable
for many) facts.

1\. AlphaGo doesn't _just_ use neural networks. If it did, it would almost
certainly loose 0-5. It also uses Monte-Carlo tree search, which is an
algorithms that made a huge leap in the strength of Go bots. The rub is that
MCTS is a flavor of "brute-force" algorithm. What does that tell you about the
nature of Go?

2\. None of the individual components in AlphaGo are theoretically novel. In
fact, all of them are very well-known: conv. nets, MCTS, data-mining,
reinforcement learning. What does that tell you about the nature of this
particular game-playing AI compared to its many brethren?

(By the way, I wonder how many of the journalists praising Google for Alpha Go
even heard of Rémi Coulom. Shouldn't he be at _least be mentioned_ somewhere
in all these articles?)

~~~
Houshalter
I don't see anything wrong with them using existing tech. Most technology is
iterative. It's like saying going to the moon isn't impressive because rockets
are 20 year old tech.

NNs have shown a lot of promise in the last few years. They are just starting
to make their way to new applications like Go playing. So in that sense
AlphaGo is a sign of things to come.

~~~
colllectorof
_> NNs have shown a lot of promise in the last few years._

Which is interesting in its own right, since ANNs have been the subject of
research for more than half a century. If anything, all their ups and downs
should make people more cautious about bold predictions.

~~~
gcp
Monte Carlo Go was also known and researched for a long while before the
breakthrough came in 2006 (how to combine it with tree search).

------
unabst
Using words such as "intuition" just makes the line between Strong and Weak AI
blurrier, but that line is absolute.

Think of it this way. Strong AI would lose to Weak AI at Go. So Strong AI is
not in AphaGo's evolutionary path. Strong AI is the kind that authors and
expresses things. Weak AIs are computational marvels.

The brilliance of Strong AI would be in its capacity to build targeted Weak AI
systems. And that is how Strong AI will beat any Weak AI system.

If Strong AI is what we have, then just look at our relationship with AlphaGo.
We created it, and it beat us at Go.

Also, science is unpredictable. So saying we're 10 years ahead of schedule is
not a scientific statement at all. Someone said it, and it was wrong, but they
weren't speaking for the thousands of scientists working on new ideas as
opposed to refining old ones.

~~~
pixl97
>And that is how Strong AI will beat any Weak AI system.

When humans build a calculator, do we say that the calculator beat humans at
math, or humans beat calculators because we create them? Not really, they are
just a tool. The problem with humans and calculators is the interface is
rather clunky. You can't directly interface with the digital inputs of the
calculator, but instead do so millions of times slower than a computer can.

With strong AI this is not apt to be the case. Yes, it will create weak
'computational' AIs to do some tasks very well, but the speed that it will be
able to interface and classify results at will be much faster than we are able
to obtain. Where you and your calculator are very different things, ASI and
it's calculators will not be.

~~~
unabst
> humans beat calculators because we create them

The new calculator will beat the old calculator. Humans don't beat calculators
themselves. But only humans have the capacity to improve on calculators. And
this is truly all we've been doing with science and technology ever since day
one. And yes, they're all just tools. AphaGo is just a tool. It told the dude
moving the pieces where to move the pieces to beat the world champion who was
moving pieces without any help. AlphaGo2 will be a better tool.

> much faster than we are able to obtain

Would that be that important though? Compare IBM Watson with the Jeopardy
champion, except, this time he gets to use Google. Watson has a direct line,
Ken Jennings does not. So what? Calculators are the same thing. We don't
benefit from a direct link, at least at that level. It gives us the answer,
and we can read it. We only want the answer anyway. Having to download all the
details is redundant. The mind reading interface is what is clunky, when we're
already wireless. Seeing is enough.

> ASI and it's calculators will not be.

Right, so you're switching to hardware now. Artificial human enhancement is
the go-to answer many will give you here, but it's a bit too sci-fi for me.

But regardless, the strong-weak divide still stands. We would create tools to
solve the problem. And this is where strong AI and calculators will differ,
regardless of what other similarities they may have, physical or otherwise.

------
return0
I like that he doesn't use the typical AI jargon. I think its time to stop
using science-fiction-writer terms to talk about our layered nonlinear unit
systems. There is so much talk about "General AI", "Strong" and "Weak" AI, and
also about "Intelligence", but none of these are well-defined enough to be
useful. These are just words, not things that exist in the world out there.
What we have here is a system that generalizes in a way that is human-like.
It's a mystery why this looks so human-like to us, given that neural networks
are only barely resembling of actual neural networks, and that is a problem.
Reading the neuroscience of plasticity, it's still not clear whether learning
takes place via synapse modifications. What these amazingly capable neural
networks do, however, is to bolster the idea of connectionism in neuroscience,
which is an interesting, given that the relationship between the two started
the other way around.

P.S. Michael Nielsen must be one of the very few people who can explain any
concept in almost absolute clear and simple terms.

------
lossolo
It's a big deal because it makes better decisions than human in decision space
that cannot be brute forced to get all solutions. Life is also that kind of
decision space, which you can't brute force to make best decision there is.
AlhpaGo has better analytical "skills" in Go than human has, without knowing
all the results of all the moves, just like human.

~~~
astazangasta
No, life is not like Go. You don't choose one of 64 squares to play your next
stone on in life. The state space is infinite. This is why AI will continue
beating humans at games but be totally inept at, say, tech support.

~~~
inopinatus
Given that many human-operated tech support departments _are_ inept, given
their demonstrable inability to deal with issues and situations for which they
lack a prior decision structure and/or script, your last point actually
(albeit inadvertently) suggests that AlphaGo has a shot at a CSR job.

~~~
lossolo
Exactly, because it has a shot. Writing that it doesn't have a chance is just
effect of lack of knowledge in ML field. You can see some work that was
already done in this research paper, before AlphaGo success.

[http://arxiv.org/pdf/1506.05869v1.pdf](http://arxiv.org/pdf/1506.05869v1.pdf)

------
andreyk
A good summary on the differences between DeepBlue and AlphaGo, and one that
recognizes the latter is part of a broader trend of successful applications of
deep neural networks. I thought this was pretty well written and accurate.

~~~
michael_nielsen
Thanks. It's for a general audience, so I gloss over many details, but tried
to do so in a way that remains broadly accurate. The Monte Carlo tree search,
in particular, involves many finnicky details that I couldn't get into in this
kind of essay.

~~~
Kiro
Thanks for writing this. I've been looking for an article to send to my
friends who don't understand what's so special about AlphaGo. I think you
nailed it.

------
YeGoblynQueenne
Reading this article I find myself thinking again how unhappy I am that a
neural network won Go. I am unhappy because neural networks are not smart.
They're the epitome of artificial stupidity: they learn, but they don't
understand anything that they learn. They can't take what they learned and
apply it to another domain, not without training again. And every time you
train one in a new thing, it forgets all about the old thing. They got
intelligence alright, but it's the dumbest kind of intelligence possible,
mechanical, unaware of itself and devoid of all purpose other than what we
give it.

And what about ourselves? Building AlphaGo is a feat of human intelligence,
but how does it improve that human intelligence itself? How do we become
smarter? We can build machines to solve problems that we don't understand, but
they solve them in ways that we still don't understand. Except- we're not even
trying anymore. We've given up. Knowledge is hard. Logic is hard. Just throw a
bunch of GPUs at the problem and make it go away.

Far from this being a promising time for artificial intelligence, I fear we're
about to see a great dumbing down as funding is cut to everything but neural
networks research. A kind of AI summer, as it were.

~~~
azag0
> neural networks are not smart

That's like saying computers are not smart. It's meaningless. Computers are as
smart as the algorithms they run and neural networks are as smart as the
neural connections within them. Neural networks are models of computation as
much as current computer architectures are.

As far as we know, neural networks indeed are at the core of human
intelligence. Human brain is an extremely complicated neural network with
plenty of side mechanisms, many of which we possibly do not know about yet.
Nonetheless it's unlikely that the underlying building blocks are not neural
networks.

What is probably several orders of magnitude more complex than uncovering the
static architecture of the human brain, is understanding the way it is
trained. Note that this is the case with artificial neural networks as well.

I doubt that we will ever understand human intelligence "from above". It
doesn't seem implausible to me that there is some general complexity theory
law which prohibits intelligence "understanding" itself.

~~~
YeGoblynQueenne
>> As far as we know, neural networks indeed are at the core of human
intelligence.

Right.

Quick intro to neural models, for the only very slightly mathematically
inclined.

Neural networks solve this equation:

Y = W ⋅ X

Where X a matrix of inputs to a function ƒ, W a matrix of its coefficients and
Y a matrix of its outputs.

Once that's done once for all N instances of X in our dataset, a time period
which we term an epoch, Y is used to calculate a measure of error, typically
mean squared error:

MSE = 1/N ∑ (Y' \- Y)²

Where Y' the true value of Y that we're trying to approximate. This is
repeated until the cows come home, or until the network has converged,
whichever comes home first (er, hint).

And that's neural networks in a nutshell. There's nothing much more
complicated than that (er, in theory), there's no magic and certainly nothing
"like the brain" in them at all. They're just optimisation over systems of
functions. We learned how to solve systems of functions with matrix
multiplication when we were at high school. I'm pretty sure we did - I've
personally repressed the memory but I'm pretty sure I did once have it.

Why do people insist on calling them "neural"? Who the fuck knows. Way back in
the 50s, the perceptron was based on a model of neural activation, where a
neuron "fires" when its electrical charge crosses over a threshold. This was
originally represented with a binary step function (the very one used in
Perceptron classification). That was, at the time, deemed to be "a good model"
of neural activation. A good model, my foot. Soon people moved on to sigmoid
functions which are "a better model" of neural activation. Whatever.

I've also heard it that, say, conv nets (convolutional neural networks) are
"like the brain", presumably because if you squint at the layer connection
diagrams long enough, conv nets start to look a bit like idealised models of
the parts of the brain that handle vision. But if you squint long enough,
everything starts to look like everything else. Hell, you might suggest that
neural networks are based on the effin Tree of Life from Cabala, and Geoff
Hinton is Alistair Crowley reincarnated.

The whole "brain" thing is just an abstraction. You should not read too much
into it. And don't listen to the people who try to convince you neural
networks are magic- they're not. It's high school maths and a lot of very
expensive processing power.

P.S. If my notation doesn't make sense to you, you may need to use a different
font in your browser; sorry about that.

~~~
p1esk
Brain has a large number of connected neurons. Which, by definition, is a
"neural network".

The artificial neural network model you described works more like a brain
than, say, a traditional von Neumann computer. It works less like a brain
than, say, HTM, which is another type of an artificial neural network. There
are even more biologically plausible NN models (e.g. those used in HBP).

The bottom line is, a large number of connected neurons is at the core of
human intelligence. Everything else is just details.

~~~
YeGoblynQueenne
>> The bottom line is, a large number of connected neurons is at the core of
human intelligence. Everything else is just details.

That's a great example of the magical thinking that prevails in discussions of
neural networks today.

You only need to put your ducks in a row to figure out why it makes no sense.
"The brain" you say "has lots of connections". "Therefore, if something has
many connections, it's like a brain". I hope that raises a big red flag with
bells on for you, 'cause it sure does for me.

Spaghetti carbonara looks a lot like a brain (it's made of fibers and it's an
off-white, sticky mess). I've never known it to be intelligent, though maybe
it's just too smart for me, eh?

------
SonicSoul
I really liked Jeff Atwoods explanation [0], and the DeepMinds CEO talk
referenced in his post [1]

[0] [http://blog.codinghorror.com/thanks-for-ruining-another-
game...](http://blog.codinghorror.com/thanks-for-ruining-another-game-forever-
computers/)

[1]
[https://www.youtube.com/watch?v=rbsqaJwpu6A](https://www.youtube.com/watch?v=rbsqaJwpu6A)

~~~
thomasahle
I'm not sure Atwoods whole point about computation power:

 _If AlphaGo was intimidating today, having soundly beaten the best human Go
player in the world, it 'll be no contest after a few more years of GPUs
doubling and redoubling their speeds again._

In the AlphaGo paper they said their cluster version (with the 170 GPUs
Atwoods mention) were the limit of what hardware could do for their
algorithms. They tried using much larger clusters, but it didn't really
improve performance.

This actually fits his point that for chess, most of the performance gain over
the last ten years has come from better algorithms. In particular for chess,
better tuning of variables, through projects like fishtest
([http://stockfishchess.org/](http://stockfishchess.org/))

It seems like many of these problems can only be solved using algorithms with
require a certain amount of computing power. However once we've reached that
particular amount of power, further progress really lies in the algorithms.

------
sgt101
The proof of the pudding will be in the applications that come from this
_specific_ line of research. I predict that there will be folks pointing to
anything with an activation network, or a search algorithm, and shouting "look
it works like alpha-go".

What is the class of problems beyond go and closed world games which this work
attacks?

~~~
whitegrape
More like anything with a guided search algorithm will be said to be like
AlphaGo. In general I think the field may be warming up to the idea that
"whenever there is a randomized way of doing something, then there is a
nonrandomized way that delivers better performance but requires more
thought."[1] MCTS does exceptionally well at Go all by itself. When coupled
with something (in this case trained neural nets) that can give it better-
than-random guidance in exploring the state space, it's no surprise it does
even better.

[1]
[http://lesswrong.com/lw/vq/the_weighted_majority_algorithm/](http://lesswrong.com/lw/vq/the_weighted_majority_algorithm/)

~~~
gcp
_MCTS does exceptionally well at Go all by itself. When coupled with something
(in this case trained neural nets) that can give it better-than-random
guidance in exploring the state space, it 's no surprise it does even better._

The tricky part is that using the "best (rather than better) than random
guidance" in the Monte Carlo simulations makes the performance worse.

We don't understand very deeply why that is.

------
jonbarker
Would be interested to see a rematch between Lee and AlphaGo. The analogy
between tricking image recognition with specific artifacts and finding weak
points or flaws in other AI applications in the article is great; given enough
games I'm confident Lee will find the flaws and be able to reliably exploit
them.

~~~
xbmcuser
Alphago is not static either it also improves. Lee would improve his game by a
lot by playing Alphago regularly. But Alphago in the same time period would
probably improve a few fold. At the end of the day Alphago has probably
changed the game of Go for ever as it has already taught Lee other ways to
think about Go.

------
mrcactu5
I am concerned... OK the AlphaGo AI plays better than any human, but does it
understand why?

When I learn Go, there are all sorts of techniques such as joseki, life-death,
tesuji. AlphaGo implicitly learned these techniques, since it learned from
professional games, which are experts in those topics.

This is my problem with neural networks in general, since they do not discuss
how go boards divide into smaller problems or heuristics for putting them back
together.

to be corny, Go is really a teaching tool for how people think logically and
abstractly. It's not about teaching computers.

~~~
drjesusphd
> This is my problem with neural networks in general, since they do not
> discuss how go boards divide into smaller problems or heuristics for putting
> them back together.

This strategy is called "chunking", and is an adaptation humans have developed
to solve complex problems. There's no reason to artificially constrain AI to
use the same crutches we do.

------
nsxwolf
The problem I have being wowed by AlphaGo is that games like go and chess are
so inscrutable to me that I cannot tell the difference between a game played
by two strong players and a game played by a players making random legal
moves.

I have never won a game of chess in my entire life. Not against any human
player or any Commodore 64 chess program on the easiest setting. I don't
expect I ever will. I know how the pieces move and that's about it.

Watson playing Jeopardy was different. That I could "get", and that impressed
me.

------
vessenes
One quick nitpick on your well written and helpful article: it's easily
conceivable that Lee Sedol has reviewed on the order of 100k games.

I think your general point stands, AI learns slowly in terms of number of
iterations (consider how many times a five year old needs to be told the
pronunciation of an unusual word or what emotion a certain tone conveys vs
training a neural net).

~~~
kqr
> it's easily conceivable that Lee Sedol has reviewed on the order of 100k
> games.

...but highly unlikely. In the best case, that works out to something like
reviewing 10 _new_ games every single day. If a game takes 1 hour to properly
review, and a person needs 8 hours of sleep, that leaves just 6 hours in a day
to eat, manage hygiene, exercise, be with family, travel to competitions, play
yourself and do all kinds of other studies you need to.

~~~
vessenes
Okay, fair enough -- but it's definitely not 10k games. Asian pros work very,
very hard. It's like any sport; time spent matters, and the best players put
in a lot of time.

~~~
kqr
It's probably not 10k, but it's _on the order of_ 10k, i.e. somewhere between
10,000 and 99,999.

------
grumpopotamus
>To get over this hurdle, the developers’ core idea was for AlphaGo to play
the policy network against itself, to get an estimate of how likely a given
board position was to be a winning one. That probability of a win provided a
rough valuation of the position.

This part is confusing in my opinion. It makes it sounds like AlphaGo is not
using Monte Carlo Tree Search, which is mentioned earlier. However, this is
exactly what AlphaGo is doing, except with the policy network in place of
previous methods for initializing move priors, and the value network in place
of traditional methods for terminating and scoring simulations.

~~~
gcp
If I read their papers right, they don't actually use the SL policy network,
because it played worse when combined with MCTS than the RL one, even if it
was stronger without tree search.

(But you need the tree search to make a strong player)

------
ericjang
Coincidentally, Quora.com just had a knowledge prize contest to answer "How
Significant is AlphaGo?".

[https://www.quora.com/How-significant-is-the-AlphaGo-
victory...](https://www.quora.com/How-significant-is-the-AlphaGo-victory-for-
AI-and-technology-in-general-How-generalizable-is-the-success-in-this-one-use-
case)

I thought that many of the answers were quite insightful.

Disclaimer: I also wrote an answer, but the submission deadline for the
knowledge prize is over.

------
Houshalter
It's a big deal because AIs can now beat humans at all perfect information
games. And NNs and maybe even MCTS are applicable to non perfect information
games. Soon they will be beaten too. Then what? The list of things humans are
better at is narrowing.

------
graycat
As we see, the OP has

> We have learned to use computer systems to reproduce at least some forms of
> human intuition.

Hmm .... Let's examine that claim:

To set aside the game of Go, we start with an analogy: A human can walk 10
miles to town, and a car can help one drive the same 10 miles to town. But
what a car does to get to town is very different from what a walking human
does. Both approaches get to town, and the car is much better in various ways,
but this does not mean that a car walked like a human. To be clear,
specifically the car did not "reproduce" the _way_ a human gets to town; e.g.,
the car did not have two legs. Both the human and the car got to town? Yes.
The car "reproduced" how a human got to town on two legs? No.

Similarly, a motor boat does not "reproduce" some forms of _human swimming_.

This pattern of using science, engineering, etc. to get results comparable
with those of unaided humans is very old with many great examples from
especially the last 100 years.

Further, on "intuition", there are many tasks that humans do with human
intuition where science, engineering, and computing do comparably well, but
that does not mean that the science, etc. has "reproduced" human intuition.

Similarly, on the game of Go, AlphaGo got some game successes that humans get
via human intuition, but that does not mean that AlphaGo "reproduced" human
intuition, not in general and not even in the specific case of the game of Go.

Early in IBM's mainframe computing, in the common publicity, there was a
related claim that IBM's computers were "gigantic electronic human brains".
Those computers were no more "human brains" than Henry Ford's Model T was
_gigantic mechanical human legs_. A car is not human legs, and an early IBM
computer was not a human brain.

IMHO, there is an important point here: From the publicity about IBM's early
computers to the claim in the OP I quoted above, there are claims that
computing and computer science are _reproducing_ some of how humans do things.
But so far this claim is false, seriously false, not even close to the truth:
Computers are no closer to _reproducing_ human intelligence or "intuition"
than Ford's Model T was to reproducing human legs.

Instead, apparently AlphaGo is a some high dimensional data fitting to yield a
_classification_ system, that is, given a board position in Go, AlphaGo
reports 0 for bad for the AlphaGo player and 1 good for the AlphaGo player.
So, they have a classifier.

But classifiers go way back, to Leo Breiman's random forests, classification
and regression trees (CART), logistic regression, a huge range of statistical
tests, e.g., in some cases of advanced radar target detection, and much more.
Many such solutions have been very valuable, and maybe AlphaGo or its
techniques will also be valuable, but none of this in any sense _reproduces
human intuition_ or really anything about humans -- not yet, not even close.

Net, while it's a free country and people can disagree, which is one role of
HN, IMHO, the quote I gave above from the OP is scientifically irresponsible.
It just isn't really true. At best it is just hype.

------
vph
Somehow, I feel that the big deal here is the cleverness to use words like
"neurons", "deep learning" and "human intuition" (the "intuition" here appears
no more than simply taking the route giving max probability of success).

~~~
zero_iq
I think you're greatly underestimating how difficult it is to come up with an
algorithm to play Go.

Even evaluating a single play state is a difficult problem, let alone looking
forwards to 'simply taking the route giving max probability of success'.
That's not simple at all.

Go is fiendishly difficult to evaluate. The traditional techniques that work
in chess don't work well in Go (if at all). All the pieces have the same
'value'. More or less pieces isn't necessarily better or worse, it depends on
the board layout. It's not at all easy to say who's winning or losing at most
points in the game, or if there even is a winner or a loser. There is no
mathematically-obvious way to compare two play states and say one is better
than the other. The branching factor is huge, making the usual forward-looking
techniques extraordinarily expensive. The number of possible legal boards
states is something like 90 orders of magnitude greater than the number of
atoms in the observable universe, making backward-looking techniques like
minimax impossible, even if you could easily assign an evaluation function to
each state.

~~~
gcp
_It 's not at all easy to say who's winning or losing at most points in the
game, or if there even is a winner or a loser._

You play out the game to the end.

 _making backward-looking techniques like minimax impossible, even if you
could easily assign an evaluation function to each state._

That's exactly how AlphaGo works though. Read this paper, it was the landmark
result that showed how to do this: [http://www.remi-
coulom.fr/CG2006/](http://www.remi-coulom.fr/CG2006/)

AlphaGo uses the same technique, just optimized a bit more.

~~~
zero_iq
_You play out the game to the end._

Nope. You could play out to one or several ends, but there is a huge number of
possible end games at any given point. Simply saying 'play out the game to the
end' is a gross simplification.

The average branching factor is something like 200, so just to fully evaluate
just 5 moves ahead would require traversing 320 billion game states. Go games
are 100-200 moves per game, so 5 moves ahead doesn't even get you close to the
end of a game. At the first move there are
208168199381979984699478633344862770286522453884530548425639456820927419612738015378525648451698519643907259916015628128546089888314427129715319317557736620397247064840935
legal board states you'd have traverse to reach all end games. Yes, that's an
exact figure.
([http://tromp.github.io/go/legal.html](http://tromp.github.io/go/legal.html))

How big is that figure? If you counted up the number of fundamental particles
in the entire universe, and for each fundamental particle there was another
complete universe of fundamental particles and you counted all of those too,
and then do all of that once for every person in the USA add up their answers,
you'd be at about 3% of the way to that value.

You simply cannot do a full evaluation to the end of the game: it's
computationally infeasible.

 _That 's exactly how AlphaGo works though_ (minimax)

No, it's not. Minimax requires working back from all end-game states, and
there are too many end-game states to be evaluated. I suggest you read the
paper yourself. It's about doing random partial evaluations and forward-
looking searches of the game space, and choosing clever subsets to evaluate,
precisely because you can't evaluate them all. The 'backing up' referred to in
the paper is backing evaluation values up the tree after you've looked ahead a
while, not starting at all possible end games and backing up from there, as in
classic minimax. Some of the basic concepts are similar to classic tic-tac-toe
or chess algorithms, but combining them in new ways.

 _AlphaGo uses the same technique, just optimized a bit more._

And a fighter jet uses the same technique as a paper aeroplane, only optimized
a bit more...

~~~
gcp
_You simply cannot do a full evaluation to the end of the game: it 's
computationally infeasible._

You can sample several randomly played out games. I think you knew this, and I
hope you understand that sampling works, but for some reason you went on a
completely irrelevant tangent.

 _Minimax requires working back from all end-game states_

Practical implementations that need to achieve good but not perfect play are
content to stop at a point before that.

Again, do you know this and are intentionally misunderstanding me and throwing
up irrelevant facts? I'm sure you did given that you even acknowledge the
above (no need for terminal states) in your previous post.

The difficulty of Go was that there was no way to combine tree search with the
uncertainty inherent in Monte Carlo samples with a low sample count. The paper
I mentioned fixed this, and hence was THE major breakthrough. The resulting
algorithm is still quite simple, yet plays arbitrarily strong, also in
practice, when given more computing resources.

It makes a minimax-like algorithm work for go because it estimates the
terminal results by Monte Carlo sampling the results of the games when played
out by a random (or in practise, not entirely random) player.

~~~
zero_iq
I think the problem is that what you meant is very different from what you
actually wrote.

 _Practical implementations that need to achieve good but not perfect play are
content to stop at a point before that._

Indeed, but that's not the basic minimax algorithm (which is what I was
discussing) and the branching factor of go makes classic minimax infeasible
even if you modify it to be forward-looking. Basic minimax is not good for Go.
At all. You have to combine it with other techniques for it to be even vaguely
effective. You know this, but didn't mention it. So how is anyone supposed to
infer that you know it?

I can see _now_ that you have a decent understanding of the actual solution,
but that's not at all what you wrote above.

For example, you said to evaluate a game board you need to 'play the game
through to the end' \-- which hides all that complexity of monte-carlo tree
evaluation, etc. Read what you wrote again from the point of view of someone
who isn't yourself.

Simply saying 'Play the game through to the end' by itself is neither correct
nor accurate. Because that implies evaluating the rest of the game to its
conclusion. i.e. every board state. You omitted any other detail or reference.
So I descibed how and why it's not feasible (which you now discount as going
off at a tangent). You gave no prior indication of understanding this.

Another example: in a previous comment I described how backward looking
techniques like the minimax algorithm are not feasible for go, and you said:

"That's exactly how AlphaGo works though".

But it isn't at all:

Classic minimax is backward-looking (starts at end states and works backwards
-- this is what I was talking about, and what I described); AlphaGo (and the
paper you linked to) is forward-looking (starts at current root and works
forwards).

Classic minimax is full-tree evaluation; AlphaGo is partially-evaluating.

Minimax is deterministic; AlphaGo is stochastic/random sampling.

Minimax uses a simple single-value single-direction evaluation function;
AlphaGo uses a complex bi-directional multi-value evaluation.

I could go on.

Read any treatise or any introduction to AI for Go and pretty much the first
chapter will be describing how classic minimax is not suitable without heavy
modification.

e.g.
[https://www.youtube.com/watch?v=5oXyibEgJr0](https://www.youtube.com/watch?v=5oXyibEgJr0)

AlphaGo uses some of the same _concepts_ , but applied very differently, as
you appear to already know. Yet after I mentioned minimax (which is well-known
as an infeasible algorithm for go), you responded:

"That's exactly how AlphaGo works though".

Which is not at all true.

And then you link to a paper that describes precisely how it's _not_ how it
works, and why minimax-related techniques alone don't work, in stark
contradiction to yourself.

I'm not intentionally misinterpreting what you wrote: I'm taking a reasonable
interpretation of what you _actually_ wrote, which appears to show a lack of
understanding on your part. I don't think there is a lack of understanding,
now, but it's hard to come to a different conclusion based on what you wrote
before.

------
llSourcell
If you guys like this, check out my '4 Reason why AlphaGo is a Huge Deal'
video
[https://www.youtube.com/watch?v=2QhVarCzscs](https://www.youtube.com/watch?v=2QhVarCzscs)

