
What we learned in Seoul with AlphaGo - Eldorado
https://googleblog.blogspot.com/2016/03/what-we-learned-in-seoul-with-alphago.html
======
nzonbi
I think that the event will help to boost investors confidence, and hence
investment in AI research. Which is great, because it means accelerated pace
of innovation. It is a huge win for the AI field. AlphaGo showed that a
special arrangement of neural networks with additional algorithms can deliver
spectacular results. They can now try more sophisticated arrangements of
neural networks, to achieve more ambitious results. I hope that we will now
start to see more creative attempts at AI. The time to finally reach to
general AI, will depend on the magnitude of the research effort that is put
on. This kind of highly publicized events are very positive.

AlphaGo was a combination of neural networks with a tree search algorithm. I
think that very interesting things could be achieved, combining neural
networks with basic knowledge representation systems -symbolic AI-. These
techniques are highly discredited for under-delivering in the past. But I
think that it is a good moment to revisit some of these past techniques, and
combine them with the more recent techniques of neural networks. It could be
an interesting base to attach neural networks. And then submerge the AI on a
basic simulated world. Something like minecraft, perhaps, as Microsoft
announced that is going to do. I think that giving the neural networks some
basic structure to depend on, can help to achieve results more easily.

In any case, I hope that high profile AI events like AlphaGo victory continue
happening, to help to further increase AI research.

~~~
Ericson2314
As a programming language specialist, I am for all things symbolic too.

[http://leanprover.github.io/presentations/20150717_CICM/#/se...](http://leanprover.github.io/presentations/20150717_CICM/#/sec-53)
The last slide hints at proof assistent + ML. I wish I knew more of what they
are up to!

~~~
amelius
I'm not sure if I'd like to see AI in a compiler backend; development might
become unpredictable (in terms of program efficiency).

~~~
Ericson2314
My guess is the machine learning is some sort of tactic. So it communicates
with Lean to generate some code (program/proof), but ultimately Lean itself
decides whether the code is correct---this is no "polluted backend".

------
olau
It's awesome that they won.

I recently read the paper, and there are a couple of things you need to keep
in mind to understand the scope and how general the result is.

They were using a big cluster to do a brute-force tree search (not brute-force
as in exhaustive, but still brute-force as in let's throw lots of hardware at
this). According to the paper, this tree search was important in improving the
play.

Basically they were using a combination of approaches, like the winners of the
Netflix competition a couple of years ago, where each approach in its own was
pretty good, but not on the level of Sedol.

The other thing is that this was bootstrapped using a gigantic database of
human plays. It's not clear to me that they could have ever achieved what they
did without this. Once they trained the neural networks up to the level of an
expert player, they could make it play against itself and learn some extra
things. But the question is how far this takes you? How much can an AI or a
human learn by only playing with itself?

Clearly, it's not yet god-like, since Sedol managed to beat it by a move it
wasn't really considering. It's not clear to me how you would improve what
they have now, without adding yet another approach, like the Netflix
competition where the mixed models got better by the sheer number of them.

~~~
visarga
> It's not clear to me how you would improve what they have now

They could improve the policy network which was based off 100,000 amateur
level games. Now they could use AlphaGo self play games which are at the level
of 9p as a training set.

Another thing they could do is let it run more self play games in order to
improve the value net even more.

------
silverlight
Congratulations to everyone involved in this tremendous achievement, both the
engineers who created AlphaGo and the Go community for being such gracious
hosts.

------
andrepd
"while the match has been widely billed as "man vs. machine," AlphaGo is
really a human achievement."

I think this is very true, and we often forget this in our rush to praise "the
machines". We built them! They aren't a being, they are tools _we_ built.

------
cryptoz
> We founded DeepMind in 2010 to create general-purpose artificial
> intelligence (AI) that can learn on its own—and, eventually, be used as a
> tool to help society solve some of its biggest and most pressing problems,
> from climate change to disease diagnosis.

This is really interesting to hear and absolutely fantastic news. I've often
wondered if we might be able to find better (cheaper, faster, more effective)
solutions to global problems like climate change by using AI. It's a thrill to
hear that's actually in Google's plans.

~~~
Obi_Juan_Kenobi
They announced some healthcare products last month, though they won't use AI
at first. It's safe to say that's part of their plans, however.

[https://deepmind.com/health](https://deepmind.com/health)

------
Madmallard
While this is an interesting breakthrough for AI, let's remember that Go is
only limitless in the exhaustive possibilities for every response move in
sequence for the game. So many moves in response to a move are just not good
and invalid you can do extensive pruning of search trees rather easily. That's
why monte carlo algorithms work so well as it is. They don't beat the best of
the best but I mean they still beat like literally 90% of Go Players.

But the game's rules are simple, the total storage of state needed for a game
is rather small, so in terms of utilizing training data it's quite easy. You
can fit an entire Go game in like 2 kilobytes of memory. There's only two
types of stones, and they have no meaning other than being different than each
other.

We would need more breakthroughs for real-time games. Compared to Go, they
have easily gigabytes of potentially meaningful data PER game. Not only is
storing millions and millions of games becoming an issue, processing them is
becoming an issue as well. And in many of those games when strategy changes
depending on which character or map you're on, it quickly balloons out in a
manner that is just not reasonable.

Maybe the next step will be specialized AIs that tackle small more easily
calculable components of real-time games and then combining those together to
make something decent.

~~~
abecedarius
The same company, DeepMind, already published about reinforcement learners
reaching superhuman performance on many realtime games -- old Atari ones.

I know there's a lot of hype and ignorant confident opinions being expressed,
but this sort of response seems really strange to me: in the last decade we
went from nobody having a clue how we might automate Go in this lifetime, to
2014 when an article on prospects for Go by a premier researcher on game AI
([https://news.ycombinator.com/item?id=11290112](https://news.ycombinator.com/item?id=11290112))
did not even mention neural nets, to professional play last fall, to 5 months
later crushing a top player and making high-level innovations, in a game said
to be among the most deep and beautiful ever invented... and the most salient
points are about how trivial it all is?

~~~
Madmallard
The same company, DeepMind, already published about reinforcement learners
reaching superhuman performance on many realtime games -- old Atari ones

This is not remotely similar to league of legends or starcraft.

~~~
stone-monkey
I would think LoL and Starcraft would be easier in terms of difficulty than
something like Go, because they both have the fast twitch aspect of play. An
automated player would have the advantage of perfect situational awareness of
the mini map and consistency in character placement and so forth. This is a
big advantsge in games where a single misclick or a missed visual can cost a
match.

~~~
Madmallard
Strategy is what matters here. That's what I was getting at with the last part
of my first message. We can make bots that do certain aspects of the game
flawlessly and we will need to find a way to integrate them.

------
johansch
I know.. don't be negative.. however, that image along with its caption
"Pedestrians checking in on the AlphaGo vs. Lee Sedol Go match on the streets
of Seoul (March 13)" was too funny.

(It shows pedestrians ignoring the giant screen that is showing the game.)

~~~
mchahn
> It shows pedestrians ignoring the giant screen

I don't see that. How do you know where their eyes are directed when they are
facing away from you?

~~~
johansch
Because most people tend to shift their heads to match their gaze when they
watch something for more than a fraction of a second.

~~~
mchahn
This argument is getting silly, but the closest two could definitely have
their heads tilted.

~~~
chrischen
I think part of his point is that _only_ two people were looking.

------
mannykannot
The comment about move 37 in game 2 - unlikely to have been made by a human -
makes me wonder if AlphaGo considers the chances of an effective human
response, either as a learned or innate (programmed) behavior. Unless most of
its training has been against humans, I would guess it could not have learned
to do that (and I don't even know if it would be a useful metric in Go.)

~~~
AaronFriel
This isn't something AlphaGo was trained to do - that is, whether or not a
board state it has seen was a human or a non-human move was not part of its
training model.

To take this to its fullest conclusion, imagine that AlphaGo was not only
trained on that information, that is, and that in addition the hundreds of
thousands of board states it has seen, it also had layers which encoded
analysis of the players before, during, and after the game (their tweets,
their weibo messages, and so on). This is the difference between what AlphaGo
does and what a human player can do, and what a true strong AI with unlimited
compute power could do. AlphaGo can't play as _efficiently_ as possible, it
doesn't have that information.

A hypothetically omniscient being, or the Hand of God, can play meta-go. It
can play in a way that cause the human player to be less likely to play a
winning game. As Eliezer Yudkowsky put it:

    
    
        With regards to tonight's match of Deepmind vs. Sedol, an example of an
        outcome that would indicate strong general AI progress would be if a
        sweating, nervous Sedol resigns on his first move, or if a bizarre-seeming
        pattern of Go stones causes Sedol to have a seizure.
    

The Hand of God could play the game in such a way that, on games 1-3, it ended
the game in a board state that issued a threat to Lee Sedol's family.

Fortunately it seems we're still a ways off from playing Go against the Hand
of God.

~~~
yongjik
> ... or if a bizarre-seeming pattern of Go stones causes Sedol to have a
> seizure.

I didn't think I could have a lower opinion of Yudkowsky, but apparently I was
wrong. What is this, a Gibson fan fiction?

~~~
AaronFriel
I don't think his comment was to be taken entirely seriously, but why do you
have such a low opinion of him?

~~~
marvin
I really don't get the hate against Yudkowsky that I've seen on HN recently.
Disagreement is one thing, but it almost seems as if a lot of people here find
his ideas and writings insulting.

I don't get it; Yudkowsky is a philosopher and technologist with very
interesting thoughts about an important subject that is not studied enough. It
would be nice if people could substantiate their criticism rather than resort
to name-calling. Everything I've read about Yudkowsky and MIRI seems dead-on:
Further research around the issue of long-term AI safety sooner rather than
later. I really don't get why this is problematic; I think it's a very good
thing that someone smart spends their effort on this.

The comment in question was obviously meant tongue-in-cheek to illustrate a
point about potential more efficient, hypothetical ways of winning a game.

------
ikeboy
>Based on our data, AlphaGo’s bold move 37 in Game 2 had a 1 in 10,000 chance
of being played by a human.

This is meaningless without context. What algorithm, has it been tested for
calibration, etc. I honestly don't know what I'm supposed to take away from
this number.

~~~
Houshalter
AlphaGo was initially trained to predict the moves that humans would make,
using a deep neural network, and given tons of game records. Then it was
trained again with reinforcement learning by playing against itself by taking
moves based on that probability, and increasing the probability of moves that
lead to wins.

So when they say it predicted 1 in 10,000 chance, it means it thinks it's
really unlikely a human would play that move. Just playing random moves means
each move has only 1 361 chance at worst, so that move must _strongly_ violate
normal human play patterns.

~~~
ikeboy
It's also odd that both numbers are 10,000. Maybe their probability function
bottoms out there?

~~~
sanxiyn
In the Nature paper, move prediction was trained on 160,000 games. So naively,
I expect probability can't go below 1 in 160,000.

~~~
ikeboy
But those have many more moves.

------
mark_l_watson
Sad to see the tournament end! Fantastic entertainment. I have started playing
Go again, but I don't think that I will start up again doing Go programming.

------
Yoda1337
Title is misleading. They didn't learn much in Seoul.. they just went there
and crushed the best Go player and demolished Go being the poster kid for why
AI can't win all games. I honestly have a very hard time thinking that people
believed computers couldn't beat humans at Go. That's what a computer AI excel
at.. give it a board position and ask it the next best move. Finite input,
finite output, almost infinite crunching power.

~~~
dmoy
Because it's not infinite crunching power, and the search space is basically
infinite.

------
colllectorof
This reads like a mix of a disingenuous PR statement and a typical agenda-
selling article from some mainstream news org.

Let's look at just one sentence:

 _> We've also had the chance to see something that's never happened before:
DeepMind's AlphaGo took on and defeated legendary Go player, Lee Sedol (9-dan
professional with 18 world titles), marking a major milestone for artificial
intelligence._

Hype-inducers: "never happened before" "legendary" "marking a major milestone"

Also, AlphaGo didn't "take on" anyone. It plays whatever games are fed into
it. It might seem insignificant, but such small details is exactly how most of
marketing works. That is how we get "ultimate" luxury cars, "curious" banks,
and insurance providers that are "always there for you".

 _> And because the machine learning methods we’ve used in AlphaGo are general
purpose, we hope to apply some of these techniques to other challenges in the
future._

AlphaGo's design has a lot of stuff highly specific to Go.

~~~
jblow
This is legendary. Most people (including me) would have thought this would
not be possible for decades.

9-dan is the highest rank in Go. It is not possible to play against anyone
higher.

So I am not sure why you think it isn't a big deal.

~~~
colllectorof
Because it's a _fucking board game_. There are tons and tons of more practical
and impressive computer science achievements that get absolutely no news
coverage.

~~~
chillacy
> more practical and impressive computer science achievements

If we came out with a polynomial time solution for the graph coloring problem,
you could use that same logic to say "it's just labeling colors, what's the
big deal?"

~~~
colllectorof
Graph coloring problem is generic and is _guaranteed_ to be useful in a myriad
obvious applications. Plus, "polynomial time" is a specific criteria, whereas
"somewhat better than a human champion" is an arbitrary milestone.

Besides, people who make advances in abstract problems like that do not
release PR statements and get 1/1000th of the hype and coverage AlphaGo
received recently.

~~~
abecedarius
NxN Go is PSPACE-complete.[1] That's an even bigger deal than NP-complete.

[1] According to some comment in /r/MachineLearning the other day. I haven't
looked into it myself.

~~~
jeeyoungk
That is with some bounds applied to it - if we allow unbounded games (ko,
superko, etc) then it is EXPTIME-complete.

~~~
jxy
Yes. Source here:
[http://senseis.xmp.net/?ComplexityOfGo](http://senseis.xmp.net/?ComplexityOfGo)

