
AlphaGo's next move - bjin
https://deepmind.com/blog/alphagos-next-move/
======
cocktailpeanuts
There was a time not too far back when people used to be considered a "genius"
for their ability to memorize things well.

Nowadays nobody thinks of them as geniuses.

Also, people used to be considered geniuses for knowing a lot of things.

Nowadays information is just a Google search away, so knowing a lot doesn't
really mean as much as it used to. What matters more nowadays is your ability
to learn synthesize the things you know to come up with creative solutions to
things.

Basically the "memory" part of human brains have become commoditized without
us even realizing.

It's still very early but I do think there have been some subtle but
significant step forward in the last couple of years. The most important
being: machines are capable of doing certain things better in ways humans
can't comprehend easily. I think this is a glimpse into the future where the
"creativity" aspect of our brains will become commoditized, also without us
realizing.

This doesn't mean machines will take over, just like machines didn't take over
the world because they have better memory. But I think this will result in
many humans taking advantage of this aspect to exert influence on rest of the
humanity.

~~~
andrepd
> Also, people used to be considered geniuses for knowing a lot of things.

This is still true, and in the eagerness to dismiss "memorization" as a thing
of the past you overlook the obvious. For example, anything you care to know
about, say, C++ programming or quantum field theory is available to you on the
internet. But does that mean you can write a C++ program as if you had already
learned it? What if you want to write a C++ program and you have to look up
everything? You will do a very poor job if at all, and you will take a lot of
time.

So yeah, until looking up stuff in the internet is as quick as effective as
looking stuff up in your brain (the quick may happen but the effective I don't
think so), then it still is a very worthy skill.

~~~
jayd16
But you've just proven cocktailpeanuts point that its about the ability to use
the knowledge, not simply recall it.

~~~
adharmad
The ability to use knowledge does not come out of thin air. It is unlikely to
find someone who knows how to program C++ extremely well, or solve problems in
Quantum Field theory, but does not remember most of the language constructs or
mathematical equations.

Continuous practice involves putting in the hours at practicing an art or a
science, which by itself builds muscle memory about the language
syntax/equations etc. It is unlikely that one can remember one but not the
other.

~~~
jacobolus
“The best geologist is he who has seen the most rocks.” – H. H. Read

~~~
YeGoblynQueenne
Well, in that case:

"All science is either physics or stamp collecting" -Ernest Rutherford.

------
binarymax
I remember vividly in 1997 when Deep Blue defeated Kasparov, and I was a
competitive chess player. The mystique of the game was immediately lost for
me, and I never found the passion for the game that I once had. My heart goes
out to the sea of Go players now searching for meaning in the game. At the
very least we can take this signal as a true indicator that our world is close
to being completely upheaved by intelligent machines, in all areas of
intellectual pursuit.

~~~
probably_wrong
There are several activities right now that people keep doing despite machines
being better suited: martial arts (and more in point, HEMA[1]), handicrafts,
several Olympic sports. Why would anyone attempt to run a marathon under two
hours, when any car can do it in 30 minutes?

I think the answer lies less in "I will be the absolute best", and more along
the lines of "I will do it better than anyone before me". And sometimes, even
"I will do my best" is an excellent reason for doing things.

I don't think Go players were in it due to a need for expertise that machines
could not fulfill until now. And if people nowadays keep practicing with
swords several centuries after the invention of firearms, Go players will do
just fine.

[1]
[https://en.wikipedia.org/wiki/Historical_European_martial_ar...](https://en.wikipedia.org/wiki/Historical_European_martial_arts)

~~~
matt4077
It does feel like there's a difference–maybe Go and Chess are just so much
closer to what defines humanity than any physical sports?

~~~
binarymax
Yes and physical prowess was never dominated by humans anyway. We are
surrounded by creatures which can fly and swim and sprint far better than any
human ever could. But our minds is what separates us from the beasts. We've
never been second place in the thought competition.

~~~
namarie
Aren't we actually better at endurance running than any other animal?

~~~
tambourine_man
Seems sled dogs beat us on this one:

[https://youtube.com/watch?v=HDG4GSypcIE](https://youtube.com/watch?v=HDG4GSypcIE)

~~~
maga
Yet again we have created something to outdo us.

~~~
hyperbovine
DeepMind could do it better.

------
nopinsight
One common comment from Go players at all levels up to 9-dan pros is that they
don't understand many of the moves. The same will happen as more and more
advanced AIs are used in the real world.

Yes, we do not completely understand the workings of current advanced neural
networks either but the effects are still contained as they are not general
enough to cause unintended impact outside their domains.

This could have started to change: a recent Google paper, AutoML, allows the
machines to design themselves to suit each task. [1] A future advance could
allow the machines to pick and learn to do new tasks that are helpful to
accomplish a given high level mission. Therefore, chances of unintended
consequences become much greater.

With human involvement only at the meta level, deep understanding of the
generated implementations becomes more challenging and, in highly complex
domains, perhaps impossible.

The major issue is, without a moral core that closely aligns with humanity's
evolved morality, there _will_ be moves that advanced AIs come up with that we
deem abhorrent, and sometimes unforeseeable, yet they perform them innocently
and we only find out the consequences once it is too late.

[1] [https://research.googleblog.com/2017/05/using-machine-
learni...](https://research.googleblog.com/2017/05/using-machine-learning-to-
explore.html)

~~~
skybrian
They don't fully understand the moves but on the other hand, the live
commentary on the games suggests it's not completely mysterious. Good moves
still tend to look good to them, in retrospect at least.

The games are apparently very interesting to study.

~~~
AndyNemmity
Against a human, the games look fairly straight forward.

From what they've released against itself, the games look like a different
game. Especially at times, Game 2 in the current crop for example.

------
ipsum2
Congratulations to Deepmind and Google for this tremendous achievement.

However, it is disappointing that the code and model will not be released
publicly after Alphago finishes competitive play. It's one thing to say that
an apple, once dropped, will fall to the ground, but another to describe its
motion as 1/2at^2 + vt.

~~~
tux3
They did announce that they would release a teaching tool which will show
AlphaGo's analysis of Go positions, as well as the paper explaining how to
build your own.

Not only do you have the principle and the formula behind it, but also a
little physics simulator tool! At this point, it is hard to complain.

~~~
gwern
> At this point, it is hard to complain.

Actually, it's very easy to complain. If they released the model, people could
generate arbitrarily many self-play games instead of depending on DM to
release 50, could create arbitrarily many tools using the model instead of
depending on DM to create and maintain a single tool, and could verify the
results of training a clone based on even sketchy descriptions of the methods
instead of depending on DM releasing a detailed enough whitepaper and then
guessing at whether a reimplementation is competitive or not. DM is only being
'generous' if you ignore how releasing the model is easier for them and
superior for us in every way.

~~~
espadrine
> _people could generate arbitrarily many self-play games_

I have doubts. Their TPU design may be a large factor into making matches at
this level within the time limits. And at this point, some implementation
details might hook into Google-specific libraries that require the ability to
spawn processes in thousands of servers, which past blog posts[0] have hinted
at.

[0]: [https://deepmind.com/blog/decoupled-neural-networks-using-
sy...](https://deepmind.com/blog/decoupled-neural-networks-using-synthetic-
gradients/)

~~~
gwern
There might be some hard to release infrastructure code for the MCTS part,
certainly, but the model on its own should be a standard TF CNN model and
highly competitive (and people can write their own MCTS wrapper, it's not that
complex an algorithm). Nothing in the AG paper or statements since has hinted
at using anything as exotic as synthetic gradients* and there is no reason to
use synthetic gradients in AG. (In RL applications the NNs are generally small
because there's so little supervision from the rewards so a large NN would
overfit grossly; a NN so large as to require synthetic gradients to be split
across GPUs would be simply catastrophicly bad. Plus, the input of a 19x19
board, a few planes of metadata, and other details encapsulating the state is
small compared to many applications like image labeling, further reducing the
benefits of size. Silver has said AG is now 40 layers but that's not much
compared to the 1000-layer Resnet monsters and even those 40 layers are
probably going to be thin layers, since it's the depth which provides more
serial computation equivalence, not width, making for a model with relatively
few parameters overall.)

* I find synthetic gradients super cool and I've been reading DM papers closely for hints of its use anywhere and have been disappointed how the idea doesn't appear to be going anywhere. The only followup so far has been [https://arxiv.org/abs/1703.00522](https://arxiv.org/abs/1703.00522) which is more of a dissection and further explanation of the original paper than an extension or application.

------
mark_l_watson
The 10 AlphaGo vs. AlphaGo games are a nice gift!

I have always liked playing through great games, both Chess (using the book
The Golden Dozen) and Go (modern games and the ancient Shogun Castle games).

I have some history with computer Go. In the late 1970s I wrote a Go playing
program in UCSD Pascal that I sold for the Apple II, and also for a lot more
money I sold the source code to a few people who wanted to experiment with it.
DeepMind's AlphaGo is a great intellectual and technological triumph and I
agree that it is an example of future AIs teaching us and working with us.

A little off topic, but Peter Norvig gave a nice talk a few weeks ago at the
NYC Lisp Users Group where he talked about the future of collaboration with
AIs and also that the ability to work effectively with AIs, adding human
insights, will be an important future job skill.

~~~
oska
Do you know if there's video or audio of Peter's talk online? I just did a bit
of a search but couldn't find it.

~~~
mark_l_watson
[https://vimeo.com/215418110](https://vimeo.com/215418110)

Enjoy!

------
candiodari
It's scary how coverage of this match in China, which is extensive, actually
manages to censor the association between Deepmind and Google.

~~~
oska
The microphones used in the post match discussion featured the name “Google”
quite prominently. I imagine that was a negotiated detail.

~~~
candiodari
There's different post match discussions.

------
temp_12345
So, a question bordering on the philosophical :

What can be done to prepare for the end of human supremacy, and quite likely
human civilization? For instance, as a software developer it feels almost
pointless to continue improving at my craft if AI systems will surpass me
within 2-4 years (even if the pessimists are right about it taking 5-10 years,
that's still an awfully small timeframe).

Likewise, it feels a little pointless to work on any endeavor - technical or
otherwise - including but not limited to AI research itself. From a purely
practical standpoint just getting up to speed on AI research will take a solid
5+ years, and from a moral vantage point I'm not sure that's even a defensible
career given the obvious and hugely negative implications that field will have
for human civilization.

Even in artistic endeavors, humans will soon be second fiddle to our own
creations - so it's not like there's any "point" to starting down that path
either.

Is it time to just engage in a hedonistic, nihilistic, fest of gluttony and
"fun" while that's still possible? Honestly, news like this just makes me
consider ending it all : it feels like none of us will have much of a future
before long.

~~~
ac29
Is there really anyone credibly suggesting software developers will be
surpassed by AI within a few years? Writing arbitrary software seems
dramatically more complex than what "AI" like systems are capable of today.

Even 10 years seems impossibly soon.

~~~
resf
People said the same thing about Go. That it was far too difficult and that a
pro-level AI was 10+ years away.

What happened was new mathematical tools and new hardware were developed, and
suddenly it was all too possible.

It's clear that with our _current_ tools, general AI is out of reach, and new
tools must first be developed. But because nobody has any idea what those new
tools _are_ , it could happen overnight or over 100 years.

~~~
lambdadmitry
It didn't come from _nowhere_ , the current streak of ML achievements rides on
the back of deep learning, which is an elaborate pattern matching at its core.
What makes Go "harder" than chess is that it's difficult to estimate how good
or bad a particular position is, so we employed a "magical box" of deep
learning and learned to estimate how good a particular move is. That's pretty
good, but let's not forget

\- it took a ton of very hard work

\- it's not transferrable to other domains per se ("elaborate pattern
matching" can be, but it's not even an AI)

\- this has nothing to do with qualia, consciousness or the theory of mind.

Programming is not about elaborate search or pattern matching at the end. It's
about formalizing a domain, stripping it from some subset of real-world
complexity, and inventing a solution to a problem in that domain. A rift
between beating someone in Go and deducing a fact that doubles wouldn't do
well in financial calculations is _immense_.

This sort of over-extrapolation of current trends is surprisingly prevalent in
the tech crowd to be honest. It's like folks in mid-20th century who saw both
airline and car industry exploding and made a "logical" guess about flying
cars being the obvious next step. Guess what, physics doesn't work that way
and flying cars are a dumb idea. The current AI craze seems very, very similar
to me.

------
theptip
> We plan to publish one final academic paper later this year that will detail
> the extensive set of improvements we made to the algorithms’ efficiency and
> potential to be generalised across a broader set of problems.

I'm fascinated to see what the next step for this AI is. Anyone care to
speculate what a system like this could most readily be applied to?

------
placebo
I find it interesting that AlphaGo improves its play by playing against
itself. I wonder what the limits of this are.

~~~
paulsutter
In RL you have two modes, "explore" and "exploit". In explore mode it doesn't
always select the best known move, instead it selects a promising move for
which it has less experience. This is how the surprising new strategies are
discovered, in self play there's no shame in losing.

------
yters
The problem is an AI that is good at Go is not at all transferable to any
other game. However, a human prodigy can apply their genius to many domains.

~~~
reckoner2
> However, a human prodigy can apply their genius to many domains

I'm not sure how true this is. It's pretty rare to find someone who is a
genius in more than one domain. Einstein was famously offered the presidency
of Israel. Sure, he could probably do well in other sciences, but he was smart
enough to know he could not apply his genius to unrelated domains.

~~~
yters
Of course there is domain knowledge that needs to be learned, along with non
intelligence related characteristics, such as personality. But general
intelligence appears to be widely applicable.

------
Entangled
I'd like to see public competitions between two AI giants like Google and IBM.
Now that would be an interesting ongoing race for AI superiority.

------
Houshalter
My browser just shows me a blank page.

------
fiatjaf
If your AI is so great you should do some deep learning thing to explain the
AlphaGo moves.

------
wiz21c
The gift Google makes to the community (some games AlphaGo,played ) is
nothing. The super tricky thing with neural networks is that you can't reverse
engineer them. Once the information is coded into the parameters, you can't
base anything useful on them. So it's a super good intellectual property
protection... Therefore one more nail in the coffin of knowledge sharing as we
know it...

~~~
yeukhon
Most importanly, it is almost inevitable they must run on a fam of computers
basically mean it becomes a service. Can we ever create a robot who can self-
learn but with the super brain power locally without having to call a service
for an answer?

~~~
wiz21c
This raises a question for me : is there currently some public infrastructure
that can rival with AlphaGo ?

~~~
yeukhon
Public means API? AlphaGo is very specialized in solving Go game. Google,
Amazon and IBM have services for various services like image recongition and
speech recongition. Startups like Clarifi also exists in that space.

The closest to a generalized AI service would probably be Watson from IBM (but
I don't have experiment with it sadly so I am not sure about the usage
experience).

~~~
nl
> The closest to a generalized AI service would probably be Watson from IBM

Ha ha ha!

~~~
yeukhon
That's not very nice.

~~~
nl
Sorry, I honestly thought you were joking.

'Watson' as it is sold today is a mishmash of random, separate services. You
can get the same from Google, MS or multiple other places.

'Watson' the jeopardy winning thing was an ensemble of search and rule-based
NLP techniques.

Neither are much like general AI.

------
codecamper
"We have always believed in the potential for AI to help society discover new
knowledge and benefit from it" Get real. You do this for your own intellectual
gain. Google does it for financial gain.

Meanwhile, Antarctica may crumble. How about putting effort into solving THAT
problem, with all your technology & knowhow Google?

~~~
sowbug
[https://www.logicallyfallacious.com/tools/lp/Bo/LogicalFalla...](https://www.logicallyfallacious.com/tools/lp/Bo/LogicalFallacies/155/Relative-
Privation)

