
DeepMind StarCraft II Demonstration [video] - Permit
https://www.twitch.tv/StarCraft
======
darkmighty
I was left unconvinced (reproducing my comment from reddit):

AlphaStar had 0 cost to sense the entire map simultaneously, and when they
introduced the attention/context switching cost (camera control) in the
showmatch, it lost.

Also very notable for me in the showmatch was it seemed completely blind and
exploitable to some strategies. When AlphaGo played top players, it made a few
mistakes, but there wasn't anything obvious that it just couldn't see. Here it
just couldn't think of making phoenixes vs. the warp prism harass which shows
strategically it isn't near human level yet. It could clearly be exploited by
back and forth harassment too (probably has to do with the limited memory
those networks have).

Finally, DeepMind were just emphasizing average APM, when it clearly reached
totally superhuman levels at times -- even a top professional can't execute
900+ flawless apm in a battle that we saw.

David Silver was clearly expecting the showmatch to be totally one sided
(hence his speech that 'this is another historic victory for AI), but I were
left with the opposite impression: that strategically top humans are still
ahead in this game. This is not the end of the game for humans yet !

~~~
roenxi
The original AlphaGo showing (vs Fan Hui) had obvious problems. They weren't
catastrophic, it was clearly playing at a professional level stronger than 1d,
but it also wasn't obviously superhuman. Lee Sedol could have expected to win
his matches, and not been arrogant about it.

The gap between that AlphaGo and the AlphaGo that beat Lee Sedol was probably
something like 10 years of human improvement compressed into a couple of
months. 500 Elo is huge.

Even if they still have further to go, comparing this to AlphaGo's progress is
reasonable. I'm going to live dangerously and make an assumption - you
probably understand Starcraft better than Go, so can see the flaws more
easily. Any mistake in Go comes back to something the AI just didn't see.

~~~
darkmighty
Yea stepping back a little I agree it's a fundamental achievement. It's all
about expectations -- I were expecting a Lee Sedol kind of event, and it was
more of a Fan Hui (or a bit further) kind of demonstration. A milestone for
sure considering how difficult the problem is with various previously
unconquered subchallenges (uncertainty, partial observability, massive control
space), different but in some ways more difficult than OpenAI's Dota 2
matches.

> I'm going to live dangerously and make an assumption - you probably
> understand Starcraft better than Go, so can see the flaws more easily

Correct :) Although I'm not great at either.

This comes closer to opening doors to real uncharted territory in robotics and
general agency; it trained in crazy low wall-clock time. But now we're this
far it makes me question even more the cost, flexibility, adaptability, of the
AI less forgivingly. How much did it actually cost to train? Would it be
economical to train one of those for every single task? (e.g. in a factory or
warehouse setting), Would it be vulnerable to those exploitable strategic
weakness in a real world setting?, etc.

It's definitely not over yet as far as SC2 goes though :)

------
kmnc
Very impressive... but it seems like the AI relies entirely on abusing blink
stalkers which with perfect micro is basically impossible to counter. It is no
surprise it can crush pros when it has perfect timing and zero mistakes in
using these units.

I think the coolest thing is how the play of AI mirrors a similar style to how
pros have developed (Micro harrasing early, early expansions, a very good
understanding of when to attack/retreat). It looks just like your average
pro..until you see its god tier micro.

Well, after seeing Mana crush it in the live game it seems the AI had zero
clue as to what to do... it seems like it calculated it couldn't win a fight
so let Mana destroy its entire base. So, just like with the Dota AI we see
pros can exploit and win easily once they play around the micro advantages.

~~~
Dylan16807
It had a variety of strategies, not always making a lot of stalkers. And
humans can do some _really_ impressive blink micro too up to a certain number
of units.

So that's one aspect of it but isn't the main strength.

~~~
orbifold
Those were not one and the same agent. If you look at the figure they released
on their website the given agent would probably always have gone for a lot of
blink stalkers.

~~~
Dylan16807
We're talking about the whole package of the AI, what the entire system is
capable of, so that technical detail doesn't affect kmnc's point or mine.

The biggest effect is that it's harder to change strategy midgame, which is
not all that critical.

~~~
orbifold
It makes the trained agent much less like a human, individual agents are not
creative, they‘ve just learned to execute a certain build order. It might turn
out that this is enough coupled with flawless execution and micro.

~~~
Dylan16807
"A human agent isn't creative, it just has a set of initial build orders it
developed and picks one before the game starts, and after that it adapts to
the situation weighted by the units it prefers."

Also the execution of the build order is nowhere near flawless.

------
larkeith
I only had a chance to see the live game, but very impressive stuff,
especially in the early game! However, it does seem to have the same issue as
all other AIs in that it is inflexible and fails to adapt to unusual
situations - As the commentators pointed out, it kept building oracles when
being constantly harassed by an immortal drop, whereas any amateur player
would be able to react more effectively by building a single phoenix.

It also fails to recognize patterns - MaNa was able to find and repeatedly
abuse a border between two strategies: whenever he was not actively dropping,
AlphaStar would attempt to move out and push, then immediately retreat to
defend when being dropped, whereas a human would recognize the reoccurring
theme and either keep pushing or stay in their base.

While not groundbreaking, still exciting to see an AI that can hold its ground
against pro players - it certainly demonstrates the potential of machine
learning for constrained problem spaces.

------
Permit
Here is another link (YouTube) that allows you to go back in time if you've
missed anything:
[https://www.youtube.com/watch?v=cUTMhmVh1qs](https://www.youtube.com/watch?v=cUTMhmVh1qs)

------
asar
The live exhibition match definitely made Alphastar look like a machine making
the decisions, not a super smart being. The micro was obviously impressive but
that should also be the easiest part to master. I share the sentiment that
DeepMind is hosting big events to paint a very one-sided picture of man vs
machine, so this last win of mana feels oddly satisfying.

~~~
dllu
It is a little disappointing that Mana's win was achieved in part by simply
exploiting Alphastar's poor response to the immortal drops by doing it over
and over again. In contrast, Lee Sedol's win versus AlphaGo involved profound
strategy and a particularly inspired "divine move" that humans get to brag
about.

~~~
Recursing
Lee Sedol "divine move" actually doesn't work, even human top players would
have punished it.

It actually was just a really weird mistake in AlphaGo

~~~
dllu
As they say... there are no good moves in Go/chess/zero-sum games etc. There
are only bad moves.

That doesn't prevent professional players such as Gu Li 9p from describing it
as a divine move though. And it does feel good for Team Humanity.

------
Veelox
Wow, Alphastar was able to 5-0 vs TLO (when he was off racing) and 5-0 vs Mana
in an impressive manner. I am quite impressed that DeepMind has been able to
take down Go and now Starcraft. From here on out, if DeepMind makes an
announcement about a demonstration of X, I expect them to beat the top humans
at X.

Edit: In the live game Mana is able to beat Alphastar. Alphastar was trained
with a more limited camera than the previous games. Mana was able to harass
Alphastar with a warp prism then push in and win.

~~~
iamjaredwalters
TLO and Mana are good players but Im not sure they fulfill your criteria of
top humans at X

~~~
Veelox
According to a site [1] Mana is 19th. While that is probably overly high, I
would say he is easily in the top 200 Starcraft 2 players in the world. I
would also say that to get from beating a top 200 human to beating all humans
is much smaller than from scratch to beating a top 200 human.

[1]
[https://www.gosugamers.net/starcraft2/rankings](https://www.gosugamers.net/starcraft2/rankings)

~~~
cjbprime
Aligulac's had Mana around 50th in the world consistently for a recent 12
month period, so I think that's a more reasonable claim, and I'm happy saying
that the 50th best player is a "top player in the world".

~~~
kzrdude
Is Mana the best Protoss player in Europe?

~~~
cjbprime
Second-best, probably. Best Protoss players, approximate from older Aligulac
data:

1\. Classic (Korea)

2\. Zest (Korea)

3\. herO (Korea)

4\. Stats (Korea)

5\. Neeb (USA)

6\. Dear (Korea)

7\. Trap (Korea)

8\. ShoWTimE (Europe)

9\. MaNa (Europe)

------
jpdus
I am somewhat disappointed that they stream only hand-picked replays. For a
big, announced presentation, they should be at a stage to stream live games
(like OpenAI did with Dota).

However, apart from that great work by Deepmind and I am excited to follow the
progress in this area.

~~~
kamkha
They're streaming a single live game right now! They explained that the
research build of the game being used doesn't have an "observer" mode so it's
not quite suitable for streaming live games — and, in fact, the games that
they were showing were played over a month ago. (The current live game is
being shown from the human player's point of view, and as such is a bit…
dizzying.)

------
maerF0x0
Some of the commentary shows that the UI is a barrier . The fact we can
understand the strategies, but cannot physically make it happen shows that at
least some of the advantage is just the precision of the inputs.

Could be a new way to play SC2 like games where we can better communicate our
intentions to the game. For example, make a type of move action where the
stalkers automatically retreat and stop to fire, instead of having to do move,
attack, move, attack series of inputs.

~~~
degenerate
This thought process (of the UI limiting the player) is a valid point when we
are talking about Human vs. AI strength and weaknesses in RTS games. However,
when pitting one human player against another, the UI limitation actually
_adds_ strategic depth to the game. Unlike pure strategy games like chess, in
Starcraft when you have to move, attack, move, attack, it puts a physical
burden on the player. This physical burden to work around the limited UI
allows one player to outperform the other on both a strategic and physical
playing field, in real time. If both players were freed of their physical
shackles, it would become a much more boring display of pure strategy, which
eventually would be "solved" and the game would cease to be fun.

If you look at the difference between Starcraft 1 and Starcraft 2, the two
most significant UI changes are the ability to select more than 12 units at a
time, and the ability to "multi-cast" spells in sequence when more than 1
spellcaster is selected. These two small UI improvements greatly reduce the
skill gap between an average player and a "pro" player in Starcraft 2, to such
a degree that many Starcraft 2 pro players went back to Starcraft 1, where
their fast reflexes give them a much greater advantage against other players
due to the limited UI. So an improvement to the UI would help human players
beat AI, but it takes a strategic element out of the game when considering
human vs. human.

~~~
maerF0x0
I agree there are pros and cons to the physical mechanic. For example
misclicks become a part of the game, whereas the RNG would be fair to each
player about errors in preprogrammed walk/attack patterns.

------
plaidfuji
This is mad impressive, for sure. With most AI problems, I can at least
comprehend the approach, and the necessary combination of models. Not so much
here.

My first question is what is the input to the AI? Is it the raw pixel array of
the display? Or does it get API-level readouts of what’s happening? Because
implementing the CV just to segment the display output in real time is crazy
enough. I would assume the latter.

I think this basically proves that any problem that can be exhaustively
simulated is solvable now. This may mark a tipping point, as every problem for
which simulations exist (essentially infinite labels) is solved - then the
balance will tip back toward making faster and more accurate sims (think
multi-scale first principles physics stuff).

~~~
triangleman
Blizzard released a client with an output accessible to machines, but still
preserving fog of war. See here:

[https://deepmind.com/blog/deepmind-and-blizzard-open-
starcra...](https://deepmind.com/blog/deepmind-and-blizzard-open-starcraft-ii-
ai-research-environment/)

Here's what I want to know, were these agents developed from scratch a la
AlphaZero in chess, or did they have to create a number of abstractions in
order to get the AI to start learning the game? In the initial demonstration
they could hardly get the AI to mine minerals or do anything. How did they
make the jump to actually good play?

~~~
johnmoberg
They mentioned that they initially used imitation learning on human replays.

------
1258989fdg
The AI makes some _awful_ decisions, such as building five observers. This
calls into question its "understanding" of the game. It looks like a lot of
its ability comes from micro, which it's unsurprising a computer can do better
than a human.

This is impressive, but with a lot of caveats. DeepMind's work on chess and go
was impressive with no caveats whatsoever.

~~~
devilmoon
I do believe the live match was with a newer version of the agent that hadn't
been tested against human yet

------
devilmoon
I'm not proficient in Starcraft, but if they have to make Pros play against 5
different agents for a Bo5 is it because the same agent would merely repeat
the same game overall and the human player would be able to see through its
strategy?

AFAIK at least for Go/Chess DeepMind wasn't handpicking agents to send against
human opponents, but it was simply a trained agent who would try its own
strategy and respond to the opponent, in this case isn't it like the single
agent only ever plays a single type of meta strat? If so, I think this is a
bit less impressive than what I had predicted.

~~~
codeflo
There's a rock-paper-scissors like aspect to StarCraft openings where you
essentially have to randomize your strategy so that the other player can't
blindly counter what you're doing. I assume each of their agents learns one
particular strategy; in that case, they could simply create a "meta-agent"
that acts like one of their trained agents at random (probably using some
weighted distribution).

~~~
devilmoon
Well, why wasn't this implemented then? For an hyped and live streamed event
I'm expecting to see something that blows me away, not hand picked agents
still in embryonic stage

------
diimdeep
10:0 AI wins. Precision micro control of units to save them from dying.

------
lawrenceyan
This was incredible. The games against MaNa even more so. TLO games were like
MNIST, but AlphaStar going against MaNa was like ImageNet level honestly. Hats
off to DeepMind.

------
porpoisely
I haven't played SC in years. Surprised that it is still going strong
considering I haven't heard much SC news in a while. I remember one trick
against the SC AI was to send a probe to the AI's base and lure all the AI's
probes out of its base. That was the easiest way to win against AI back then.
Comparing DeepMind/Alphastar now to what SC AI was a decade ago really puts AI
advancement into perspective.

------
Buttons840
Twitch.tv chat is full of shit-posting no doubt, but I did find some of the
comments amusing:

    
    
      - "You only have 50 GPUs in 2019, LuL"
      - "AlphaStar is a cyberbully"
      - Many were claiming the AI had "1500 APM", not sure where that idea came from
      - and lots more

~~~
gamegoblin
There were moments during clutch blink micro where the onscreen APM number for
AlphaStar exceeded 1000.

------
codeflo
I'm starting to get annoyed with these DeepMind publicity stunts. They don't
release any code, and don't let anyone verify their results. Their chess AI
beating Stockfish involved some at least a bit questionable setup. And here?
I'm a big fan of TLO, but he's currently not even in the top 50 even with his
main race. With Protoss, he's clearly making amateur-level mistakes. Why
choose this setting to show off your supposedly superhuman AI?

(Edit: This comment was made during the stream and it just looks like my point
will be addressed before the stream's even finished, they'll let the AI play
against a true top Protoss. Yeah!)

~~~
apetresc
The fact that the community has been able to use the published paper to
replicate all of AlphaZero's results in both chess, go (and a number of other
similar combinatorial games) and get similar performance should lay your
concerns to rest.

~~~
gambler
Can you provide some links?

~~~
methodover
LeelaChess is one example for Chess.

[https://github.com/LeelaChessZero/lczero](https://github.com/LeelaChessZero/lczero)

~~~
gambler
This project actually proves the point of the root comment. The community
spent tons of time on tuning/training this network, and it still routinely
loses to Stockfish, which runs on inferior hardware. It illustrates that:

1\. Deep mind kept a lot of information about their methods undisclosed.

2\. Despite all the claims of generality, the algorithm requires _insane_
amounts of fiddling to train. Just read their blog.

3\. The hype around AlphaZero crushing every other algorithm in chess is
overblown. It's competitive, but not clearly superior.

Still, Kudos to people running Lela project for doing what DeepMind should
have done - describing how things really work and testing in real-life
conditions.

------
knicholes
I'm having a hard time knowing if it's using a single agent of the five top
agents or an ensemble of their five agents.

~~~
saulrh
It sounds like each match in the Bo5 is against a different agent. It's a hair
dirty, but I don't think it's too bad because the same kind of random-
selection strategy is just as possible "in the wild".

------
throwaway19472
The singularity is near (5 yeats away max)

