
DeepMind – StarCraft II Demonstration - steve_musk
https://news.blizzard.com/en-gb/starcraft2/22871520/deepmind-starcraft-ii-demonstration
======
neovive
For those interested in building their own AI to play Starcraft II, here is a
very comprehensive, 17 video series by sentdex (Python AI in StarCraft II):
[https://www.youtube.com/watch?v=v3LJ6VvpfgI&list=PLQVvvaa0Qu...](https://www.youtube.com/watch?v=v3LJ6VvpfgI&list=PLQVvvaa0QuDcT3tPehHdisGMc8TInNqdq).
He walks you through building and training a neural network to play Starcraft
II via the same API used by DeepMind.

~~~
krasi0
And for those interested in making their own AI to play Starcraft 1 BroodWar
and become a part of the huge BroodWar AI community (get in touch on Discord
at: [https://discord.gg/w9wRRrF](https://discord.gg/w9wRRrF)), they can follow
the following tutorial:
[https://sscaitournament.com/index.php?action=tutorial](https://sscaitournament.com/index.php?action=tutorial)

Your AI would then be able to participate in tournaments and fight other AIs
on the existing ladders.

Additionally, one could watch BW bots fighting each other on:
[https://www.twitch.tv/sscait](https://www.twitch.tv/sscait)

~~~
sonko
And those who would like to follow developing a bot from scratch, I'd like to
plug in here my blog: www.makingcomputerdothings.com There, I also host a
podcast about SC AI development, which can be interesting for everyone. The
podcast is uploaded to youtube as well:
[https://www.youtube.com/channel/UCHPl6OFov2v8SK14oQUHY2w](https://www.youtube.com/channel/UCHPl6OFov2v8SK14oQUHY2w)

------
SketchySeaBeast
"DeepMind has been hard at work training their AI (or agent) to better
understand StarCraft II. Once it started to grasp the basic rules of the game,
it started exhibiting amusing behavior such as immediately worker rushing its
opponent, which actually had a success rate of 50% against the 'Insane'
difficulty standard StarCraft II AI!"

Well, that's fun. I mean, if it works, it works.

~~~
b_tterc_p
That doesn’t really sound like it understands the game... is this not training
by playing against itself? How is it not learning the flawed reasoning behind
this strategy?

~~~
chrisco255
"Cheesing" as it's called, is classic SC2 strategy. People absolutely hate it,
but it occasionally works, even though it feels cheap. It's kinda like
Blitzkreig in chess...

~~~
fernandotakai
it's a valid strategy that brought some super exciting moments, like boxer
bunker rushing yellow 3x at 2004 EVER OSL.

~~~
pvg
I believe the technical term you want here is 'legitimate strategy'.

[https://www.youtube.com/watch?v=bLMYTQUw8Lc](https://www.youtube.com/watch?v=bLMYTQUw8Lc)

------
wpasc
I wonder how Starcraft2 compares to Dota 2 in terms of difficulty for the AI.
Moreover, I'd be interested to see how OpenAI and DeepMind's techniques,
models differ in approaching these games.

~~~
WhompingWindows
The main difference would come in the simultaneous control of all of the
different units in Starcraft. Both MOBAs and RTS share many characteristics,
since Warcraft 3 was the origin of MOBAs to begin with. For instance, micro,
unit pathing, lowground highground, fog of war, spellcasting, etc. However, in
a MOBA there is more emphasis on team-play, on the interactions of 5-hero team
compositions, which is decided upon at the start of the game. There is also
itemization, which is much more stylistic than anything that can be purchased
in an RTS.

In Starcraft 2, it would be fantastic to individually control all 16 marines
in your 2 medivac stim timing, pulling them back when they become the targets
nearly instantaneously, perfectly shuffling them into/out of the medivacs,
while focus-firing the defenses. Another example would be a reaper rush, a
beastly AI could in theory micro 5-7 reapers individually, potentially
breaking the game as human fingers/muscles/nerves have limits that CPUs and
GPUs do not have.

The latter concern, about sheer speed and potentially limiting APM of the
Starcraft 2 AI, is a very interesting one. It would be interesting to allow
the AI to match the average APM of the Global Finals players, for instance,
which might be around 300 actions per minute. If the computer was allowed to
utilize 3,000 actions per minute, it could surely perform much greater feats
of micromanagement.

The largest edge I see human players having vs the Starcraft AI would be
something strategic, get a sense for how the AI plays then pull off a specific
strategy which counters it. It may not work well in the current human-meta,
but perhaps the AI will have its own meta that you can out-think it on?

~~~
vvillena
> In Starcraft 2, it would be fantastic to individually control all 16 marines
> in your 2 medivac stim timing, pulling them back when they become the
> targets nearly instantaneously, perfectly shuffling them into/out of the
> medivacs, while focus-firing the defenses.

Single unit pullbacks, medivac shuffling, and focus firing, all together, are
commonly seen in pro-level matches. The guys are superhumanly insane. The
current pro players can sustain 500 APM during battles. The only reason they
don't do this even more is that they are usually fighting on two or three
fronts at the same time, while managing base and unit production to the
second.

The real limit of the players is the single viewport and the single
instruction input pipeline. Managing dozens of units with those heavy
restrictions is pure madness. That's the main difference between RTSs and
MOBAs. MOBAs are defined by strategy and tactics, teams win and lose because
of those two factors (at pro levels). A game like Starcraft II adds another
factor: players can simply break down when they're not able to keep up with
the game. Even at pro level, players can be overwhelmed by the amount of
information they have to process in tenths of a second, over and over again,
for 15-25 minutes, or they can get into a spot where there's too much to do
and not enough time to do it.

AI has two advantages. It has better information channels, in the form of
perfect vision, while humans have a single pair of eyes that have to scan
through the screen. It also doesn't get tired over the course of the game.

~~~
make3
> Single unit pullbacks, medivac shuffling, and focus firing, all together,
> are commonly seen in pro-level matches. The guys are superhumanly insane.
> The current pro players can sustain 500 APM during battles. The only reason
> they don't do this even more is that they are usually fighting on two or
> three fronts at the same time, while managing base and unit production to
> the second.

Not to the level AI can do it. Did you see the video where zerglings dodge
tank shot splash? a human could never do that

~~~
vvillena
That zergling video depends on a zero reaction time, it's true that's an
advantage of AI. No matter how high the APM, if the reaction time is high
enough, some things just cannot be done.

Perfect micro against siege tanks is impossible for humans, because the time
between the beginning of the shot and the moment the damage is applied is seen
by us as instant. The game features other units where that time is bigger, and
pro-level games usually feature perfect micro on those instances (e.g. widow
mines, banelings, disruptors, locusts, hellions...)

------
alexandercrohde
I'm confused, I don't see anything new here. I looked at the youtube channel
and don't see a demonstration...

~~~
recursive
It's not published yet. It's scheduled for Jan 24.

~~~
benrbray
then maybe OP should've waited until Jan 24 to post this...

~~~
defertoreptar
But then I wouldn't know to set a reminder to see this live.

------
sct202
I wouldn't be surprised that an AI would be good at StarCraft. If it's allowed
to have high constant APM, an AI could do micro maneuvers that most people
can't reasonably do like consistent unit splits and fixing any weird unit
pathing issues at bottlenecks.

~~~
summerlight
They limit APM under 200 or something around that number.

~~~
gwern
180 APM, specifically: pg6
[https://arxiv.org/abs/1708.04782](https://arxiv.org/abs/1708.04782)

------
b_tterc_p
I have a hypothesis and I’m curious to hear responses. The ability of Ai to
work as a team with other AI has shown to be relatively weak. They can do
individual tasks well but are relatively agnostic to others (thinking the Dota
AI from a while ago).

Might this be improved by forcing the AIs to send messages to each other at a
constant interval? I feel like this would lead to the development of simple
language built around intentions and help the Ai to consider its allies.

~~~
simmanian
Im woefully qualified to answer any AI related questions, but I must ask, why
do you think forcing bots to send messages to each other at a constant
interval would lead to the development of simple language and encourage bots
to be more aware of each other? If you reward the behavior of writing in chat,
bots would start writing gibberish in chat all the time.

Communication in animals can be traced all the way back to cells sending each
other messages to organize themselves. Cells began to have division of labor
and work together, and as multicellular organisms increased in complexity, so
did the complexity in their communication. But what would happen if you put
these cells in charge of playing Dota and force it to send each other
messages? Would the cells somehow develop a simple language built around
intentions? Or does it still not understand the concept of intention?

Video games like Dota are extremely complicated and made specifically for
human biases. We've obtained these biases over many many many generations of
evolution. Machines are getting better at doing human tasks, but they still do
not possess all the human biases (and I think it'd be quite scary if we
actually get there). That's why these bots/algorithms/AIs can do simple tasks
like identifying objects really well but they don't know what to do with them,
yet.

Having machines engage in meaningful conversations with other machines and
plan together is a much bigger problem to solve than having them farm up, buy
items, and attack things.

~~~
b_tterc_p
I don’t think you understood my point. I’m not saying train the robots to say
things by rewarding them for doing so. I’m saying “force” them to do so as a
hard coded rule. And also force them to read the chats of the other AI allies
as an input. Continuously train the model to map the messages into an
embedding space and use the embedding for decision making.

This would likely benefit by some cherry picked training data, but I think AI
could get good at communicating their state to each other. Star craft has too
much entropy to make a simple example. Imagine the game was just, you pick a
random number and your ally has to guess what it is. I think Ai’s could learn
to communicate this using a similar training design.

------
DonnyV
I recently got my ass beat by a player that had a 400+ APM. Is that even
humanly possible or is it one of these bots?

~~~
lhorie
Definitely possible. A progamer like Serral for example can maintain 500+ apm
on 20+ min games, and can peak to 700+ apm.

~~~
postalrat
70 real actions and 630 useless clicks.

~~~
Veelox
This comment makes me think you have never played starcraft.

~~~
Rebelgecko
Have you? It's "easy" to get an APM over 1,000:

Step 1: Hotkey units on 1-9

Step 2: Slam face on keyboard numpad

Holding down a hotkey to build things (even if you don't have the minerals)
also increases your APM.

Lots of players (especially the pros) will spam actions in the quieter parts
of the early game to keep their fingers warmed up, or to start building a unit
the millisecond they have enough minerals to afford it.

------
fartboy69
As though StarCraft multiplayer wasn't difficult enough!

In all seriousness, this is pretty neat. Does anyone know if it already knows
everything on the map, or if it has to discover units like the rest of us?

~~~
SketchySeaBeast
> As though StarCraft multiplayer wasn't difficult enough!

But it just won't be the same without someone telling me that I'm awful and my
mother is of questionable morals.

... machine learning will probably pick that up too.

~~~
TulliusCicero
Personally I've found that Starcraft is one of the less salty/bm games out
there. At least in diamond 1v1, people doing "gl hf" at the beginning and "gg"
when they lose is common. And you don't have much time to whine in practice
because there's no downtime until either you or them are basically dead.

------
throwaway415415
I couldn't find an actual video of a game.

~~~
a_f
The demo is tomorrow on Twitch and Youtube (24th Jan)

------
fingerfoodtasty
In other news water is wet

------
anotheryou
tl;dr: wait for jan 27th

I really hate marketing pre-news...

------
oskkejdjdkjd
Everyone likes sci-fi. Star Wars, halo, Star Trek, etc. all these franchises
have something in common: they depict AIs working alongside humans. I think
that it’s an important demonstration of cognitive dissonance about AI. Fans of
these franchises aren’t bothered by the glaring contradictions that _some_
fictional elements create. For example, Star Trek features transporters that
teleport people across space and in reality the existence of such a machine
would change the _economics_ of many different things very drastically. Life
would look a lot different if teleportation were allowed to effect the
economics of human life. Another example is communication technology. Star
Trek had what we now call smart phones. But they still lived their lives the
way people did in the 80s. We now know that when smart phones are present,
people stare at them all the time, don’t go out much, are never in trouble
because they can’t communicate (car broken down) and etc. it’s not the best
example but it’s the only one that actually played out irl. My point here is
that people in general do not seem to appreciate the change in economics that
are prompted by new things — people are only able to imagine some new
technology being used in some niche case. It doesn’t bother anyone, or break
their suspension of disbelief, when some groundbreaking new technology
completely fails to alter the world and society in a realistic way in a
fictional setting. But any other breach of “realism” is somehow very
offending.

In halo, you have an AI that lives in your power armor and her name is
Cortana. Cortana can sense what you see, hear, etc. Cortana is seen to be
super smart on multiple occasions and it is clear to the player. This doesn’t
bother anyone. It bothers me now, though. If all of this is true, the
following would happen.

Cortana senses the presence of enemies faster than you. she hears them and is
able to identify how many based on their foot steps. She becomes annoyed with
how sloppy you are, how many enemies you didn’t even notice and how slow you
are to respond to them. She suggests that she take over your power suit. You
find yourself in a situation you can’t handle and are forced to give her the
reigns. She jumps into action dispatching enemies with speed and accuracy that
you’ve never seen before. She shoots enemies _as_ they step out from cover.
Your movements are so fast that you vomit. She wipes out army after army as
you passively watch, bewildered.

As you go through the game, you are on a mission to unravel the mysteries of
halo and find a way to survive. Cortana assesses the battle space and
concludes that you need to travel east. She notices several patterns of
ancient runes and directs you to investigate a hidden passage. Her assessments
seem incredible and bizarre but they consistently prove to be correct and
eventually she ends up making every decision.

At the end of the day, if Halo were real then it would be a story about a man
who passively sat inside a suit and watched an ai save the world. He probably
would die in the suit at some point due to experiencing high g forces during
some adventurous fight or maneuver and it would be a story about an ai who
saved the world with a dead guy in the suit. It doesn’t make you feel warm and
fuzzy like halo is supposed to, does it?

~~~
EForEndeavour
Wrong thread?

~~~
oskkejdjdkjd
My comment is about ai. The thread is for an ai post.

~~~
blackbrokkoli
It's not about an AI in the sci-fi sense though. It is about an actual real
world AI playing the game StarCraft II.

~~~
oskkejdjdkjd
My comment is about ai. It is about how people perceive ai, and sci-fi is a
very good demonstration of how incorrect human intuition can be when it comes
to ai. This incorrectness is present not only in stories about ai, but policy
and opinions regarding the very real and dangerous developments that are
taking place. Developments such as the one that this thread regards.

It is therefore only through deep, contemplative and unintuitive ruminations
that one can come to understand that AI is going to end human society as we
know it, and that the result will be very unpleasant. This is not a guess, it
is the truth. People have a huge amount of difficulty seeing this. I thought
the example of halo and Cortana would make it easier for people to understand.
I could have written it better but why bother? Nobody ever seems to get it.

