
DeepMind’s StarCraft II agent will play anonymously on battle.net - modeless
https://starcraft2.com/en-us/news/22933138
======
modeless
The most interesting part of this announcement, I think, is the new
capabilities and restrictions they disclosed for AlphaStar. It is now playing
the latest version of the game, with all three races unrestricted, on multiple
maps.

It also has much more realistic limits on its capabilities. Restricting the
bot to one screen of information at a time is great. There's not much detail
on the APM restrictions, but "designed with pro players" is a good sign.
Hopefully they've designed some realistic restrictions more complex than just
a cap on APM.

Of course the anonymous nature of the testing is interesting as well. Big
contrast to OpenAI's public play test. I guess it will prevent people from
learning to exploit the bot's weaknesses, as they won't know they are playing
a bot at all. I hope they eventually do a public test without the anonymity so
we can see how its strategies hold up under focused attack.

~~~
drchewbacca
APM is an interesting question I think. Like what is the relative importance
of APM to strategy?

If I have 20% more APM than you do I beat you even if I totally pick the wrong
units and build order? Could a super smart bot win against pro's even if it's
limited to 20% less APM?

So yeah it might be interesting if the bot gets superhuman to trail several
versions with progressively lower APM caps to see how much you have to limit
it before humans can beat it again.

~~~
mafuyu
APM is misleading because it can mean a lot of things. There are bots out
there that will perfectly micro units to win battles that would typically
trade out.

SC2 is really about attention management- every action you take, like moving a
worker to construct a building that you'll have money for in the future, takes
time and attention, and good players will take the minimal action needed to
get something done and then move their attention elsewhere while they wait
(almost like CPU pipelining). This is why players will do runbys/drop harass
at the same time as a big fight. It forces the defending player to spend more
attention than the attacker, or risk losing key workers/buildings. APM is just
a rough representation of the ability to multitask.

From this perspective, AlphaStar's APM doesn't really matter. If it can
instantly understand and juggle different scenarios occuring on the map at the
same time (scouting, macro, battles, drop defense), that immediately puts it
at an advantage, and many pro level strategies rely partially on pressuring
the opponent until they make a mistake.

The high level strategy stuff exists on top of all this, and that's what makes
SC2 so difficult. AlphaStar is really impressive, but once it understands the
game at some level, I think it's inevitably going to be better than human
players. Human players are going to get wins off AlphaStar by forcing tactical
errors in weird scenarios, but it's not going to look like typical human
games.

~~~
jammygit
It’s worth mentioning the huge advantage that impossible dropship or blink
micro can give. It isn’t raw apm, but perfect timing sometimes that is an
issue. The management of attention is truly important

~~~
ericd
Sounds like the AlphaStar team should add some randomized jitter lag between
when Alphastar issues a command and it’s executed. Would make it learn more
robust strategies that were less dependent on perfect timing, and might look
more like a human player.

------
dsjoerg
APM is effectively bandwidth, but just as important if not more important is
latency -- which is called TTFA, Time-To-First-Action. See
[https://illiteracyhasdownsides.com/2016/12/23/time-to-
first-...](https://illiteracyhasdownsides.com/2016/12/23/time-to-first-action-
and-comparative-player-analysis/) and the Skillcraft.ca study.

If they're not limiting AlphaStar's TTFA, then it can respond instantly to
problems all over the battlefield, which is superhuman in an uninteresting
way.

~~~
philipkglass
In any real-world application (robotic manufacturing, self driving vehicles,
precision guided munitions targeting...) extremely fast reaction times are an
essential and expected advantage of machines. I see AI game playing as
interesting mostly because it provides a way to pit software against people
without building a bunch of expensive physical infrastructure. If anyone ever
builds a "real" robotic army it will surely use all the APM and TTFA
advantages it can possibly achieve. In that sense, allowing the same
advantages in combat simulation games is closer to a proper man-machine
contest than one where those advantages are taken away from the machine.

~~~
bduerst
Precisely. Focusing on APM is kind of a red herring anyways because during
previous matches there were only a few milliseconds that it spiked above human
levels, and it was usually when moving workers:
[https://deepmind.com/blog/alphastar-mastering-real-time-
stra...](https://deepmind.com/blog/alphastar-mastering-real-time-strategy-
game-starcraft-ii/#image-15401)

Even so, this latest version has max APM limits instated to appease pro-
players. Since Alphastar is forced to perceive the state of the game through
machine vision of the screen, it's reaction time is already on par with humans
anyways (~350 ms for Alphastar vs. ~250 ms for humans).

~~~
friendlybus
Alphastar did some inhuman stalker micro that the APM debate seeks to cover,
but doesn't.

Stalkers have a player operated ability to instantly move a short distance
'blink' once every 5 seconds. When in a fight, you optimally let the
stalker(s) taking damage soak up as much damage as possible and then blink
them backwards so they can recharge shields and continue firing from behind
other stalkers. They don't stop firing, so all the work the opposing force did
trying to kill a shooter, resulted in no outcome at all.

That functionality is balanced by the fact it is hard for a human to time
activating the abilities of many stalkers at once in time with the damage they
are taking and perform the many other actions the game requires at the same
time.

Alphastar can perfectly blink back stalkers with limited apm because timing
things is obviously not a problem for it, making stalkers way more value-for-
money than they should be and can hold off high investment attacks more cost
effectively. Ultimately sc2 is a game of economy and timing so this small
change gives a massive advantage.

~~~
eric-hu
If I'm not mistaken, the blink micro was unrealistic because that version of
Alphastar played a modified version of the game. It's "screen" covered the
whole map and it had no need for a minimap. Human players can do some of the
blink micro that Alphastar did, if restricted to just one screen of space.

What made its micro different was that it did it consistently from flanks on
several sides of an army exceeding the boundaries of a human screen. It was
also notable that it did that while macroing at home, but some macro actions
during intense micro is done among better pros.

The blink micro advantage should be far reduced if Alphastar is playing the
same StarCraft II installation as humans now.

------
JyB
The AI they so proudly showcased last year learned to beat the limitations put
in place. It was really funny.

The DeepMind team put a cap on the _average_ APM it could perform over some
period of time. Being a micro intensive game, having a perfect micro even for
a short time is a game changer.

The AI just learned that by dropping it's APM low it would be allowed to have
insane and inhuman bursts during critical fights and still be within the
"rules" made up by the DeepMind team. It turned out it was one of the most
successful way to win games.

~~~
ehsankia
I feel like this really undersells what they showed though. Sure, APM was an
issue, but all the different and varied strategies in managed to come up with
was still really impressive.

It seems like they have since focused more on the "single screen" model, with
more restrictive APM limits (hopefully also limiting short peaks). I'm really
curious to see how far they've come. I'm assuming the big showcase will be
during Blizzcon.

------
mikenew
If AlphaStar is playing at the top of the ladder it probably won't be too hard
to figure out that it's AlphaStar. There's only around 15 or so players with
MMR above 6000, and they tend to know who they've been matched up against,
even if the opponent hides their name with a barcode (they name themselves
"IIIIIIIIIIII" so you can't recognize who it is. And AlphaStar has a pretty
distinct play style too.

Unless it's playing more towards the middle of the ladder where there's lots
of players I don't think the test would be very anonymous. Regardless, I'm
pretty excited to see what happens. Hoping to luck into a game.

~~~
herogreen
I am sure they will find a way to dampen its strength while improving it at
the same time. Some ideas: reducing number of neurons, reducing neural network
running frequency.

~~~
opportune
If they are using monte carlo tree search or some other minimax algorithm the
standard way to do this is to limit the depth of the search tree. But not sure
how they would use MCTS for starcraft. Could be they are using unsupervised
learning to figure out how to represent the search nodes

------
ctrlaltdylan
The other quieter announcement is that AlphaStar can play as multiple races.
The first public version just showed playing as Protoss.

There's some controversy about the AI's field of vision. A person is
restricted to using the small mini map for the "whole view" of the playing
area. Whereas AlphaStar can view events without a visual restriction.

But, if you're into following competitive games the broadcasted match of
AlphaStar vs Lambo is pretty incredible to watch. The AI used new and novel
strategies that weren't considered before.

The AI showed that Stalker can be actually a superior unit to the Immortals if
they can be micro'd efficiently.

~~~
Triesault
> There's some controversy about the AI's field of vision. A person is
> restricted to using the small mini map for the "whole view" of the playing
> area. Whereas AlphaStar can view events without a visual restriction.

That has changed and AI's 'view' is now restricted.

From the FAQ:

> Q. How does AlphaStar perceive the game?

> A. Like human players, AlphaStar perceives the game using a camera-like
> view. This means that AlphaStar doesn’t receive information about its
> opponent unless it is within the camera’s field of view, and it can only
> move units to locations within its view. All limits on AlphaStar’s
> performance were designed in consultation with pro players.

~~~
Fnoord
Feels like RTS games have not fully exploited the potential of multi-monitor
setup. Was the AI ahead of the game? Although eyes can only focus on so much
(screens would distract, I guess). Bigger monitors also would suffer from
peripheral vision effect.

~~~
scarejunba
Supreme Commander allowed you a minimal view on your alternative monitor. It
was neat but I felt it wasn’t particularly awesome or anything.

~~~
bsder
Supreme Commander also allowed continuous zoom because the maps were generally
much larger than Starcraft.

I _HATED_ the fact that Starcraft II wouldn't let me do this when it first
came out.

I loved Starcraft I, but the emphasis on twitchy play meant that I simply
couldn't be bothered to finish Starcraft II.

------
ralusek
The most disappointing part of AlphaStar to me was the fact that each game was
played by a completely separate AI. I was sad to see that each intelligence
seemed to only be able to converge on a single strategy/unit composition. It
was a failure to settle on a more generalized intelligence, as well as an
indication that many of its victories in reinforcement were potentially
achieved as a result of having composed units as hard counters to other
compositions.

~~~
s1artibartfast
Regarding your disappointment, human player goes into a game with an intended
strategy, how different is this different than selecting a strategy based AI
for each game?

I would like to rewatch the replays with this in mind, but the AI did appear
to transition between different unit compositions and strategies at different
points in the game. I wonder the degree to which this was predetermined
opposed to reactive.

~~~
pinouchon
It was pretty pre-detemined (at least in the matches vs Mana. The best pros
can react very fast to unusual strategies (cheese), or change their strategy
if the situation calls for it (eg: terran all-ins). I agree with the sentiment
of disappointment of having to make a "league" with specialised AIs. It seems
to be lacking a high level reasoning and executive module, and the league acts
as a crutch. The league idea still super impressive and clever, but human
players do not go around in gangs where only one of them takes a seat for each
match.

------
kregasaurusrex
Having watched the livestream from January[0], the AlphaStar agent was able to
create its own individual agents to play against and then learn from in the
same way AlphaZero learned from chess & baduk/go historic games. In
AlphaStar's case this learning period was described as experiencing the
equivalent of hundreds of years of aggregate gameplay.

This allows an agent to adjust to novel strategies not seen before because
Starcraft II's number of possible game states exponentially increases as the
game progresses; compared with chess and go that have large albeit fixed board
states. There's also a competition for individuals who build their own AI
engines[1] for Starcraft II much like how Stockfish and LeelaZero compete
against one another in their own respective AI chess leagues.

[0]
[https://www.youtube.com/watch?v=cUTMhmVh1qs](https://www.youtube.com/watch?v=cUTMhmVh1qs)

[1] [http://wiki.sc2ai.net/Main_Page](http://wiki.sc2ai.net/Main_Page)

------
z3phyr
Starcraft BroodWar is very easily accessible for writing bots. Check out
[https://bwapi.github.io](https://bwapi.github.io)

I encourage all the fellow SC players to experiment with it. (SC2 is a huge
download, and thus I never moved to try it)

~~~
dgant
The Brood War AI development community is very welcoming and supportive.

You can see bots in action at
[https://twitch.tv/sscait](https://twitch.tv/sscait)

Most discussion happen on Discord at
[https://discord.gg/QU457H](https://discord.gg/QU457H)

------
hartator
Does anyone know if DeepMind is fed raw images of the screen to process via ML
or it's already a parsed data of the game state?

~~~
marviel
From the article:

>Q. How does AlphaStar perceive the game?

>A. Like human players, AlphaStar perceives the game using a camera-like view.
This means that AlphaStar doesn’t receive information about its opponent
unless it is within the camera’s field of view, and it can only move units to
locations within its view. All limits on AlphaStar’s performance were designed
in consultation with pro players.

~~~
hartator
I don't think that answers the question. As it can just mean the AI has to
move a camera-like view (sic) but still doesn't have to use computer vision to
get which unit is which, health bars, and things in the like.

~~~
evv
It is a particularly interesting question in StarCraft because of the cloaked
units, which are visible only by distorting the background (and from their
sound).

Most humans will be able to identify cloaked units, but only if they look
carefully, and not in heavy battle. Can the AIs see and/or identify cloaked
units?

It begs the question if fair gameplay is even possible unless the AI uses
machine vision against the same interface. For fair human-vs-AI games, they
shouldn't allow cloaked units!

~~~
herogreen
I am not sure this question is really interesting because the aim of solving
Starcraft is to improve planning and strategies, not computer vision. I
suppose there are much more interesting tasks train on in order to solve
computer vision.

~~~
baddox
I agree. I'm not super impressed with the ability of human Starcraft players
to detect and parse the meaning of health bars. That's not really done by the
same part of the human brain that I would consider to be "good at playing
Starcraft." I'd be fine if the AI had a clean API to tell it the units that
are currently on screen, perhaps with some limit on the speed at which the AI
can iterate through everything (e.g. it should definitely require a "click" to
check an enemy unit's upgrades).

I do agree that detecting cloaked (or burrowed) units is a bit of an
interesting case, since a simple API like I mentioned would make it trivial
for the AI to detect these units. Perhaps you could do some sort of
probabilistic system, or some sort of "attention" system where the AI has to
choose where on the game screen to spend its limited attention, and more
attention in an area around a cloaked units would increase the probability of
the AI "detecting" the cloaked unit. That probably comes close to matching how
humans detect cloaked units, e.g. when you're deliberately looking for an
observer around your army you have an almost 100% chance of finding it if it's
there.

------
cf498
I wonder in which way an ethics board was consulted for this. Is it ethically
justifiable to introduce an anonymously "cheating" AI like DeepMind in the
competitive ladder? After all

> A win or a loss against AlphaStar will affect your MMR as normal.

While the scenario at hand might be relatively unproblematic, at what point
does an ethics board get consulted for anonymous trials concerning AI - human
interaction? Is this something on the radar of the DeepMind team?

~~~
leggomylibro
Also, is it legal in California?

[https://leginfo.legislature.ca.gov/faces/billTextClient.xhtm...](https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=201720180SB1001)

I'm not a lawyer, but it sounds like they're probably okay as long as the bot
doesn't advertise anything or incite people to vote.

~~~
cf498
I missed the part of the article which mentions getting matched against
deepMind is opt-in in the first go through. So everything should be fine.

~~~
baddox
It’s opt-in, and also it doesn’t even come close to matching the descriptions
in the bill.

> for the purpose of knowingly deceiving the person about the content of the
> communication in order to incentivize a purchase or sale of goods or
> services in a commercial transaction or to influence a vote in an election.

------
yters
I'm very interested in what sort of adversarial gameplay can be used against
these sorts of AIs. I've read that the chess versions all have weak points,
where if you know you are playing an AI, you can easily exploit them and win.
They usually seem to deal with strategies over many moves, since the AI look
ahead approach cannot go very many plies deep. I suspect that once a human can
break a micro spamming AI tactic he can consistently beat an AI with a more
long range strategy. And perhaps this approach is in general undefeatable by
an AI, since it seems that AI play can only either be short term spam attacks
or long play look up tables. The search space for short term play is
tractable, but long term play is exponential, so AI brute forcing will lose
out to human principled reasoning. Thus, without human input in the form of
game tables, the alphazero variant AIs can only optimize for short term
spamming. And the alphago/deep blue variants are always downstream from human
play, since they depend on human gameplay tables for deriving long term
strategy and insight.

~~~
philipkglass
> _I 've read that the chess versions all have weak points, where if you know
> you are playing an AI, you can easily exploit them and win._

That may have been true in the past but it hasn't been so for more than a
decade. A laptop running Stockfish handily beats top human players, even if
the players know a year in advance the exact hardware and software revision
they will be playing against.

[https://en.wikipedia.org/wiki/Human%E2%80%93computer_chess_m...](https://en.wikipedia.org/wiki/Human%E2%80%93computer_chess_matches#Kramnik%E2%80%93Deep_Fritz_\(2006\))

 _Kramnik, then still the World Champion, played a six-game match against the
computer program Deep Fritz in Bonn, Germany from November 25 to December 5,
2006, losing 4–2 to the machine, with two losses and four draws. ... There was
speculation that interest in human–computer chess competition would plummet as
a result of the 2006 Kramnik–Deep Fritz match. According to McGill University
computer science professor Monty Newborn, for example, "I don’t know what one
could get out of it [a further match] at this point. The science is done.".
The prediction appears to have come true, with no further major human–computer
matches, as of 2019._

~~~
yters
July 7 seems pretty recent:
[https://chess.stackexchange.com/questions/24867/a-horrible-s...](https://chess.stackexchange.com/questions/24867/a-horrible-
stockfish-chess-engine-evaluation)

~~~
philipkglass
Judging better than Stockfish on one board configuration isn't enough to win a
game, much less a tournament series.

~~~
yters
But it seems there is this general problem with long term strategy that led to
the board configuration misevaluation.

------
Havoc
That's so awesome.

Not just from a AI perspective but also gaming. Gaming AIs are on a consumer
level still pretty rubbish.

~~~
ryandrake
Indeed. Most “AI difficulty” settings in games involve either crippling the
player or giving the computer player extra bonuses/abilities. They are “how
much should the computer cheat” sliders. (I’m looking at you, Civilization) It
would be great if the norm in game AI was to allow the computer player equal
game resources/rules to the human player.

~~~
mikepurvis
It's probably gotten harder and harder to dedicate dev resources to building a
robust bot player for your game, given that the top tier of players are just
going to play online anyway.

~~~
opportune
It's not only dedicating the time, it's also hiring/maintaining a strong game-
AI team able to consistently pump out effective models at the same pace as
games are released. You can't just take a regular ol game developer and expect
them to make a really competitive game AI without cheating.

Even though the work is interesting I doubt you're going to be able to build a
full team of reinforcement learning experts for cheap. I would guess that
maintaining a 5 person team would cost about $2-3m/year.

~~~
vbezhenar
It might be easier "tomorrow", something like universal AI, where you plug
your game objects with some kind API, provide some hardware to simulate games
and it learns automatically. Something like Google does now, but more
universal and easier to use. Might be a good idea for startup :)

------
zwaps
Looking forward to AlphaStar learning to spam bad manner chat commands to
unnerv the other player

------
nitwit005
Is the screen view limitation much of a limit? You can scan through the map
quite quickly using the minimap to quickly move the view.

~~~
shawnz
They claim that adding the limit barely affected the performance of the bot.
So I'm guessing you're right. Requiring it to scan through different screens
is not really taking away the inherent advantage of being a computer; it just
means a few extra calls.

~~~
MichaelDickens
AlphaStar is APM-limited, which means it can't just rapid-fire click every
location on the map. It has to judiciously pick which places to look in the
same way humans do. (At least that's how it worked for the version they showed
last year.)

~~~
ergothus
> It has to judiciously pick which places to look in the same way humans do

There's a lot in that statement that can mislead.

It has to "look" \- that's true, AFAIK, and is super interesting. But that's
not the same as a human. If AlphaStar is watching 5 points of interest on the
map, there is basically no chance that it screws that up. For humans, that
would involve bouncing between the minimap to commands (assuming we're not
talking about hotkeyed locations), which invites lots of dexterity/accuracy
issues, not to mention visual comprehension.

Do you really think AlphaStar will have issues making out the blur of a
cloaked observer? (for example).

Consider some blink stalker micro - a human CAN select each unit as it's
attacked and blink it to the back of the pack, keeping a steady rotation of
fresh shields to tank the damage. But that is error prone, and even the best
pros only do it in limited skirmishes. Not because of APM - it's not the raw
clicks that are the problem - but because of accuracy and the opportunity
costs of the attention. Alphastar won't have accuracy problems, and the
opportunity costs of the attention are VERY different from the human costs.

I think Alphastar is a great experiment, and I am glad they are cutting off
some of the brute force advantages a program has vs a human opponent, but
that's not the same as saying it is doing it "the same way humans do".

~~~
tempestn
I suppose if they wanted to, they could pretty easily add a click accuracy
limitation as well, and force it to work around that, as with APM and map
view.

~~~
baddox
My prediction is that no matter what limitations you add, some people will
always reject the notion that a computer can be competitive with a human at
anything that humans are perceived or expected to be uniquely good at. In
chess, there were decades of resistance to the idea that computers were truly
competitive the best human players in a "fair" match, even though that game
has little or no physical component (I suspect this persists to this day).

For some people, either there will be endless litigation of every tiny
physical difference between the computer and human player that makes it
"unfair," or the premise will just be abandoned and we'll hear things like
"well, yeah, computers are great at RTS games, but RTS skill isn't really a
sign of true intelligence."

~~~
ergothus
As the objector above, I feel the need to defend myself despite the lack of an
attack :)

I think you are correct. There will always be those people.

I hope I don't fall among them - I raise the distinction about being "like" a
human not because I think it's makes a good/bad qualitative difference, but
because I'm hoping to avoid such comparisons. To me, far from proving it is
"good" or "bad" at the game, AlphaStar is interesting for the behaviors it
uncovers that are useful to humans. (example: AlphaStar overloaded workers - a
strategy that had been discarded by almost all pro years before, and is now
enjoying a reevaluation as a result). Paying attention to how the I/O is
different matters to such elements, even if it is pointless in the comparison
of "true intelligence".

FWIW, when it comes to AI I have a more Minsky-view of things (in my limited
understanding) and think that we're comparing apples and oranges without any
awareness that its all fruit - we only KNOW apples. I think AlphaStar already
has a better understanding of RTS than I do (low bar), even if we ignored the
differences. That, however, isn't terribly exciting. AlphaStar showing us new
tricks we can use - THAT'S interesting. (And now I want a segmented apple,
dangit)

~~~
baddox
I think philosophically the idea of humans competing with computers comes down
to balancing two sometimes opposing things: A) which types of skills are
interesting to test, and B) which types of skills are inherently interesting
for _humans_ to have.

For A, this comes up a lot in discussions about video game design and balance.
Do we really want to be testing how good players are at detecting cloaked
units or exactly counting groups of units in battles? I tend to think those
aren't super interesting strategically, tactically, or mechanically.

For B, there's a reason that human competitive weightlifting or sprinting is
still interesting, even though everyone knows that machines could trivially
win those competitions. Of course, those aren't really tasks that are
considered primarily measures of intelligence (although, see Moravec's
paradox). It's damn cool to see the limits of human ability stretched.

Of course, questions about expensive gear, performance-enhancing drugs, and
even prosthetics and cybernetics can already challenge our philosophy of what
makes human competition "fair." We inherently want to test the inequalities of
humans, both we're only interested in _certain_ inequalities. Generally, we're
interested in who can lift the most weight, not in who could take the most
growth hormones without dying.

------
Iv
Meanwhile, a pro-player hides his/her identity on the Korean ladder under the
name "AlphaStar" and was the top 10 a few months ago.

------
ggggtez
Very curious how this is going to turn out... This sounds like they are trying
to train a bot to actually _not_ be perfect? Otherwise I think they would say
straight out: "You'll only have the option if you are grand-master". Since
they don't say that, it feels like they are probably training the bot to "play
like it's at Gold League" etc.

If it could make the same types of mistakes and overlook things in the same
way a human would, that would be very interesting for game creators. Most game
creators would tell you: usually the hard thing is not making a good AI, but
making an AI that is fun, but still loses. We've all seen that AI can usually
just win any game, if you give it enough juice.

~~~
nicwilson
> This sounds like they are trying to train a bot to actually not be perfect?

(At least part of) The training has MMR as a label from unsupervised learning
from human games so as part of the configuration they can set the agent to
play like someone with a set MMR.

Lex Fridman's interview with Oriol Vinyalis on Lex's AI podcast covers this in
more depth.

------
jakegold
In January 2019, DeepMind introduced AlphaStar, a program playing the real-
time strategy game StarCraft II. AlphaStar uses a reinforced learning to learn
the basics of the Protoss race based on replays from human players, and later
played against itself to enhance its skills. At the time of the presentation,
AlphaStar had knowledge equivalent to 200 years of playing time; it won 10
consecutive matches against professional players, and lost just one.

------
jammygit
Just tried the mobile twitch app to check this out. Apparently you can’t watch
without logging in now. Amazon much want to know what we’re watching and
saying as a requirement of service

------
olliej
I suspect the micro will give it away to higher level players

~~~
MichaelDickens
I suspect it will play unusual opening build orders, like it did in the show
matches last year. That would give it away earlier in the game than its micro.
I'm hoping to see AlphaStar come up with some viable build orders that pro
humans haven't thought of.

------
cronix
Greetings, Professor Falken. Would you like to play a game?

~~~
solstice
I agree. do we want to teach AIs war? This seems like a decision/trend that
humanity might regret profoundly in a century or two...

------
gourou
I guess they're learning for OpenAI's mistakes on Dota2 (people were
exchanging strategies online)

[https://www.reddit.com/r/DotA2/comments/6t8qvs/openai_bots_w...](https://www.reddit.com/r/DotA2/comments/6t8qvs/openai_bots_were_defeated_atleast_50_times/)

------
Nimitz14
With its very weird playstyle not knowing one is playing against it will be
very annoying. Oh well, still curious to see how it will do.

------
gcoda
Effective APM limits is not enough.

No human can maximize target firing before each shot like AlphaStar did. Not a
lot of people even talked about it

~~~
defertoreptar
Sounds like they're addressing this concern:

> These caps, including the agents’ peak APM, are more restrictive than
> DeepMind’s demonstration matches back in January, and have been applied in
> consultation with pro players.

~~~
bduerst
Alphastar's APM wasn't even abused that much either back in January:
[https://deepmind.com/blog/alphastar-mastering-real-time-
stra...](https://deepmind.com/blog/alphastar-mastering-real-time-strategy-
game-starcraft-ii/#image-15401)

~~~
JyB
It was. It's APM burst of picture perfect micro in tense fights were insane.

Human player holding down a key that produces thousands APM certainly skewed
this misleading chart.

~~~
bduerst
This chart is all bots, not humans players...

~~~
JyB
Mana and TLO (red and yellow) are human players.

------
beders
I'm a gold league noob getting beaten by the run-of-the-mill smurf with a new
account. I couldn't tell if it is DeepMind or not ;)

------
Waterluvian
Website won't load for me. Sorry if it's already explained.

Does this implicitly mean it's okay if I make bots and let them play on battle
net?

~~~
JakeTheAndroid
It does not appear so, this seems to be a dedicate partnership not something
open to the public.

------
jammygit
I had no idea there were so many Starcraft players on hn. This thread reminds
me of team liquid :)

------
tomrod
If there is a SC3, a logic rule/AI element to the competition would be super
cool.

------
grumpy8
Will it start in bronze and then just crush everyone until it's rank #1?

~~~
ergothus
No different than every pro player smurf account, right?

~~~
grumpy8
right

------
ionwake
Does anyone know if it is possible to run Alphastar locally?

------
gigatexal
Comment deleted cuz I’m too busy to read details of articles.

~~~
dEnigma
It is opt-in, as it says right in the second paragraph and then again further
down the page:

 _If you would like the chance to help DeepMind with its research by matching
against AlphaStar, you can opt in by clicking the “opt-in” button on the in-
game popup window._

~~~
gigatexal
I should have skimmed the article better. Thanks.

------
jakegold
Humans tend to think we’re adept at the games we create, but computers have
proven time and time again that we’re just not fast enough to stay on top.
Machines have defeated us in chess, Jeopardy!, and even the deviously complex
board game Go. Google-owned DeepMind gets credit for that last one, and now
it’s dominating another game: StarCraft II. After just 18 months, DeepMind has
an AI that beats the world’s best StarCraft II players, and it’s not even
close.

DeepMind called its Go-dominating AI “AlphaGo,” and the StarCraft-playing bot
got a similar moniker. It’s called AlphaStar, and it has more than 200 years
of practice under its belt. Back at Blizzcon in November, DeepMind said its
machine learning platform had managed to beat the “Insane” difficulty in-game
AI about half the time. Well, it’s gotten much better since then.

AlphaStar is a convolutional neural network. The team started with replays of
pro matches, giving AlphaStar a starting point to begin playing the game.
Through intensive training with competing models, DeepMind was able to teach
AlphaStar how to play the game as well as the best human players. Over time,
it whittled the AI down to the five best “agents,” and that’s what it deployed
against some of the most skilled StarCraft II players in the world.

The matches actually took place in December, so today’s internet broadcast
mostly featured replays of those matches. First, AlphaStar battled a player
known as TLO, who primarily plays Zerg in StarCraft. However, he had to play
Protoss as that’s the only race AlphaStar trains with right now. This
competition wasn’t even close — despite TLO’s best efforts, AlphaStar beat him
five games to zero. Next, a different AlphaStar agent went up against a
seasoned Protoss player called MaNa. Some of these matches were closer, but
AlphaStar still won five games to zero. MaNa also competed against a new
AlphaStar agent live on the stream, and this time MaNa finally pulled out a
win.

AlphaStar demonstrated impressive micromanagement of units throughout the
matches. It was quick to move damaged units back, cycling stronger ones into
the front line of battles. AlphaStar also controlled the pace of battle by
bringing units forward and dropping back at just the right times to inflict
damage while taking less fire itself. This isn’t just a function of brute
force actions per minute (APM) — AlphaStar has substantially lower APM
compared with the human players, but it’s making smarter choices.

The AI also had some interesting strategic quirks. It often rushed units up
ramps, which is dangerous in StarCraft II as you can’t see what’s up there
until you move in. Still, it somehow worked. AlphaStar also eschewed the
tried-and-true tactic of blocking off the base ramp with a wall of buildings.
That’s StarCraft 101, but the AI didn’t bother with it and still managed to
defend its bases.

It wasn’t until the final live match that the human challenger spotted a flaw
in one of the agents. That version of AlphaStar committed to moving almost its
entire army as one with the intention of swarming MaNa’s base. However, MaNa
was able to repeatedly warp in a few units at the back of AlphaStar’s base.
Each time, AlphaStar would turn its army around to deal with the threat. That
gave MaNa enough time to build up a more powerful force and take the fight to
the AI.

At the end of the day, AlphaStar won 10 matches against pro players and lost
just one. If AlphaStar learned from that last match, it might be unbeatable
next time.

------
Taniwha
They'll have to up its trash-talking game

------
izzydata
That sounds like cheating doesn't it? Can I write a bot to play for me too?

~~~
rasz
Is it using special API giving it all the information all at once instead of
scraping the screen?

~~~
levesque
It is not looking at the raw pixels. It gets fed a bunch of arrays, but is
limited by what is currently on the screen and not in fog. The idea is that
this would be perfectly doable with state of the art machine learning and
would not measure our ability at making better AIs, merely add a failure point
to the pipeline.

~~~
rasz
It still has potential of being hugely unfair when only receiving everything
ordinary server broadcasts. As an example in World of Tanks server sends "a
tree has fallen" to the client even if said tree is behind 50 other trees,
totally in theoretical field of view! = there are cheats letting you know
about unseen enemy movement. Then we get to previous blunder, is APM still
averaged over last 5 seconds? FAQ carefully omits any numbers, merely
promising "more restrictive than demonstration matches".

Personally I am not a fan of training our future slave masters how to
efficiently kill human targets.

------
soup10
Honestly don't get the fascination with AI for Starcraft. Most of the skill
when humans plays comes down to who has better micro and macro mechanics. It's
not really a "thinking" game like chess or poker.

~~~
pkcsecurity
Watch some of the games and it might change your opinion. The thesis that "AI
in Starcraft will only win via improved mechanics" is false - the AI was
making some fascinating decisions / fundamentally different meta strategies.

~~~
MichaelDickens
Or watch [this
game]([https://www.youtube.com/watch?v=vUfwb4nOL84](https://www.youtube.com/watch?v=vUfwb4nOL84))
between the top StarCraft AI (outside of AlphaStar) versus Serral, one of the
top humans players. Unlike AlphaStar, this AI is not APM-limited; IIRC the top
bots tend to play with about 100,000 APM, compared to 400 for the top humans.
Serral won easily, despite the AI's vastly better mechanical skill.

------
gmueckl
Real time strategy games are peculiar because they are played mostly moment to
moment in a very reactive style with very little medium or long term planning
(beyond the next batch of units in production). So a neural network with a
tiny bit of memory should be able to play that by essentially learning
counters and a few harrassment and attack tricks. Sure, there's a bit more
involved, but when you look at it, any RTS strategy decomposes into a lot of
short term behaviours that are sequenced based on momentary triggers.

A more interesting challenge in my mind is a game that focusses genuinely on
longer term planning, where actions taken at a certain time lead to results 10
to 15 minutes later in game that is simulated in a continuous fashion (that
is, definitely not turn based and with no way to discretize it perfectly into
turns). The loss of a turn structure makes applying min-max-search based
strategies hard to apply - each side can make any number of moves in sequence.
Add non-discrete moves (e.g. turn unit with a certain turn rate for an
arbitrary amount of time, then move forward for another arbitrary amount of
time) and the search space becomes vast and unstructured, potentially even
unbounded. Solutions to that should be very interesting.

~~~
philipov
> Real time strategy games are peculiar because they are played mostly moment
> to moment in a very reactive style with very little medium or long term
> planning (beyond the next batch of units in production).

This is simply not true at a high level of play. Timing attacks, base
expansions, and tech transitions involve both medium and long-term planning.

~~~
gmueckl
Base expansions and tech transitions as actions are all triggered by certain
momentary events. This is simply due to the fact that each of necessary
actions for that have prerequisites. And a neural network will simply learn
long term strategies as sequences of trigger responses without any actual long
term memory involved.

~~~
philipov
if you're not thinking several moves ahead with your tech transitions, you're
going to have a bad time.

~~~
gmueckl
How big is the decision tree for tech? About ~100 nodes total? This is quite
small and handling that is not so impressive. That's roughly one move in a mid
game chess position (and you'd need to think two or three moves ahead to be
moderately successful at that game).

