It also has much more realistic limits on its capabilities. Restricting the bot to one screen of information at a time is great. There's not much detail on the APM restrictions, but "designed with pro players" is a good sign. Hopefully they've designed some realistic restrictions more complex than just a cap on APM.
Of course the anonymous nature of the testing is interesting as well. Big contrast to OpenAI's public play test. I guess it will prevent people from learning to exploit the bot's weaknesses, as they won't know they are playing a bot at all. I hope they eventually do a public test without the anonymity so we can see how its strategies hold up under focused attack.
If I have 20% more APM than you do I beat you even if I totally pick the wrong units and build order? Could a super smart bot win against pro's even if it's limited to 20% less APM?
So yeah it might be interesting if the bot gets superhuman to trail several versions with progressively lower APM caps to see how much you have to limit it before humans can beat it again.
SC2 is really about attention management- every action you take, like moving a worker to construct a building that you'll have money for in the future, takes time and attention, and good players will take the minimal action needed to get something done and then move their attention elsewhere while they wait (almost like CPU pipelining). This is why players will do runbys/drop harass at the same time as a big fight. It forces the defending player to spend more attention than the attacker, or risk losing key workers/buildings. APM is just a rough representation of the ability to multitask.
From this perspective, AlphaStar's APM doesn't really matter. If it can instantly understand and juggle different scenarios occuring on the map at the same time (scouting, macro, battles, drop defense), that immediately puts it at an advantage, and many pro level strategies rely partially on pressuring the opponent until they make a mistake.
The high level strategy stuff exists on top of all this, and that's what makes SC2 so difficult. AlphaStar is really impressive, but once it understands the game at some level, I think it's inevitably going to be better than human players. Human players are going to get wins off AlphaStar by forcing tactical errors in weird scenarios, but it's not going to look like typical human games.
More APM does not mean you have more attention to spend, it means you have better micro at the position you are currently spending it.
I'm pretty sure that if an AI would perfectly spread 100APM over three fronts, it would already play at pro or close to pro level.
If it plays at 3000APM, which it could do, it would beat pro's just because of micro which is less interesting.
You seem to be talking about an underlying, unknown parameter for which APM provides a crude estimator. When you use the term APM as a proxy for that hidden parameter, people will often mistake your position for claims about the raw number they see on screen, which does not perfectly correlate with the desired attribute.
In the case of APM, there are two major sources of error that make it a less-than-ideal estimator:
1) Not all actions are counted by APM. Moving the camera, moving your eyes to look at the mini map/minerals/supply counts/unit health, moving the mouse without clicking anything, pressing hot keys for targetable actions without confirming a target; these are all things which are not reflected in APM.
2) Many redundant actions are recorded as APM. Ordering a unit to move to one location and then clicking again in the exact same spot is a big one. Same goes for repeatedly selecting and deselecting groups of units without issuing any commands.
Given the above sources of error, it is not unreasonable to believe that a player with 40 APM can defeat a player with 400, it's just less likely.
The problem with extending this discussion to an AI is that the software does not suffer from these limitations. We can measure (and in fact control) the true number of actions the system uses to play the game. Unless programmed incorrectly, an AI should not waste any actions at all, so its measured APM won't be any higher than necessary. This makes it difficult to compare to a human player's APM.
Not all actions need to be.
> 2) Many redundant actions are recorded as APM. Ordering a unit to move to one location and then clicking again in the exact same spot is a big one. Same goes for repeatedly selecting and deselecting groups of units without issuing any commands.
The assumption here is that there is an end-all be-all metric for APM. It isn't. Just look at the discussions around EPM vs APM to get a taste. Anybody who thinks about APM understands that it isn't a 100% accurate measurement, and it just doesn't add anything to the conversation when everybody realizes that APM isn't 100% accurate.
Further, it's no accident that AlphaStar's APM has been limited, stemming from the kinds of advantages it provides in a game like SC2 where APM is a limited resource for every player such that it lays constraints on and is related to or is constrained by all kinds of things from attention to cognitive load to strategy to micro to macro. APM is itself a metric that's incredibly important to the game, not just a proxy. The fact that it's related to so many other parts of the game is just an indicator of how important it is.
Also, it's a big jump to say that all of AlphaStar's actions are effective and not wasted. At most you can say that there's no spam, because there's no need to. Pros will spam clicks just to keep their hands warm and moving, something which an AI doesn't need to do.
We don't know how our attention span is limited and how that relates to tactical planning.
The whole point of these APM shenanigans is to make sure that the competition is about intelligence, not interface. Even without any fancy technology at all, the computer starts out with a massive interface advantage. An API is just better that a screen and a mouse. It would be really cool if we could hook up a human brain to SC2, to give the human the same interface, but we can't. So we go the other way and limit the AI's Interface.
The final step in that would be to give the AI the same interface as the human, i.e. point a camera at the screen and connect a robotic arm to the mouse. But that just creates computer vision and robotics challenges, which are probably seen as a distraction at the moment.
But such a robotic arm would definitely have a limited APM, so the current approximations don't seem to bad to me.
Limiting map vision seems a kind of arbitrary limitation to me, an artifact of the particular minimap size Blizzard chose that you can't change; you should be able to change its resolution, size and placement as you like, and then face the computer on roughly equal terms (without its ability to do ridiculous micro, i.e. try to produce pure strategy games).
Then you could restrict the AI to a certain amount of memory that is available to it. You could also play with latencies/exactness of the memories that is available to it.
Thing is, you are probably going to pick units that require more APM. Some units like the blink stalker really benefit from micro-management if you can do them quick enough. If you have enough APMs to juggle 5 or 6 immortals out of 2 warp prisms, you have a huge advantage even in front of an opponent with a superior strategy.
And in the previous rounds, AlphaStar did have heavy APM limitations but they were criticized a lot because they allowed for very high APMs bursts. Being able to perfectly micro-manage engagements of 20+ blink stalker is going to do the equivalent of doubling your unit count.
The other thing that differs from a human is that as the APM increases, the precision can suffer. Humans will misclick, will fail to select all the units they want, etc... AlphaStar will not because it essentially sees Starcraft as a turn-based game and can react perfectly to each tick.
> Could a super smart bot win against pro's even if it's limited to 20% less APM?
AlphaStar cheesy game against Mana (https://www.youtube.com/watch?v=GKX6AcgFOZU) convinces me that yes. It already happened, with a very creative strategy and adaptation.
An AI which doesn't misclick or toggle between buildings constantly should in the long run probably be limited to a much lower burst apm to be comparable to humans.
I am looking forward to seeing it applied to a problem where we don't tie one its hand behind its back though.
There are probably real life situations (hopefully not war, although that will happen for sure) where being able to focus on 10 things at the same time and react extremely quickly without any fatigue would make an AI way more performant than humans.
Come to think of it, fatigue might manifest in thermal throttling unless we conceive its chassis better than we do our phones and laptops.
One more reason why I am saddened that SupCom is not more popular.
It would be very interesting to see AlphaSC trounce humans at this game.
The far away view that was called "cheating" in the alphaStar exib match ? In this game it is just what high level players use most of the time (the zoom level is free.. from way too close to a continental view)
Ketroc plays Master level Terran SC2. His APM is poor, he sometimes gets supply blocked, his micro isn't great. But his strategic understanding is excellent. Ketroc grasps exactly what's going on, and also how other players are paying the game, which leads to him winning at Masters level (not pro but very good) where most competitors have high APM.
Ketroc was famous in early SC2 for "that blue block is moving" a phrase from commentary of a game in which his opponent takes Ketroc's base but Ketroc's superior strategic understanding leads to an unconventional win.
https://www.youtube.com/watch?v=FbfIVf49zXA&t=530s is Day9 commentating on Ketroc doing something similar, but now aggressively, for a "FunDay Monday" (Day9 fans are tasked to play SC2 in some particular way, partly as training mostly as amusement).
At 20%, no. Pro players beat other pro players with 20% more apm all the time, even if they pick reasonable unit comps (this is especially common when terran players do tank focus builds against zerg). At 200%, especially if it's effective and not spamming, maybe. There are videos of bots microing marines at very high apm to beat big baneling balls, which cost much more resources and are supposed to directly counter marines. Protoss players could have perfect blink micro or immortal dropping, etc.
He blinked stalkers at once from 3 sides of MaNa army (which was several-screens-wide). Such micro wouldn't be possible for a human player, because you would need to know to move the screen to see a particular stalkre just in time to blink it when he has almost 0 hitpoints, and immediately move the screen so you see another stalker on other part of the map to blink it, and so on.
And the bot did this, making the strategy part of the game irrelevant (blink stalkers are supposed to lose vs immortals).
That is also why amateur players copying pro-players can loose to a "weaker" unit combinations. Because when you do things at 50 actions per minute you will not get the advantages of certain unit types.
In the context of AlphaStar it was clearly visible in its first games. The AI chose ranged units and every commentator during that game was saying that AI has an inferior army. However when within the battle its APM were allowed to reach 1000 that type of unit became the best possible unit to have.
It outmicroed MaNa (one of the most micro-focused protoss players) in several games, attacking and winning from positions it shouldn't be possible to do so. In one game it countered MaNa's big immortal-archon army with pure blink stalkers. The general consensus is that immortal-archon is the ultimate counter to stalkers :) The bot was doing perfect blink stalker micro from 3 sides of MaNa army at once over an area that was like 3-screens wide.
It would be physically impossible for a human player to play like this, because moving the screen to blink each stalker as it gets near 0 hitpoints back and forth just can't be done fast enough, with 50 stalkers fighting - but for a bot it's easy.
After these games they limited the alphastar to have limited speed of moving the camera, and that bot lost to MaNa (but it was also trained for less time so it might be not a good comparison), and MaNa strategy was a little cheesy in that game - harrasing with warp prism making the bot move his army back and forth.
Starcraft 2 is as much a strategy game as a mechanical arcade game, different mechanical restrictions change which strategies work well vs which other strategies. The balance of the game is designed with human limitations of micro in mind, so when a bot plays it has to imitate these restrictions, or the game isn't fair.
With perfect micro mass marines beat everything with ridiculous cost-effectiveness.
Even from the games where MaNa lost, AlphaStar made incredibly poor big-picture strategic judgements, in terms of where to deploy its forces. It can't seem to smartly deal with harassment.
Show me an alpha star with robotic arms and camera eyes using a mouse and keyboard. Then we’ll be playing the same game.
Or, I suppose, give humans a non-physical neural interface.
What has changed in the bots, since the first competitions where having hundreds of zerglings used to beat all the other entries?
Another way to formulate this is: if you made AS as broken as it could possibly be by micro, what macro strategies would humans have to discover to beat it?
Ignoring APM - what if it turns out that the better of two players isn't the one who makes better strategic decisions, but the one who can issue more effective tactical commands in a short burst? We can't measure that as it isn't APM but it is likely to be a huge factor. StarCraft AI have historically been handicapped by being unable to model tight tactical situations - if machine learning can come up with any model at all that works (and we now know it can) computers are simply so much better at making quick decisions that there is little left to see that is interesting. There isn't much left to prove.
I wouldn't so much talk about APM, as about time-costs of information.
In real-time games like Starcraft, you have to spend time to look around the map and figure out what's changed in each area since you last saw it. The more of this "espionage" you do—and the longer it takes you to absorb what you see—the less time you can dedicate to actually doing things, making APM matter more and more.
This—presumably—won't be a problem for AlphaStar, though, because with a limited APM, it will be able to process the information it receives far faster than it will be allowed to emit actions. Think of it like a robot that could read (and absorb!) a book by just flipping through the pages at speed. If AlphaCraft only needs to just slide their viewport across the whole map as fast as the client allows scrolling to happen, in order to learn everything about what's going on, then the time cost of their information is very low, so their APM is basically irrelevant. They have all the time in the world to do whatever they like.
I'm looking forward to the presumptive second stage (AlphaStarZero? ZeroStar?) where the AI will train by playing itself. Because both sides would have equal APM potential, there's no reason to limit APM in that case. And, therefore, the game becomes a much more real equivalent to a normal StarCraft match, because once again the AI is put in a situation where the other player is reacting quickly-enough to their actions than they have to weigh "time spent learning what happened" against "time spent performing actions of their own based on incomplete information."
People in the comments made points about "perfect timing" and "management of focus" and I concur wholeheartedly.
Depends on the strategy, and the skill level you play as, and what that APM is being used for.
Among the lower 90% of play, a player with a very good understanding of strategy, and good prioritization of their actions (Spend money, don't get supply blocked, don't lose your army to a dumb blunder) will win ~100% of their matches with ~40 apm, assuming they plan to use a low-apm strategy (no fancy drop play, preference to units that don't require a lot of micromanagement.)
On the masters/pro scene, though, you need a lot of APM just to stay alive. When there's three medivacs cycling drop harass in your main and natural, while helions keep running into the worker line of your fourth base, while your opponent's army is jostling for position with yours, trying to get a good angle to land EMPs, or to siege up in a good position, you're not going to be able to physically stay alive, unless you are devoting over a hundred APM to dealing with these threats.
> If I have 20% more APM than you do I beat you even if I totally pick the wrong units and build order?
That depends on what 'the wrong units and build order' means.
Some build orders will straight up die to other build orders, assuming perfect play on both sides. Assuming non-perfect play, it depends on which mistakes are made by each side.
Some build orders are soft-countered by other build orders. They can, theoretically, hold against a 'countering' build order - but will die, if the person using them makes some mistakes, and their opponent makes none.
Likewise, 'the wrong units' is huge a bag of worms. Are zealots better units, cost-per-cost than marines? Yes, unless there's a critical mass of marines, at which point zealots become the 'wrong units' for that engagement. Throughout a game, there are ebbs and flows of strengths and weaknesses, between two players, as they hit their various tech timings/unit counts/positional advantages, and try to transform those short-term advantages into economic damage, or efficient army trades.
A big part of pro play is knowing when you have an advantage you can exploit, how much you can exploit it, how to exploit it efficiently. You need game sense for the first, good strategic thinking for the second, and APM for the third.
 I'm a pretty casual player, but I've had no problems hitting Master (5%) level in every season I've cared to play. I generally sit at ~90 APM, when I don't spam - and I presume that if I were much smarter, better with my timings, and more knowledgeable about the game, I could be in the 10th percentile with half the APM. The amount of extra game knowledge that pro players have, compared to me, is staggering.
When trying to solve problems, if you add constraints in certain areas, you can sometimes come up with really creative solutions you wouldn't have thought of before.
I'm actually curious on the details of this, because it makes for a huge difference. The pro players spend most of their time looking at the mini-map, which shows you what's happening all over the map, in areas where your units/buildings have field-of-vision. You can't see exact units/buildings on the mini-map, but you can make out unit movements as well as expansions. Not giving AlphaStar access to the mini-map would be a severe handicap.
It would be interesting to see how the AI distributes it's views if it were capped in some significant way.
Also, can the bot only issue commands on one screen at a time?
If they're not limiting AlphaStar's TTFA, then it can respond instantly to problems all over the battlefield, which is superhuman in an uninteresting way.
The original commenter's remarks about TTFA seems to be equivalent to what I'm referring to.
Relevant reading: https://www.alexirpan.com/2019/02/22/alphastar.html
Has it? I'm not a RTS player but I thought that simple bots couldn't compete with top StarCraft players just with speed. It required strategy plus speed to first become competitve (though less strategy than required with slow APM and TTFA).
One such video (skip to about 4:30 for opinions on mechanics): https://www.youtube.com/watch?v=EP9F-AZezCU
Alphastar does already factor out reaction time:
“We measured how quickly it reacts to things,” Silver said. “If you measure the time between when AlphaStar perceives the game. From when it observes what’s going on, then has to process it, and then communicate what it chooses back to the game. That time is actually closer to 350ms. That’s on the slow side of human players.” https://venturebeat.com/2019/01/24/alphastar-deepmind-beats-...
That may be true in some narrow technical sense, but it was heavily disputed by the StarCraft community. If you watch the exposition games, AlphaStar has superhuman ability to micro-control its units one-by-one during critical battles, which allowed it to beat its human opponents even when its army was far inferior on paper.
For instance, imagine you’re playing a chess game with a 100 ms timer and first to exhaust their timer loses. No human will win and I could create a program that could best Kasparov trivially by advancing each pawn. There’s the game and then the input layer problem.
Maybe Civilization IV ;)
Even so, this latest version has max APM limits instated to appease pro-players. Since Alphastar is forced to perceive the state of the game through machine vision of the screen, it's reaction time is already on par with humans anyways (~350 ms for Alphastar vs. ~250 ms for humans).
Stalkers have a player operated ability to instantly move a short distance 'blink' once every 5 seconds. When in a fight, you optimally let the stalker(s) taking damage soak up as much damage as possible and then blink them backwards so they can recharge shields and continue firing from behind other stalkers. They don't stop firing, so all the work the opposing force did trying to kill a shooter, resulted in no outcome at all.
That functionality is balanced by the fact it is hard for a human to time activating the abilities of many stalkers at once in time with the damage they are taking and perform the many other actions the game requires at the same time.
Alphastar can perfectly blink back stalkers with limited apm because timing things is obviously not a problem for it, making stalkers way more value-for-money than they should be and can hold off high investment attacks more cost effectively. Ultimately sc2 is a game of economy and timing so this small change gives a massive advantage.
What made its micro different was that it did it consistently from flanks on several sides of an army exceeding the boundaries of a human screen. It was also notable that it did that while macroing at home, but some macro actions during intense micro is done among better pros.
The blink micro advantage should be far reduced if Alphastar is playing the same StarCraft II installation as humans now.
If you believe Aleksi Pietikäinen -- and pretty much every one of the professional players who played against it -- the claim you're repeating here is so misleading as to be fairly considered an intentional lie on the part of the deepmind team.
For example, TLO's inflated APM are presented in that chart without comment. Specifically, without the comment that his high APM counts come from a particular game context in which holding the mouse button down (i.e. a single click with a duration) is counted by the game as thousands of APM.
AlphaStar APM spiked to ridiculous inhuman levels during stalkers micro. On top of that, it controlled units that were screens apart at the same time, which is not supposed to happen.
DeepMind's refusal to aknowledge it, on top of the sketchy
and misleading TLO chart didn't do them any favor.
Reference? I'm under the impression that the game provides several "layers of information" directly to AlphaStar, not the actual screen.
> Q. How does AlphaStar perceive the game?
> A. Like human players, AlphaStar perceives the game using a camera-like view. This means that AlphaStar doesn’t receive information about its opponent unless it is within the camera’s field of view, and it can only move units to locations within its view. All limits on AlphaStar’s performance were designed in consultation with pro players.
Your assumption us correct about the show matches against MaNa and TLO that many people are talking about in here. That was not the long term goal of the Alphastar team to keep it on that heavily modified version of the game. For one thing, it meant that Alphastar needed a customized version of the game that it couldn't play on the ladder. As far as an AI challenge goes, it's also really weak if the AI gets more direct access to game data than its human opponents.
Because in the real world, you have to get a bunch of sensor data, potentially run it through it's own neural net to recognize objects, and then feed it into the decision making system. All that takes time - most likely much more time than the 1 frame it takes for the AI to make an API call.
If the AI actually played through the same interface as humans, i.e. it simply gets the rendered image as an input, and produces mouse/keyboard inputs as an output, then maybe we should disable artificial APM/reaction time limits. But as it stands now, the AI has an absolutely massive advantage simply due to using a much better interface, which it won't have in the real world.
Not necessarily true. Yes, the machines are faster, but sometimes that is not a good thing: https://arxiv.org/pdf/1906.09765.pdf
Knowing speed runners and the like, I can imagine they will quickly find a way to determine if the player is in fact DeepMind via some sub milisecond method.
If you watch the first-person view of top human professionals it already looks pretty instant or "mechanical," and is sometimes hard to follow for me even as a semi-competent player and avid spectator. What's the TTFA for a pro when an enemy drop appears on the minimap? I would guess somewhere around 200ms at the quickest? That would be similar to the latency of the DeepMind neural network (supposedly 350ms). And, of course, Starcraft 2 already has an approximate input latency of 200ms (so that all players can receive all other players' inputs and run them against their game state).
At the end of the day I don't see pure reaction speed as being a huge issue in Starcraft 2. Perhaps it would give a computer an "unfair" advantage in some rare cases like two cloaked ghosts running into each other and trying to snipe each other.
This is how APM is treated in the professional scene as well. Everyone knows that bursts of 500 APM when you're spamming at the beginning of the game aren't some incredible display of skill. But sustaining a good number of useful actions per minute is incredibly important, and a huge part of the mental process in Starcraft 2 is constantly deciding where and how to invest your actions.
TLDR: Both their APM and TTFA is comparable to human pros.
> AlphaStar has built-in restrictions, which cap its effective actions per minute and per second. These caps, including the agents’ peak APM, are more restrictive than DeepMind’s demonstration matches back in January, and have been applied in consultation with pro players.
So yeah, they've tweaked this specifically, although not much details as to how.
The DeepMind team put a cap on the average APM it could perform over some period of time. Being a micro intensive game, having a perfect micro even for a short time is a game changer.
The AI just learned that by dropping it's APM low it would be allowed to have insane and inhuman bursts during critical fights and still be within the "rules" made up by the DeepMind team. It turned out it was one of the most successful way to win games.
It seems like they have since focused more on the "single screen" model, with more restrictive APM limits (hopefully also limiting short peaks). I'm really curious to see how far they've come. I'm assuming the big showcase will be during Blizzcon.
There are videos from several microbots such as Automaton 2000: https://www.youtube.com/watch?v=IKVFZ28ybQs that showcase what super APM can do. The DeepMind AI that beat some pro players did it by having better micro while generally having worse strategies. The DeepMind matches showed how better micro can turn a generally weaker unit composition into the winning unit composition just as that Automaton 2000 video did.
To me it seemed that the DeepMind team figured out that the only way to beat competent players was to do what all those microbots do and pump APM into godly levels. They decided to limit APM, however like you said, just average APM and have the bots explode APM when needed. The real funny match was when MaNa beat Deepmind by completely breaking the AI with generally simple drop strategies and made Deepmind look very much inept.
Meh. That statement is euqivalent to "Turns out being able to calculate 400.000 chess positions per second it one of the most successful ways to win games". Given perfect Micro, SC2 is a totally different game that what we are playing.
Unless it's playing more towards the middle of the ladder where there's lots of players I don't think the test would be very anonymous. Regardless, I'm pretty excited to see what happens. Hoping to luck into a game.
There's some controversy about the AI's field of vision. A person is restricted to using the small mini map for the "whole view" of the playing area. Whereas AlphaStar can view events without a visual restriction.
But, if you're into following competitive games the broadcasted match of AlphaStar vs Lambo is pretty incredible to watch. The AI used new and novel strategies that weren't considered before.
The AI showed that Stalker can be actually a superior unit to the Immortals if they can be micro'd efficiently.
That has changed and AI's 'view' is now restricted.
From the FAQ:
IANAPGM, but one can observe that TA and SC are indeed different RTS games in the sense that TA has a territory control focus (so perhaps a bit more strategic), while SC relies more on the micro/macro focus (workers, some skillshot abilities - which incidentally led to the birth of the MOBA genre).
Depending on how that's implemented, I could see that rapidly becoming de facto "pay to win".
I HATED the fact that Starcraft II wouldn't let me do this when it first came out.
I loved Starcraft I, but the emphasis on twitchy play meant that I simply couldn't be bothered to finish Starcraft II.
I know several of the later games, and even the first games (as far as I remember) allowed two fully independent screens with the UI buttons only appearing on the first one - I used the heck out of it.
Thank you, The Register
I would like to rewatch the replays with this in mind, but the AI did appear to transition between different unit compositions and strategies at different points in the game. I wonder the degree to which this was predetermined opposed to reactive.
This allows an agent to adjust to novel strategies not seen before because Starcraft II's number of possible game states exponentially increases as the game progresses; compared with chess and go that have large albeit fixed board states. There's also a competition for individuals who build their own AI engines for Starcraft II much like how Stockfish and LeelaZero compete against one another in their own respective AI chess leagues.
I encourage all the fellow SC players to experiment with it. (SC2 is a huge download, and thus I never moved to try it)
You can see bots in action at https://twitch.tv/sscait
Most discussion happen on Discord at https://discord.gg/QU457H
Basically SC1 was reversed via memory location reading and was long the staple playground for AI competitions. University teams and what not.
Blizzard knew of this and silently condoned it. When SC 2 came around they helped move things over because who doesn't want that epic AI PR.
Not 100% sure how it's achieve technically on SC2 but blizzard is definitely actively helping push this technically. (within limits - they counterweight is they don't want to help cheaters hack their stuff)
>Q. How does AlphaStar perceive the game?
>A. Like human players, AlphaStar perceives the game using a camera-like view. This means that AlphaStar doesn’t receive information about its opponent unless it is within the camera’s field of view, and it can only move units to locations within its view. All limits on AlphaStar’s performance were designed in consultation with pro players.
Most humans will be able to identify cloaked units, but only if they look carefully, and not in heavy battle. Can the AIs see and/or identify cloaked units?
It begs the question if fair gameplay is even possible unless the AI uses machine vision against the same interface. For fair human-vs-AI games, they shouldn't allow cloaked units!
I do agree that detecting cloaked (or burrowed) units is a bit of an interesting case, since a simple API like I mentioned would make it trivial for the AI to detect these units. Perhaps you could do some sort of probabilistic system, or some sort of "attention" system where the AI has to choose where on the game screen to spend its limited attention, and more attention in an area around a cloaked units would increase the probability of the AI "detecting" the cloaked unit. That probably comes close to matching how humans detect cloaked units, e.g. when you're deliberately looking for an observer around your army you have an almost 100% chance of finding it if it's there.
The github repo shows that the AI takes an Observe action, and it essentially gets data on every unit it can see in a machine friendly format.
The previous version got info on every unit it could conceivable witness, including units that were offscreen, but technically 'visible' according to the game rules. This version seems to have changed that to only return units that are actually /on/ the screen.
It might be "unfair" for some almost purely number-based confrontation like a phoenix-versus-phoenix battle. But are we really that impressed by testing which human happened to be better at estimating the number of stacked phoenixes? To me, that's not what's interesting about competitive Starcraft 2. I'm more interested in micro, like pulling back injured units, or choices like "it makes sense for me to force us to trade out our phoenixes right now even if I lose the battle."
Because of that they have a problem with invisible and burrowed units - human players can miss the shimmer/shadow because it's barely visible, but their AI sees it every time. I think they gave up on simulating this stochastic human behavior.
Still, the game is complex enough that the fact that they can play it at a professional level might as well be a miracle.
> A win or a loss against AlphaStar will affect your MMR as normal.
While the scenario at hand might be relatively unproblematic, at what point does an ethics board get consulted for anonymous trials concerning AI - human interaction? Is this something on the radar of the DeepMind team?
I'm not a lawyer, but it sounds like they're probably okay as long as the bot doesn't advertise anything or incite people to vote.
> for the purpose of knowingly deceiving the person about the content of the communication in order to incentivize a purchase or sale of goods or services in a commercial transaction or to influence a vote in an election.
That may have been true in the past but it hasn't been so for more than a decade. A laptop running Stockfish handily beats top human players, even if the players know a year in advance the exact hardware and software revision they will be playing against.
Kramnik, then still the World Champion, played a six-game match against the computer program Deep Fritz in Bonn, Germany from November 25 to December 5, 2006, losing 4–2 to the machine, with two losses and four draws. ... There was speculation that interest in human–computer chess competition would plummet as a result of the 2006 Kramnik–Deep Fritz match. According to McGill University computer science professor Monty Newborn, for example, "I don’t know what one could get out of it [a further match] at this point. The science is done.". The prediction appears to have come true, with no further major human–computer matches, as of 2019.
Not just from a AI perspective but also gaming. Gaming AIs are on a consumer level still pretty rubbish.
Even though the work is interesting I doubt you're going to be able to build a full team of reinforcement learning experts for cheap. I would guess that maintaining a 5 person team would cost about $2-3m/year.
But for games like grand strategies (eg civ), the player population doesn't typically run high enough to support only humans each match (at least partially because the games run too long), and so its more important to develop a decently sane AI.
I don't think so. Years ago gaming programming books were all about how many CPU cycles can you spare for AI. Literally omg the graphics are too slow we have to make the AI stupider.
These days you can throw spades of power at it (comparatively).
This game was made 15 years ago and every time you play the game (just like the guy says) the encounters are different. There are ways to cheese the game still but it isn't easy to do it and tbh it almost not worth doing.
I think the problem is that a lot of games have been constrained by two things consoles being quite limited compared to PCs and the industry trying to constantly sell graphics and celebrity voice acting over gameplay.
Most of the open world games except for GTA are essentially pretty stale.
Also, we need a way to put all those new cores AMD is bringing to desktop CPUs to good use :)
For a FPS like doom its trivial to make a "perfect" AI that never misses.
Much of the focus is on RTS though. (Real time strategy - like starcraft)
It's a little difficult to describe but practically the "quality" of a AI is very obvious to seasoned players. i.e. You feel this & there is no masking the truth easily.
e.g. AI opponent nukes my bases where maximum damage is inflicted. Meanwhile I know the AIs scouts didn't have visibility of that area. There is no way the AI could have legitimate know that's where to place the nuke for max effect.
The AI won. It didn't outsmart me. It cheated via being omniscient and having full sight of map while I'm constrained by "fog of war".
Excellent AI - killed me. Yay.
Bad game. Bad experience. Unhappy user.
Shoutout to the Queller AI mod on planetary annihilation with titans.
There's a lot in that statement that can mislead.
It has to "look" - that's true, AFAIK, and is super interesting. But that's not the same as a human. If AlphaStar is watching 5 points of interest on the map, there is basically no chance that it screws that up. For humans, that would involve bouncing between the minimap to commands (assuming we're not talking about hotkeyed locations), which invites lots of dexterity/accuracy issues, not to mention visual comprehension.
Do you really think AlphaStar will have issues making out the blur of a cloaked observer? (for example).
Consider some blink stalker micro - a human CAN select each unit as it's attacked and blink it to the back of the pack, keeping a steady rotation of fresh shields to tank the damage. But that is error prone, and even the best pros only do it in limited skirmishes. Not because of APM - it's not the raw clicks that are the problem - but because of accuracy and the opportunity costs of the attention. Alphastar won't have accuracy problems, and the opportunity costs of the attention are VERY different from the human costs.
I think Alphastar is a great experiment, and I am glad they are cutting off some of the brute force advantages a program has vs a human opponent, but that's not the same as saying it is doing it "the same way humans do".
For some people, either there will be endless litigation of every tiny physical difference between the computer and human player that makes it "unfair," or the premise will just be abandoned and we'll hear things like "well, yeah, computers are great at RTS games, but RTS skill isn't really a sign of true intelligence."
I think you are correct. There will always be those people.
I hope I don't fall among them - I raise the distinction about being "like" a human not because I think it's makes a good/bad qualitative difference, but because I'm hoping to avoid such comparisons. To me, far from proving it is "good" or "bad" at the game, AlphaStar is interesting for the behaviors it uncovers that are useful to humans. (example: AlphaStar overloaded workers - a strategy that had been discarded by almost all pro years before, and is now enjoying a reevaluation as a result). Paying attention to how the I/O is different matters to such elements, even if it is pointless in the comparison of "true intelligence".
FWIW, when it comes to AI I have a more Minsky-view of things (in my limited understanding) and think that we're comparing apples and oranges without any awareness that its all fruit - we only KNOW apples. I think AlphaStar already has a better understanding of RTS than I do (low bar), even if we ignored the differences. That, however, isn't terribly exciting. AlphaStar showing us new tricks we can use - THAT'S interesting. (And now I want a segmented apple, dangit)
For A, this comes up a lot in discussions about video game design and balance. Do we really want to be testing how good players are at detecting cloaked units or exactly counting groups of units in battles? I tend to think those aren't super interesting strategically, tactically, or mechanically.
For B, there's a reason that human competitive weightlifting or sprinting is still interesting, even though everyone knows that machines could trivially win those competitions. Of course, those aren't really tasks that are considered primarily measures of intelligence (although, see Moravec's paradox). It's damn cool to see the limits of human ability stretched.
Of course, questions about expensive gear, performance-enhancing drugs, and even prosthetics and cybernetics can already challenge our philosophy of what makes human competition "fair." We inherently want to test the inequalities of humans, both we're only interested in certain inequalities. Generally, we're interested in who can lift the most weight, not in who could take the most growth hormones without dying.
Simply thinking out loud, take a human player skilled in chess and Deepmind's chess player, and ask them both if we could teach a badger to drive a school bus, and what changes we'd need to make to the bus and the badger and the system around it; facetious as I'm being, this is the kind of situation in which I don't see AI making any qualitative inroads, which humans remain good at - massively out-of-context problems.
As an aside, on the sbject of "true intelligence" or "general intelligence"; I'm not convinced there's any such thing, and if there is, I'm even less convinced that humans have it.
consciousness (and the higher-level awareness that feeds general intelligence) costs calories. Thus, we are evolved to MINIMIZE THE NEED. We like to think of ourselves as self-aware, and we can be...but most of the time we're in a lower state. When this near-lizard-brain state runs into something it doesn't have a preprogrammed response to, consciousness is engaged. We figure out how to respond to such situations...and we shut down again.
Once we learn to break this cycle, a great many wonders and horrors will be unleashed as we wrestle with what to do with a larger quantity of time being AWARE.
I myself am terrified that I don't get bored the way I used to as a kid. I assume this is because my lower-awareness self has plenty of pre-programmed tasks to manage and my higher awareness just doesn't get activated nearly as much.
So two immediate and obvious flaws were:
1. It could still burst, with overwhelming APM during combat
2. Pro APM is not the same as bot APM, as humans have far more limited precision, and a lot of human APM is wasted on redundant behavior as a result (eg spam clicking to make sure the order goes through)
If its still the same, then the bot probably still has lots of excess APM to burn through without impacting micro performance.
> A. AlphaStar has built-in restrictions, which cap its effective actions per minute and per second. These caps, including the agents’ peak APM, are more restrictive than DeepMind’s demonstration matches back in January, and have been applied in consultation with pro players.
The new version is limited to one screen of information at a time as well as a 350ms reaction time. My question is if it can "click" through all possible screens in 1 ms, then click through each screen 350ms later with a reaction.
The other question is what constitutes meaningful APM. If only 20% of human APM meaningful do to redundant clicks and hotkey cycling, what is the appropriate AI APM.
If it could make the same types of mistakes and overlook things in the same way a human would, that would be very interesting for game creators. Most game creators would tell you: usually the hard thing is not making a good AI, but making an AI that is fun, but still loses. We've all seen that AI can usually just win any game, if you give it enough juice.
(At least part of) The training has MMR as a label from unsupervised learning from human games so as part of the configuration they can set the agent to play like someone with a set MMR.
Lex Fridman's interview with Oriol Vinyalis on Lex's AI podcast covers this in more depth.
No human can maximize target firing before each shot like AlphaStar did.
Not a lot of people even talked about it
> These caps, including the agents’ peak APM, are more restrictive than DeepMind’s demonstration matches back in January, and have been applied in consultation with pro players.
Human player holding down a key that produces thousands APM certainly skewed this misleading chart.
Does this implicitly mean it's okay if I make bots and let them play on battle net?
If you would like the chance to help DeepMind with its research by matching against AlphaStar, you can opt in by clicking the “opt-in” button on the in-game popup window.
DeepMind called its Go-dominating AI “AlphaGo,” and the StarCraft-playing bot got a similar moniker. It’s called AlphaStar, and it has more than 200 years of practice under its belt. Back at Blizzcon in November, DeepMind said its machine learning platform had managed to beat the “Insane” difficulty in-game AI about half the time. Well, it’s gotten much better since then.
AlphaStar is a convolutional neural network. The team started with replays of pro matches, giving AlphaStar a starting point to begin playing the game. Through intensive training with competing models, DeepMind was able to teach AlphaStar how to play the game as well as the best human players. Over time, it whittled the AI down to the five best “agents,” and that’s what it deployed against some of the most skilled StarCraft II players in the world.
The matches actually took place in December, so today’s internet broadcast mostly featured replays of those matches. First, AlphaStar battled a player known as TLO, who primarily plays Zerg in StarCraft. However, he had to play Protoss as that’s the only race AlphaStar trains with right now. This competition wasn’t even close — despite TLO’s best efforts, AlphaStar beat him five games to zero. Next, a different AlphaStar agent went up against a seasoned Protoss player called MaNa. Some of these matches were closer, but AlphaStar still won five games to zero. MaNa also competed against a new AlphaStar agent live on the stream, and this time MaNa finally pulled out a win.
AlphaStar demonstrated impressive micromanagement of units throughout the matches. It was quick to move damaged units back, cycling stronger ones into the front line of battles. AlphaStar also controlled the pace of battle by bringing units forward and dropping back at just the right times to inflict damage while taking less fire itself. This isn’t just a function of brute force actions per minute (APM) — AlphaStar has substantially lower APM compared with the human players, but it’s making smarter choices.
The AI also had some interesting strategic quirks. It often rushed units up ramps, which is dangerous in StarCraft II as you can’t see what’s up there until you move in. Still, it somehow worked. AlphaStar also eschewed the tried-and-true tactic of blocking off the base ramp with a wall of buildings. That’s StarCraft 101, but the AI didn’t bother with it and still managed to defend its bases.
It wasn’t until the final live match that the human challenger spotted a flaw in one of the agents. That version of AlphaStar committed to moving almost its entire army as one with the intention of swarming MaNa’s base. However, MaNa was able to repeatedly warp in a few units at the back of AlphaStar’s base. Each time, AlphaStar would turn its army around to deal with the threat. That gave MaNa enough time to build up a more powerful force and take the fight to the AI.
At the end of the day, AlphaStar won 10 matches against pro players and lost just one. If AlphaStar learned from that last match, it might be unbeatable next time.
Sure, if you can get Blizzard to specifically allow yours in competitive matches for research purposes.
Lots of smart people...Kids in high school competing could motivate them to learn more math.
So presumably it will get the same info you or I could get from the minimap (presence of units in a particular area but no details as to what) + detailed information about anything in its camera-like view. This will have some effect on how the bot manages its APM budget, since it now needs to consider camera management.
But honestly, from the perspective of the Alpha research, getting information from the pixels probably isn't an interesting problem. They can pretty safely assume that they could build a Computer Vision setup that could get the same info that the API gives, given enough time + GPU horsepower.
Any possible inaccuracy/delay that might result from using Computer Vision instead of the API can be simulated. You could add a response delay (already in AlphaStar), or a random chance that the API delivers incorrect information from time to time (which I don't think they do).
Personally I am not a fan of training our future slave masters how to efficiently kill human targets.
A more interesting challenge in my mind is a game that focusses genuinely on longer term planning, where actions taken at a certain time lead to results 10 to 15 minutes later in game that is simulated in a continuous fashion (that is, definitely not turn based and with no way to discretize it perfectly into turns). The loss of a turn structure makes applying min-max-search based strategies hard to apply - each side can make any number of moves in sequence. Add non-discrete moves (e.g. turn unit with a certain turn rate for an arbitrary amount of time, then move forward for another arbitrary amount of time) and the search space becomes vast and unstructured, potentially even unbounded. Solutions to that should be very interesting.
This is simply not true at a high level of play. Timing attacks, base expansions, and tech transitions involve both medium and long-term planning.
The momentary triggers you mention are the distilled knowledge of long term planning and experience. A player executing a baneling rush is a long term plan. Then that player scouting the opposing terran player building a bunker and two barracks to defend can make him to decide to double expand instead of going forward with his baneling attack. This may seem re-active but in truth those decisions depend on long term planning. Pro players can make this long term planning calculations quite fast, the decisions may even appear re-active, however they are based on the long term calculation of the most probable path to victory.
??? Have you ever competitively played an RTS?