That said, I'm not sure I agree that it was winning mainly due to better decision making. For context, I've been ranked in the top 0.1% of players and beaten pros in Starcraft 2, and also work as a machine learning engineer.
The stalker micro in particular looked to be above what's physically possible, especially in the game against Mana where they were fighting in many places at once on the map. Human players have attempted the mass stalker strategy against immortals before, but haven't been able to make it work. The decisions in these fights aren't "interesting"--human players know what they're supposed to do, but can't physically make the actions to do it.
While they have similar APM to SC2 pros, it's probably far more efficient and accurate so I don't think that alone is enough. For example, human players have difficulty macroing while they attack because it takes valuable time to switch context, but the AI didn't appear to suffer from that and was extremely aggressive in many games.
I think a far more interesting limitation would be to cap APM at 150 or so, or to artificially limit action precision with some sort of virtual mouse that reduced accuracy as APM increased.
IIRC OpenAI limits the reaction time to ~200ms when playing DoTA2. AI employing better strategies than humans will always be more interesting than AI that can out click humans.
It struggles with camera placement like real players :) And uses popular divert-attention tactics, which shows it understand that part of the game - for example when it sends oracles to mineral line at the same time as it attacks in front. Previous versions didn't do that, because they were taught playing vs cheating AI - so no point diverting attention of something that has instant access to any unit on the map :)
It also struggles to defend against adept harras beacuse it has "tunnel vision" - controls its oracle instead of defending probes at home. Mana actually managed his attention budget a lot better (this is a crucial pro-player skill in starcraft - harras is effective because it trades little of your attention for a lot of attention of the enemy, it's a skill that becomes irrelevant when opponent doesn't really have "attention" and can perceive and interact with all units on the map at once like previous version of alphastar).
This one is much more human, and much lower level. In my opinion it lost unfair advantage, so the mistakes in its errormaking are revealed. Previously it never was behind and never had to react to human player strategy - it rarely even scouted because what's the point - it wanted to build mass stalkers anyway.
Or, what if we slow down the game, so that the human can actually pause the game each second and consider what to do next. That's basically what the computer is allowed to do
Upgrade building 3 comes available when you have enough resources.
A separate tab with insufficient resources gives you an overview with what you need to finish a,b,c.
A red alert appears when an enemy is spotted. You can click nearby units attack or a FSM with the attack strategy.
An finished building automatically will be placed near the town center.
Not working farmers can search for resources.
A wall is suggested by your current buildings, you can set an margin of eg. 20 meters.
The question is, how much programming will the custom UI need ( and how deep) to make it a lot more efficient
Macro-wise, it would be like an unwieldly minimap which already exists so people can get a sense of where the enemy is moving. With a giant screen, information is not focused on a small area, so you are limited to your FOV. Minimap which shows unit strength in terms of armor hp or shields as well as placement would be ideal information.
Micro-wise, it would be like sitting in front of a giant text display looking at a whole book. You still have to focus on a small section to read it.
While this would make it more fair, it would just make the micro game more similar to chess or go. I don't think humans would necessarily win in the end.
Starcraft is like chess in some sense. The largest fundamental difference is that it isn't a perfect information game.
But ofc, there’s no tbs or grand strategy currently out there with a real tournament scene, so you can’t really count on the devs implementing an AI-API, or even properly balanced / bug-free (far more user-testing goes into sc2/dota2 than say civ, simply by virtue of its playerbase).
That's the primary benefit imo. The bigger action space is largely composed of non-strategic elements, at least in the sense of long-term strategies, eg micro and mini-skirmish tactics, that I don't think are as interesting. Ofc its clearly a conflict of interest, but my feeling was the most interesting aspect of Go/Chess is the AI making unintuitive discoveries that benefit the long-term. The human-collective machine is pretty good on its own at finding the shorter-term strategies; I don't think AI will make much significant impact in that space.
games as a medium to study upcoming real-world applications (eg cars), RTS makes sense; but as a medium to study AI beating humans, TBS is more appropriate (their ability to explore large search-spaces is far more interesting/potentially impactful). Studying both would be ideal ofc, but in a pick-one situation, TBS is better imo. But only RTS are even really viable atm, which is disappointing.
The former is an interesting AI challenge/achievement, the latter is a space in which computers are already known to outperform humans.
I would like to see a setup akin to that of Ender Wiggin, with one commander overseeing and recommending overall strategy, and, say, five others managing different areas or groups. That seems like the way to get the best human performance, and might be enough to beat the AIs—at least to nullify chunks of their advantage.
As an aside, a few pro gamers prefer to play on windowed mode for exactly this reason.
I'm not saying you're wrong, but 6 posts with no profanity is hardly "uproar" by blizzard forums standards.
They could train against the API, reinforcing the AI trying to predict the state from vision. But with limited APM it would be pretty difficult for the AI to keep track of everything. And, potentially, it would still not be the same as a human looking at it. I'm not sure whether human attention is a particularly bad example of efficient resource allocation. I'm very biased to think it is still the gold standard. But the fact that deepmind didn't focus on this implies they were not finding it interesting enough, and/or too difficult.
Anyhow, (visual) exploration is a step up from mere image recognition
"Brute force" in AI context is usually reserved for traversal of the entire search space. I think "superhuman micromanagement" is a better term. And before AlphaStar superhuman micro wasn't insurmountable obstacle for human players.
At that point the name of the game will be maximizing the advantage the body/infrastructure provides the AI, not minimizing it.
I would love to see these AIs get handicapped even more like a full second and really force them to out think humans.
A simpler model would be to limit the bot to, say, one action per 250ms, introduce a slight delay in his reaction time, require him to move the camera to gain detailed information and take further actions, and have camera movements count as actions.
The skepticism in this thread is absolutely justified but I think it's important to note the lengths to which DeepMind has gone to address and assuage the fears of superhuman mechanical skills being employed in these games.
This is compounded by the fact that almost all of AlphaStar’s actions are “useful” whereas a significant amount of the human actions are spammy.
You will typically see a human select a group of units, and fast-click a location in the general direction they want the units to move (to get them started moving that way), and then keep clicking to continuously update the destination to a more precise location. Every click counts as an action. An AI can be perfectly precise and “clicks” the right place the first time.
Fair point about humans needing minor adjustments though. Another comment also mentioned a bug in the APM measurement: https://news.ycombinator.com/item?id=18994350
Meanwhile humans are literally spamming keys to keep their physical fingers loose and ready - they're not performing anything close to 400 useful APM on a regular basis (or in TLO's case - 1500 ... He kept walking his units straight into death while spamming keys).
Exactly, a human doing 500 APM during intense moments is going to be way different than an AI bursting 1000 APM with pixel-precision during the most crucial moment in a game.
TLO spent a ton of time at >1000 APM and walked his army directly into enemy shots all the time. MaNa had much better control at ~400 APM. So APM is really irrelevant to control - for humans.
I suspect the AI, on the other hand, makes each action precise & count for something.
This graph, which I think was supposed to show that the AI was being "human", IMO is pretty damning. We saw the APM spike to >1000 during a critical moment and we saw the APM at <30 during lulls, so we know it uses its APM at important moments, presumably with important pixel-precise actions.
As a hopefully illustrative comparison, you could give any top player a day of play time per move against the top Chess AI being given a minute of play time per move and the AI will still win. That's how much better the AIs are than humans now. There's no reason in principle this won't be possible with StarCraft AI too.
And then when you drop the APM limit, suddenly all the learned optimal ai strategies start falling apart, and the whole thing has to be relearned.
More annoyingly, there’s not much for human players to learn from innovative ai strategies that are based on inhuman accuracy of play (because we couldn’t possibly execute it).
StarCraft is more random than chess, so I do think it's possible humans will always be able to take occasional games off of fairly constrained AIs just based off blind luck in picking counter builds, it will be interesting to see what % that is.
Yes, I have one from it and wasn't even playing that high (I averaged less than 100 apm). I understand that it's a common problem.
Most music should be playable without excessive risk of serious injury to arms / wrists / hands, but from what I understand very high notes on e.g. the violin are hard to play without using an over-flexed wrist, which is definitely a problem if playing music requiring such a position for long stretches of time, or many rapid switches between high and low notes.
Some of the string players with most risk are novices who have not been taught proper technique.
For professional PC game players, the design of the standard computer keyboard and furniture is absolutely terrible from an RSI perspective (worse than any common musical instrument, and without any of the design requirements of acoustic instruments as an excuse), and it is shocking to me that there has not been more effort to get more ergonomic equipment into players’ hands. The way game players typically use a computer keyboard is generally more dangerous than the way typists or e.g. programmers do. As someone who spent a few years thinking about computer keyboard design, I can think of at least a dozen straight-forward and fairly obvious changes that could be made to a standard computer keyboard to make it more efficient and less risky for game players. There is a lot of low-hanging fruit here.
Whether or not the equipment is changed, the most important single thing when using a computer keyboard (or any hand tool for that matter) is to avoid more than slight wrist flexion or extension, especially while doing work with the fingers. Excessive pronation and ulnar deviation of the wrist are also quite bad. Watching pro players, many of them have their wrists in an extremely awkward position while doing fast repetitive finger motions for hours per day without breaks, which is a guaranteed recipe for RSI.
"Liquid regretfully announces that Dario “TLO” Wünsch will be unable to play for the next few months due to the Carpal Tunnel Syndrome he experiences in both hands. He will however continue to be involved with E-Sports even as he takes a break from gaming to give his wrists time to heal. Sadly, this means that he will not be attending Dreamhack Summer or the Homestory Cup III as a player."
I like the idea of having action noise that's linearly related to APM
It isn't surprising that its fast, the surprising part is that it can make human-like decisions. The only way to compare whether its thinking is human-like is to restrain it from "brute forcing" the contest through speed.
The model has likely learned that the faster it does things the better the outcome. What it needs to be measured on is strategy.
In that context, you can't really measure strategy without accounting for timing/speed because a lot of tactics and strategies only become viable once the player has the required speed to actually realize them aka "micro".
It’s not good at strategizing with all the options available to it given it’s micro ability, it has “one” strategy that leveraged the micro as much as it could, and when given a strategic challenge by mana, it didn’t know what to do.
They want to make an AI that can teach new ideas to humans. New strategies that human bodies are physically capable of executing, but no human was "smart enough" to think of yet. An example is when the AI built a high number of probes at the start. That's "smart".
The only way to train an AI to be able to come up with new ideas, is to force it to be "slow". Otherwise, it will just always do the easiest way to win, which is out-micro. There is nothing interesting about a game like that. That only shows the AI is fast, but it won't be clear that it's "smart"
Making an AI that wins by outsmarting humans, on the other hand, is what we are all interested in.
They don't, because AI doesn't use physical objects to move stuff in the game. AI just "thinks" that this stalker should blink and it blinks. Human player has to deal with inertia of his hand and of mouse.
If you want fair competition of micro - make a robot that watches screen through it's camera, moves mouse and presses keys to play starcraft.
Then the bandwith of the interface is the same for both players, and we can compare their micro.
If we want to measure strategy, I agree with you, and out of curiosity we might do it. But the goal is winning, so is strategy important as long as it wins? The AI can take every shortcut it finds IMHO. People do take shortcuts.
Cars and planes bring us across the world exactly because they don't walk like people and don't fly like birds. Wheels, fixed wings and turbofans are shortcuts and we're happy with them. We can build walking and wing flapping robots but they have different goals than what we need in our daily transportation activities.
If you want to make it fair - place an AI-steered robot in front of the screen, and make it record the screen with camera, and actually move the mouse and press the keys.
Then I can agree it's fair :)
But then of course AI would be incredibly bad.
Right now the advantage doesn't come from faster thinking, but from much higher bandwith and precision that AI has when controlling the game. It's anything but fair.
With chess it's not a problem, because interface overhead is negligible.
It's surely interesting technology with positive impacts in a lot of areas but is it that the important part of the experiment? Humans need keyboards and mice to interface with computers, computers don't (lucky them.)
Sorry to insist on that analogy, but it looks to me as if my car should be able to fit my shoes and walk before I admit that it goes to another city quicker than me walking.
When you're trying to individually blink 30 stalkers at the perfect time they have almost 1 hp - latency is everything.
Camera has latency. Depending on various factors it takes even milliseconds of exposure for camera to gather enough light that it registers as a clear image frame. Human eye works on a different basis, but also isn't instant. You cannot cut that in software, human player cannot train to lower this. But AI doesn't need to do it - it has image provided as a memory buffer.
Image recognition has latency (both in the brain and in computer). Even as simple stuff as recognizing where the computer screen is as opposed to the background. It takes time. AI doesn't need to do it.
Muscles (engines in robot hands) have latency.
Mouses and hands have inertia and can't be moved instantly - have to be accelerated and stopped and even if you have optimal algorithm to be 100% accurate - it takes time.
It's not only hard to implement, it's also physically IMPOSSIBLE to do without introducing significant delays.
AI that is controlling the ui directly doesn't have to deal with most of these tasks, so it has a huge advantage in a game like starcraft. It's not that AI is so much better, it's that AI is high-frequency trading and human player is sending requests to buy/sell by telefax. By the time your request is processed the other guy had opportunity to do 10 different things.
If you want to focus on the part of the job that is doable now - sure, go ahead. But then don't abuse the unfair advantages you have and announce you "won". It's very low threshold to win in starcraft when your opponent has effectively 100 times the lag you do.
I'm sure someday we will have AI that can beat human player in starcraft without abusing this advantage, And I'm pretty sure the fastest way to this isn't to put a real robot in front of a screen, but it's to limit the intraface bandwidth of the AI to be on the similar level as that of human players.
> Sorry to insist on that analogy, but it looks to me as if my car should be able to fit my shoes and walk before I admit that it goes to another city quicker than me walking.
Let's remove the roads that we made specifically for cars and speak about this again :) Will your car move you through an untamed wilderness quicker than your legs? Possibly. Or not at all.
If I walk into a bullet train, slowly walk inside it, and walk out of it at the end of the route I will be even faster than the fastest car. Is it fair to say I'm faster than a car? After all it's not my fault the car doesn't fit inside that bullet train :)
We need to compare apples to apples, and comparing AI that doesn't need to deal with half the sources of latency with a human player that does, in a game where latency is very important - just isn't fair.
You could make an AI which tries to hack the human computer to force a leave. That would also constitute a "win". Or one which hacks its own computer and displays "You win" immediately. Or one which tries to kill the human player, if we want to be really dramatic about it.
hooking it up to a camera looking at a screen and a robot arm with a mouse would be more fair though.
edit: ok they did have a camera version, but i still want a robot arm.
The ceiling here is going to be incredibly high, much higher than the level of play that people are capable of, even when restricted to a single window.
Part of the difficulty here is describing what a 'fair' match might be. Specifically, I think fairness has to do with a goal many people have for AI: to improve human play. The strategies in Chess or Go that were employed could conceivably be used by human players. There aren't any hard restrictions preventing humans from learning from that play, even if the AI is entirely superior.
It would follow that a 'fair' SCII match would employ strategies that humans could implement. Making extra workers, for instance, might be a real lesson from AlphaStar play. The insane stalker micro, however, could never be done by a human.
From this perspective, I think the important takeaways were:
* The AI leaned heavily on super-human stalker micro.
* The AI had some strategic blind-spots, namely the immortal harass.
* The APM comparison isn't terribly meaningful; a lot of human APM is spammy/twitchy button presses that doesn't do all that much, whereas the AI can presumably make each action count. There were also AlphaStar APM spikes that likely go along with the stalker-micro issue.
When AlphaGo first one, people said it wasn't fair because it was running on a whole cluster of computers. Well, within not much time at all, it was good enough to run on a single computer and still beat top humans. We are dealing with exponential progress here. The writing is on the wall.
It absolutely does matter whether the AI can use obviously super-human techniques, because then it's not nearly as interesting for human observers. I'd much rather watch an AI that was a strategic genius that won despite being hamstrung in terms of micro/techniques.
> There's no reason whatsoever to suppose that humans are fundamentally better at this game than an AI can be.
Who's claiming this?
Different problems have different difficulties. Solving simple problems quickly doesn't mean we'd also be able to just as easily solve the hard problems. Often the comparably simpler problems have the best reward/effort ratio and thus make quick progress, which doesn't need to be the case for hard problems.
If you had bet against AIs reaching parity with top human players in any previous game, whether it be Checkers, Chess, Go, etc., you'd have lost. I see no reason why StarCraft II should be any different.
We can reconvene in the comments here a year from now and see where AlphaStar is then.
One of the hardest parts about these kinds of human vs ai expositions is making sure the AI has explored the full possibility space, so that can handle all situations. The techniques at play lack the ability to perceive a completely new situation and formulate a good response. (Though anyone who's lost to cheese in games they later learned easy counters for know that humans, while better than state of the art AI, aren't perfect here either.)
Now, the questions are how many more such glitches will show up and can they eliminate them with better algorithms?
It’s likely safer to say the AI was confused in general at that point, possibly related to the camera change, but we didn’t really get to see the quality of stalker micro that game
In software, changes in assumptions can break what depended on them. There could be many assumptions in its neural net centered on full visibility. They should probably retrain all or just some from scratch with the camera change in from the beginning to see what happens. Then, it will be firmly encoded into the strategies over time.
"The AlphaStar league was run for 14 days, using 16 TPUs for each agent. During training, each agent experienced up to 200 years of real-time StarCraft play. "
MaNa probably played less than 2-3 years of Starcraft in his whole life (by that I mean 24hr x 365d x 3), and was learning with a much less focused/rigorous methodology.
Humans don't have to learn to process, recognize, and classify objects in visual sense-data, for example. We can do that from the moment we're born, because we already have hundreds of precisely-tuned "layers" laying around in our brains for doing just that. We just need to transfer-learn the relevant classes.
For example, humans, even from infancy, prefer games where it is possible to punish cheating (i.e. take revenge upon cheaters) to games where it is not. This isn't just "we're animals that have evolved to enact tit-for-tat strategies [by e.g. injustice triggering rage] because they lead to cooperation which leads to egalitarian utility"; this is actual analysis—instantaneous, intuitive analysis—of a system of rules, to notice, in advance of ever being slighted, whether you'll be likely to end up in an "unjust" social situation if you agree to the given ruleset. There is an "accelerated co-processor" of high-level abstract game-theoretic information—and layers to extract that information from sense-data—that ship as part-and-parcel of the human brain model. We never need to learn how to judge unfairness, any more than we need to learn how to see.
If a 19 year old is good at Starcraft, he's good at Starcraft because he spent two or three years playing a shit load of Starcraft and we are much more efficient at learning higher level strategies than AI are. These AI agents nead to try damn near every possibility to adjust their weightings for various actions. Humans understand pretty much the first time when something goes wrong, oh better not do that OR similar things again.
It's incredibly impressive that a given human can become GM level at Starcraft within a few years and to take an AI to that level takes 200 years of training, as well as an inhuman reaction time, perfect micro/clicking, etc. It shows how amazing our learning skills are.
Totally agree with how impressive humans are, though. In fact, one of the most amazing things to me about robotics is finding out how close to global optimal some humans can actually get.
GP's claim, "99%-baked deep net that was wired up during foetal development from our DNA" is also unfounded, if not completely overblown. I am far from a student of biology, much less an expert, but intelligence is still seen as an emergent property. The real kicker might be that organizing thoughts might be a "game" of it self, that is learned in development and constantly exercised. Talk about self-play.
I recently read a similar question about "inherent mathematical language", ie. capability, and the given opinion was that there is no consensus, except perhaps for basic addition, which I guess concerns vision, ie. seeing a set of things and knowing the count is +++++. That works only up to around +++++++ items at best, according to findings.
Calling it premature is ironic, if we reach nominal maturity only after 10 or more years as far as fertility is concerned--the equivalent in AI would be the procreation of a neural net, perhaps after exploiting a bug in the game, breaking out to rewrite a better version of itself, or colluding with itself in self play. Yes, this is going off-topic.
The consensus in the evolutionary-anthropology community is that our hips (pelvic bones) have to be the size they are, in proportion to the rest of us, to make us able to walk upright. "Building bigger" doesn't really work, for the same reason that you can't make a giant robot—if you scale humans up, the pelvis would need to be made out of something stronger than bone to support the additional load.
The same is not as true, though, if you just make the person wider—because then you spread the same load over "more pelvis." (This is just a personal unfounded hunch of mine, but I think some human subgroups—e.g. midwestern Americans—who are at the genetic limits of baby head size, and who avoid C-sections, are currently selecting toward bigger-boned-ness.)
> I would almost say that longer pre-natal development was suboptimal, because we'd either become bored, or supersmart, but anyhow superegoistic for lack of nurture.
Keep in mind that we wouldn't be conscious for any of it. The development stage that "wakes you up" to the outside world would just occur later on, as occurs in animals with longer gestation periods (e.g. elephants, with a gestation period of 18-22 months.) This would give things like your ocular layers longer to finish developing, without really having an impact on the parts of your brain that learn stuff or think stuff.
Being born “prematurely” might allow for more flexible brain wiring. Adapting better to an environment quite distinct from ancient ones we had evolved in is possibly one of our key cognitive advantages compared to other animals.
Do you have a citation for this? It doesn’t jibe with my understanding of development. For example,
animals born paralyzed are blind: https://io9.gizmodo.com/the-seriously-creepy-two-kitten-expe...
All the stuff we've learned about games and so on have come from our current lifetime. I don't have caveman memory for how to fight a tiger.
For one example: any smartphone's face-recognition feature. Each such feature is a DNN which took millions of hours of face data to train... but the resultant model fits on an ASIC.
Our DNA doesn't directly encode such a model, but it encodes a particular morphogenic chemical gradient, and set of proteins, that go together to make specialized neural "organs" (like your substantia nigra, or your basal ganglia, or your superchiasmatic nucleus, etc.) which manage to serve the same function to your brain that access to a pre-trained "black box" DNN model would serve an untrained NN in achieving transfer learning.
The "training" of our deep net happens during our lifetime. We are not born with a trained deep net so your analogy that somehow we are born with a highly capable deep-net encoded into 1.6GB of DNA makes no sense.
Can you imagine how capable a human being would be if it was born into a world with no other humans or learning sources? Imagine a new born baby born into a world with some accessible food/water close by so it wouldn't die from lack of nutrition or wild animals, but crucially without any other humans. It would be utterly fucking useless, no language/reading means no way of assimiliating new knowledge. That baby would end up being a totally incapable human, regardless of the DNA or structure of the brain.
As far as we currently understand, if infants aren't exposed to language and communication at a very young age, they are either incapable or severely stunted in terms of communication for the rest of their life.
My point is, that we are very much dependent on the learning that we get from the point of birth ONWARDS. We get the amazing capacity to learn from the structure of our brain and body, but we'd be absolutely incapable idiots without other people to teach us, our books, language etc. We understand "games" and game theory from playing games with other kids, we're not born with "game theory" encoded into our DNA as one other commenter seemed to think, the same for language learning, and everything else.
Anyway, the point of this whole debate was that it's incredibly impressive that humans can learn to play a game as complex as SC2 in a tiny fraction of the time it takes a cluster of GPUs using a huge amount of energy and resources. Not forgetting that we also have to use a physical body to control our actions in the game, which adds a whole other level of complexity since we have to understand how to manipulate a mouse/keyboard etc, whereas the AI is essentially acting directly with the game, like a human with a neural link. The other kicker, is that if you just changed one aspect, like picking a new map neither player had seen, the AI would be sent hurtling back to square one whereas the human would only be partially affected. These series of demos only make me more impressed that given the huge resources given to Google, they can just about beat a human and even then after 200 years of training time and various other artificial advantages.
You are insisting that because humans do not have instincts at a certain level of abstraction (playing video games) that no part of these instinctive brain functions play a role in the development of skill at Starcraft. This is wrong. Abstract reasoning is not simply learned, but it is HONED by experience and neural development. An AI has to do an enormous amount of work in order to replicate functions that humans can already do. This is the basic visual problem in AI that stumped researchers in the 60s who thought that tasks like visual recognition, spatial rotation, etc would be trivial because they are trivial to evolved organisms.
You're relying on some kind of mental model where brains are just masses of neurons that form all of their connections and complexity after birth. This is ultimately a political idea, and it's wrong. No neuroscientist believes this. Brains have pre-defined areas (with fuzzy borders) and many behaviors do come baked into the template. Complex behaviors like language do not, perhaps, although even there, the underlying functionality that permits language is an evolved trait (which is why other animals can't learn language). Research the FOXP2 gene, as just an obvious example.
Edit: Your post contains "structures of the brain". What exactly do you think the structures of the brain are, if not evolved modular solutions to complex problems? Your visual center is somewhat trained after birth, but it already exists. The same goes for speech, motor control, and all of the other unconscious or semi-conscious processes that all humans (and other animals as appropriate) share.
This gives them reserves when attacked and some workers killed. They can also ramp up mining at a new base quickly by moving the extra workers there.
Apparently the benefits outweigh the costs for these workers for AlphaStar. It will be interesting to see if some pros decide to adopt the technique and if it improves human performance as well.
Disclaimer: I do not have much Starcraft experience.
Let's say you make 4 extra at a cost of 200 minerals and then lose 4 workers to harassment. You are out 200 minerals in both cases, but the prebuilt workers in the prebuilt case will mine an extra... 100 minerals? (40 + 30 + 20 + 10).
This doesn't take chronoboost into account though. I don't know, the gain is marginal, and the opportunity cost is having a smaller army (2 zealots for example)
Please correct my numbers if I've made a mistake, I forget build times and havent played since hots
The extra workers aspect was the most interesting decision-based adjustment AlphaStar made on conventional pro level wisdom of "standard" play. It has a couple of factors in play, that I trust the AI factored in and more and tested over several games for its long-term benefit to winning a game:
- every 8 probes you build requires a pylon as well. total cost of 500 minerals
- workers are safer in the main than in an unoccupied natural (long distance mining) to harassment and pressure
- when your expansion completes, having 4 workers vs 8 workers vs 16 workers potentially has huge impact to the immediate spike in income
- what you mention -- the prebuilt workers will dampen the impact of most worker harassment to purely the resource cost of the lost workers.
My guess was that well executed harassment by an opponent in practice games put AlphaStar in very limited situations with a crippled economy that it couldn't fight its way out of, so this was a catch-all harassment "counter" -- it's ok if you kill a few probes, at least it won't throw off my economy completely and I can still continue my overall gameplan.
After that I think the next most important aspect was planning ahead for a bigger income spike when their expansion was done without waiting to build out another 16 workers after the nexus was ready.
I'd love to watch the results of constraining the AI so instead of seeing the whole map at once it has to pan around the same way a human would to get updated information on each battle. Counting those "info-gathering" window pans against the actions tally might yield slightly fairer APM metrics.
(EDIT: Turns out they built a new agent for game 11 to do just that)
One of my biggest beefs with strategy games of this genre occurred around the time sprites went 3D and the player viewports got smaller (presumably to showcase all the cosmetic detail, and since it became harder to distinguish between visuals when zoomed out farther). I always feel too constrained on the modern games - like I can't see enough of the map at once. In my opinion that "full size viewport" gives a multi-tasking edge to the engine that the player doesn't share (beyond the human cognitive overhead from context switching you already pointed out).
On the other hand I find it fascinating our AI's have become strong enough at our games that we're having to handicap them to avoid players crying foul that they're not fair.
Or better yet, imagine zerg where you can burrow every unit.
Regardless, the article describes cheesing as the common tactic in early iterations, with economic-play being learned later — one of the described cheeses is dt rushes, which the AI apparently learned to deal with, so it should have some understanding of invisible units (alternatively it learned to ignore the dts and base trade or something).
I don’t think the shimmer is useful enough to be a significant loss for these prospective AI’s quests for world (sc2) domination
An AI trained from human strategy might end up more limited than one that could learn from scratch. It could be stuck in a local maximum of play and be unable to escape.
An AI technique that requires a large dataset of pro play to learn will be much more limited in terms of applying it to other games.
In StarCraft 2, the game IS the interface. That is to say, the developers have constructed the game in such a way as to be difficult to control; and human mastery of the interface is a large percentage of the game. Strategy in the game is important, of course -- but this is not chess, where human beings are not limited by the interface of the game. In StarCraft, you are intentionally given a limited interface to monitor and control a gigantic game while under incredibly tight time controls.
And I should also note that Blizzard is extremely reluctant to add features that make it easier to control the game. I have a friend who works on the StarCraft 2 team. We talked at length about this one feature that he designed and proposed for the team to make a specific aspect of the game friendlier towards players. It was turned down for exactly the reasoning above -- the game is the interface. By making the game easier to control, it disrupts the entire experience; an StarCraft 2 that is easier to control is no longer StarCraft 2.
Essentially try to quantify the advantage of increased view area.
Attention/APM is often called the "third resource" (after minerals and gas), spending it wisely when you have several areas at any given time that could use attention is part of the strategic and tactical decisionmaking. For example, usually in a battle you wanna be paying most attention to the fight rather than your base, but sometimes it's actually better to jump out back to your base to increase production or economy, and knowing which situation is which can be challenging.
Obviously, if you make the game mechanics too easy to control (letting the computer do more of the work), then this part of the game becomes less interesting, because you don't have to weigh trade-offs as much anymore.
I would say yes, because StarCraft was very clearly balanced for human players. We already saw some indication that when played with super-human micro, mass blink stalkers is a stronger strategy than when humans are in control. Without the active intervention of game balancing, RTS metas tend to devolve into "mass one or two units" which was what happenes to every Command & Conquer game (and why SC is a respected eSport while C&C is not).
I suspect this will happen when you have agents playing parameters that don't match what the game was balanced for. The strategic landscape will shrivel up and the game cease to captivate us.
Still, he didn't do that either.
Typically, you don't see more than 1-2 photo cannons, because you don't usually want to "over-invest" and lose what advantage you gain.
Printf is part of a fairly small group of cannon rushers that don't simply see it as just another cheese, because what generally defines a cheese strat is that it can be easily countered if you know it's coming; not so with their cannon rushes.
Now, with that said, Printf (or any other "I always cannon rush" player aren't winning tournaments), but that's partly because not many players decide that they want to stake their development on any one strat like that, and if they do, it'll likely be one that's deemed more legitimate by the community.
It was also extremely active with the stalkers, deciding to split them in three and not let Mana cross the map with his immortals.
What's that hireability like?
Damn I really need to watch these games :)
Wasn't the APM closer to half that of the pros?
During the fights, the critical moments in when MaNa would top out at ~600 humanly inaccurate APM (this is 10 inputs per second), the AI would jump up to over 1000 - we don't know exactly what it was doing, but it was presumably pixel-precise. Meanwhile the physical inertia of the mouse is a challenge for humans at that speed - imagine trying to click five totally different places with perfect precision in a single second.
By comparison selecting a single stalker, and having it jump to a new location is much more effort, but counts as fewer actions.
Additionally, and subsequent to the matches, we developed a second version of AlphaStar. Like human players, this version of AlphaStar chooses when and where to move the camera, its perception is restricted to on-screen information, and action locations are restricted to its viewable region.
I was really curious whether they would attempt moving the camera like a human. Sounds like it's still a work in progress, but very exciting! Even this isn't enough to make it fully like a human player, as I believe it is still getting numerical values for unit properties rather than having to infer them from the pixels on the screen. But it seems possible to fix that, likely at the cost of drastically increasing the training time.
The benefit of using pixels, of course, would be that the agent would become fully general. It would probably immediately work on Command & Conquer, for instance, while the current version would require deep integration with the game engine first. But I think the training time would be impractically long.
I guess it makes sense that the AI would favor such a micro-heavy unit. I imagine it would be a nightmare to deal with perfect blinking.
- cost-for-cost, they are more efficient in a faceoff (resources)
- immortals are space-efficient dps (damage per second) in a battle. In a given battle, an army of 4 immortals is far more likely to all be in range of an enemy and doing damage than an army of 8 stalkers bumping against each other trying to get to the priority target
- immortal shots do not have projectiles, but are instant. No matter how perfect your stalker control, once an immortal targets a stalker, it is guaranteed to take 30+% of its hitpoints in damage.
The last point is very important. Once MaNa had 3+ immortals, even with perfect blink micro, a little bit of target fire and timing micro on MaNa's part allowed him to slaughter the stalker army one stalker per volley, while it takes them longer to clean up the immortals (especially with shield battery support).
Another thing glossed over in this discussion -- AlphaStar did more than classic blink micro. It did a very technical maneuver (the casters briefly allude to it) of triggering the barrier on one immortal with a single laser, then focusing all fire on an immortal whose barrier was already down from a previous iteration of this tactic, and then walking away until the barrier has worn off (while blink-microing weakened stalkers). Repeat. This is a detail of increasing the efficiency of trading stalkers with immortals that humans don't often even think about, let alone execute (because good blink control is often more impactful). That AlphaStar came up with this shows that it's not just about perfect execution of micro, but also perfect understanding of micro.
But as you say, showing that a bot can have perfect micro is not very interesting. Of course a computer can have better control of well defined tasks like moving a unit away just before it dies, especially doing so for many different units concurrently. What is interesting is the wider strategy and how the computer deals with imperfect information.
Other than that, I agree, and am also much more interested in what happens when you have a more level playing field (using camera movement rather than API, limiting reaction times and CPM, etc). I look forward to future matches where this happens.
If humans can, under ideal circumstances, see cloaked units... Maybe the only mechanic that shows up (like for bots or an API) is the inability to be targeted using an attack command (i.e. you can still be hit with splash damage from ground targeting)
don't get me wrong, it's a major accomplishment in AI regardless, but it's a significant advantage and it would be easier for me to appreciate the AI's skill if I didn't have to keep reminding myself that it can see the whole map at once. it's such an information advantage.
While it is an unfair advantage in competitive gaming, but in more realistic settings, there is no requirement that AI needs to have only 2 eyes. It can have as many as it could handle, while human can't scale the same way.
Chess and Go don't have any form of micro and AIs are nevertheless dominant there.
I'd say, give AI development another year and I wouldn't expect there to be any kind of game, in any genre, that humans can beat AIs at. Whether it's Chess, Go, other classical board games, Civilization, MOBAs, RTSes, FPSs, etc.
Yes, but chess and go have a tiny problem space compared to something like Starcraft. People want to see an AI win because it’s smart, not because it’s a computer capable of things impossible for humans. If the goal was perfect micro they could write computer programs to do that 10 years ago.
Even if you limit the AI to max human APM, it's still going to dominate in these micro-heavy battles because it's going to make every one of its actions count.
right, and we saw that with the incredible precision with stalker blink micro. There are many ways you could make it more comparable to humans. They have already tried that by even giving it an APM.
> You can't fault the AI for winning at the game because of the way the game itself works.
But it does make the victory feel hollow when it wins using a "skill" that is unrelated to AI (having crazy high APM with perfect precision because its a computer). Micro-bots have been around for decades, and they are really good. The whole point of this exercise is to build better AI, not prove that computers are faster then humans.
It would like if they wanted robots to try and beat humans at soccer, and the robots won because they shoot the ball out of a cannon at 1000 KPH. They win, but not really by having the skills that we are trying to develop.
Beating the world champion in Chess was, at one point, considered an impossible achievement for computers. Now it's considered so routine it doesn't even count as AI according to many. And in a few months when AlphaStar is beating top human players without having to use APM or viewport advantages, what will the next goalposts be?
There's nothing impressive in coding something that can execute something far faster than a human, or be so accurate and beat a human. There were Quake 3 bots that could wreck any human alive 10 years ago because they react in milliseconds and shoot you in the head perfectly. So what? It's obvious a computer can do that. It's like being surprised that a bullet beats a human in a fight, that's by design.
I would be impressed if a computer learned from scratch without knowing anything about the game beforehand, about the controls, or anything else, with ordinary human limitations. Using vision processors to look at a screen to see the inputs and controlling a physical mouse and keyboard. That would be impressive. But watching a computer do perfect blink micro at 1500apm is just underwhelming, since that isn't new tech, you could hand code that without deep nets.
Yeah, exactly. And when calculators first came out, people were very impressed by them. They upended entire industries and made new things possible that had simply never been possible before with manual calculation. When you're pooh-poohing the entire computational revolution you might want to take a step back and reconsider your viewpoint. It only seems not impressive now because we were born in a world where electronic calculation is commonplace and thus taken for granted.
If you don't find this achievement impressive, then go look at some turn-based game where reaction time is eliminated entirely that computers still dominate at, like Chess or Go. The AIs are coming. Or give it a few months and they'll come back with a version hard-limited to half the APM of the human players and it'll still dominate. It's clear which way the winds are blowing on this. People who bet against the continued progress of game-playing AIs invariably lose.
Go read the comments here for this exact same discussion: https://news.ycombinator.com/item?id=10981679
And this is exactly what is being argued here. Let's see that in particular, not a demonstration that computers are faster than humans. Of course they are. Whoever argued that, ever? This has been known and envisioned even before calculators were invented.
What people here are arguing with you for is that we want human-level limitations of the controls for the AI so it can clearly win by better strategy.
Isn't that the goal here?
It can be good enough in a certain problem space, such as chess. But unlike chess or go, which are purely mental games, Starcraft has large physical component (vision, APM, reaction time). It can make it hard to determine when it has “mastered” this RTS. Like you said, it may be a few more months (years?) before AlphaStar can master Starcraft on “mental” level. The physical component is trivial for a computer, so mastering that is not much of a milestone.
In other words, this isn't an interesting handicap to apply.
It's not at all arbitrary. SC2 match is won by a combination of reflexes and physical quickness with which the actions are executed, and strategy.
The whole point is to even the playing field in the area of the physical limitations so that only the strategy part is the difference. You know, the "Artificial INTELLIGENCE" part?
For game design the problem is, the border to macro is not a straight line, but fuzzy, so how far does it go.
For SC2 and this specific bot, the problem isn't there, if the AI merely controls a strategy over hard coded tactics.
1980: X = Tic-tac-toe, Y = Chequers
1990: X = Chequers, Y = Chess
2000: X = Chess, Y = Go
2019: X = Go, Y = StarCraft
2030: X = Any video game, Y = ???
That's the great accomplishment and nothing like that could have been done 10 years ago.
If they beat human performance in this (non-AI-building) field by humans painstakingly coding rules for specific situations, then that's cool I guess but not groundbreaking, because the solution doesn't generalise.
If they beat human performance in a field heretofore intractable by software by throwing the basic rules and a ton of compute at an algorithm and then waiting for six weeks while the algorithm figures the rest out by itself, then that absolutely is qualitatively different.
The reason being, of course, that if they can find an algorithm that works like this across a wide enough problem space then eventually they'll find an algorithm which will work on the question of "build a better algorithm." After which, as we know, all bets are off.
Watch and learn from data alone is why modern machine learning is considered a revolution and novelty. Buying compute time in the cloud is in comparison (to devs and hand coding) dirt cheap and the results are often better.
Deepmind is not working on this problem for the benefit of gamers or the Starcraft community. Making the perfect bot is not the aim. Tackling the next hurdle, next hardest problem in machine learning is. On the way to become better at generalizing the learning algorithms.
And yes, of course computers are much better at doing things more quickly than humans. It's not even remotely close for us. The AIs are clearly better. It's not cheating either; they are legitimately better at it than us.
It sounds like you're simply objecting to pitting people up against computers in real-time games entirely.
The Deepmind team knows the challenge isn’t to beat humans at Starcraft. That is trivially easy with the advantages you mentioned. The challenge is to be better at strategy then a human. That is why they tried to add artificial rules to make the AI have similar physical limitations to a human (emulated mouse, rate limited actions, emulated screen and visibility). There have been micro AI bots for years that could out preform any human. They knew they weren’t just trying to build another micro bot, because if they were it wouldn’t be much of an accomplishment.
It's not trivially easy at all. No one had come close before. It took an entire team of ML experts at Google to pull it off. These hard-coded micro bots you're referring to didn't holistically play the entire game and win at it. They're more akin to an aimbot in FPSes, not a self-learning general game-playing AI.
This is yet another in a long string of impressive AI achievements being minimized through moving the goalposts. It's facile and it's boring.
This is not 100% true, the AI still skips the mechanical part (it doesn't have a mouse, keyboard and hands) in this particular case. This alone can introduce insane amounts of additional complexity, and will make AI to not be pixel precise.
Check out: https://youtu.be/cUTMhmVh1qs?t=3189
blink stalkers are basically perfect for an AI because of the precision they can blink them around.
I watched the live broadcast of this announcement where they did a recap of all 10 previous matches (against TLO and Mana) and they talked about this concern. During today's announcement they presented a new model that could not see the whole map and had to use the camera movement to focus properly. The deepmind team said it took somewhat longer to train but they were able to achieve the same levels of performance according to their metrics and play-testing against previous version.
They did a live match vs LiquidMana (6th match against Mana) against the latest version (with camera movement) and LiquidMana won! LiquidMana was able to repeatedly do hit-and-run immortal drop harassment in AlphaStar's base, forcing it to bring troops back to defend its base, causing it to fall behind in production and supply over time and ultimately lose a major battle.
> it could observe the attributes of its own and its opponent’s _visible units_ on the map directly
1. While it's true that a human player could see everything the AI is seeing, the human player has to spend time and clicks to go see those things, whereas the AI sees it all simultaneously without having to use any actions or spend any time to take it in.
2. Emphasis on the computer seeing it all simultaneously. The computer can see the state of two health bars on opposite sides of the map at the same time, or 100 healthbars in a hundred places at a time. A human cannot do that, and even trying to move the view around fast enough to do so would render it impossible to actually do anything else.
3. If it's true that seeing more at once is not advantageous, then it must also be true that seeing less at once is not disadvantageous. So by that reasoning a player playing on a 1 inch x 1 inch screen would not have any disadvantage, since after all they're getting just the same amount of information as long as they move the screen around enough! Reducto ad absurdum, a player with a 1 pixel x 1 pixel screen has no disadvantage either, because they have access to the same information as long as they move around quick enough. It quickly becomes evident that smaller screens inhibit your knowledge of the game state, and therefore larger screen benefit your knowledge of the game state.
This might be why changing to having to observe only one screenful at a time (rather than the zoomed out view) didn't seem to have as large an effect.
Starcraft is a single-threaded game, so I would think that the AI ultimately still has to enumerate through each visible unit one-by-one to collect their information. Why is that so much different than enumerating through each visible screen and then enumerating through each unit on that screen? Either way, the AI could do it much faster than a human, whether it had to click through the screens manually or not. How would it be possible to eliminate this advantage? It seems to me that it's just part of the nature of AI.
Let me put it the opposite way: If you gave the human player a real time list of every visible unit on the map and all of their information, such that they didn't have to move the screen around manually and could see everything at a glance just like AlphaStar can, would that take the advantage away from AlphaStar? No, it wouldn't because AlphaStar could still go through all that data much faster than any human ever could -- no matter how it's formatted or what you have to do to access it. To AlphaStar, checking all the visible screens is just as much work as scrolling through a list of units.
EDIT: I guess you would still have to wait for the next frame to get rendered, which could add up. True, that does change things a bit, but of course a computer could still do that way faster than a human.
Similarly, robots are physically stronger than people at any given task you can think of. That's a real advantage of them.
Yes, likely! I wasn't doubting it's possible or even likely. Only that seeing an AI do flawless 1000 APM stalker micro and macroing perfectly, while pretty cool, is not as exciting as seeing an AI use a novel strategy (edit: especially one that a human could theoretically execute)
When they were talking about delay they were talking about delay between new information -> deciding/acting, which I think obscures the fact that humans have to do new information -> deciding -> acting, where acting takes non-zero time.
After just decades in development, it is clear that the endeavors of those research scientist have finally bore fruit. And today its in the form of:
Intent based modeling, augmented with AI, which provides the reality we see today in both gaming and weapons systems.
The user, who must be human, is provided a range of inputs based on the desired outcome of the interaction with the systems and the real world.
What results, is truly remarkable.
A human is capable of multi-dimensional abstract thought, in a way that a computer cannot. As such - their intent is wired over to a swarm of objects with the most amazing capabilities.
A user can direct a swarm of either virtual bots or physical drones to accomplish a task. She can also combine the efforts of both physical and virtual for even greater effect.
Here we see a swarm of bots who are thwarted by a physical barrier.
The human driver can then instruct his virtual AI bots to attack the security system of the building to allow his drones to have passage.
But she does this through merely the intent for the portal to be open. The bots do the rest.
All the while the user is updated with real-time information on how the operation is progressing.
So, in the future, you may soon see just such technology applied to your kitchen or living room, where bots will cater to your every waking need - and sometimes your non-waking needs as well.