There are a lot of areas that OpenAI needs to improve on. the bots generally;
1. terrible at warding and dewarding
2. are terrible at using spell spam to farm. while their aoe
spell usage is great in team fights, they suck at using spells to farm when they are alone. Gyro used his ult to farm a normal sized lane and still missed half the creeps.
3. don't understand bouncing spells like the lich nuke. In fact many times lich used that spell as a poking spell.
4. Bad at non-nuky/long running ultimates. DP made really bad usage of her ult several times outside of team fights.
5. Bad at judging Roshan respawn.
6. bad at using ability runes.
7. Are bad at juking. They have the right idea to start the juke when the chasing enemy is on a high ground ramp, but they can't time the highground fog jukes properly.
8. not good at prioritizing specific heroes in team fights (cores over supports, if both look equally chasable)
9. terrible at dealing with split pushes. They prioritize defending their towers over everything. Also bad at split pushing, in general.
10. bad at properly utilizing buybacks. If some heroes buyback during the defense, it is expected that the remaining heroes should ensure that the enemies can't escape, otherwise buybacks would be pretty useless.
11. when they are behind, they simply don't have any coherent way of catching up other than taking huge team fights. Which the enemy team will deny the AI, if they are smart.
For now winning lanes and having overwhelming team fighting abilities is the only meta that the AI seems to be doing.
Warding is a tough one. Outside of hard-coding it, how would you suggest the AI would learn the best locations for warding and de-warding? Can they surmise that an opponent's gank or quick reaction to their movement means there may be an obs ward? If so, where precisely is that?
I don't know if the OpenAI supports can play the warding mind-games, which also for me personally involve 1000+ games' experience. I see interesting wards from teammates occasionally and file them away in the memory bank. Especially "hipster wards", I like to call them, just wards that "see" the opponents but are not in the highest-probability spots like on pedestals or on ramp edges. Just throw an obs ward near a couple medium camps, for instance, and you can really gain some important intel w/o a sentry ruining your ward. Can the OpenAI team learn to do this sort of behavior?
> Can the OpenAI team learn to do this sort of behavior?
This is the core of what I want from open ai and feel like I'm not seeing. I want the AI to _reason_ that blocking the creepwave brings the equilibrium back to your tower and results in an easier lane. Not just to say "this thing works so I'll do it a lot" - but to say "if I do this then it will have this impact on the game state".
What we got was the devs specifically training models for goals like creep blocking. Which, you know, just seems a bit meh
I assume that the robots are god-level last-hitters and farmers. perfect farming efficiency is enough to beat 99% of human teams. most players struggle to get 50 last hits in the first phase of the game, nevermind actually get every single piece of eligible farm.
edit: i would also expect that the robots are deny nazis, who severely crimp the farming of anyone who opposes them in the lane.
That's not really the same version of the AI that's playing 5v5 now though. The Shadowfiend 1v1 bot was near perfect at all the skills needed for that match-up, while the current bots are far worse at even some mechanical skills like last-hitting.
Yes, along with the fact that they are not really playing the same game that humans do:
- 5 invulnerable couriers per team allow for very constant poking and pushing styles as regeneration items can be ferried back and forth from the base. This favours team fighting deathball oriented strategies where you just have non-stop aggression.
- Very limited hero pool, this further compounds on the earlier issue as many good deathball team fighting heroes are on the available pool but some very common counters to this strategy are not.
Also, humans lack experience in this version of the modified "metagame" or they would likely be able to adapt. This is IMO the biggest hurdle for humans, they haven't really played this version of the game and they probably weren't exactly expecting it, the AI has trained over many "computational years", the humans have a couple of games to catch up.
That's what you would tell human players, yes. That's what a trainer would tell you after looking at your gameplay. That's what you would then try to improve.
BUT!!
This is AI, so what OpenAI needs to improve on, if anything, may be the ability to tell this stuff directly to the AI so it could skip hours of play.
Also, maybe our view is too limited and the AI actually learns even better strategies from competitive self-play. Strategies our human way of improving would miss.
> Also, maybe our view is too limited and the AI actually learns even better strategies from competitive self-play. Strategies our human way of improving would miss.
This is generally a good point, but in this specific case we can see that no such strategies showed up. The bots did terrible, obviously bad things (like stacking wards in bad places on top of each other or having a weak support hero take the aegis). More importantly, they didn't win.
You may be missing the point here: it's AI, so it doesn't "think" in terms of strategies or tactics like you do, it just runs countless simulations, figures out what works in those, and then blindly applies that. So if the human opponents do things it has not been trained to cope with, they'll have a significant edge.
Aside: As a non-Dota-playing HNer, the only one of those points I found comprehensible was #11. Gaming jargon can be pretty impenetrable to the uninitiated...
Not sure if it helps much (I'm sure you'll find plenty of better definitions online):
Purpose:
- 5 vs 5 heroes over 3 lanes and a jungle
- destroy the enemy team's towers in order push towards the enemy's Ancient (base) and destroy it
> warding and dewarding
wards serve to create visibility (over the fog of war)
2 types of wards: one for general visibility, and one to detect invisible heroes (but has no visiblity over FOW)
> spell spam to farm
having a hero use multiple spells to rapidly kill a bunch of Creeps sitting somewhere in the jungle
> bouncing spells like the lich nuke
- poke spells target and damage a single enemy hero
- bouncing spells target one enemy hero but can jump to nearby enemy heroes as well
> Roshan respawn
Roshan is a large creep in the jungle. It drops a precious item that resurrects heroes when they die.
> ability runes
Runes are temporary enhancers (movement speed, attack damage, ..) found in the jungle
> bad at juking
when one tries to play mind games with the enemy
> buybacks
when a player's hero dies they're allowed to pay a fee to revive their hero instead of waiting (can only be used once)
I'll try to give another translation. Overall, OpenAI excelled at short term thinking, but showed weaknesses in some areas of the game that demand long term thinking. This is not to say that it was bad at all kinds of long game thinking -- it seemed to do a good job at deciding what areas of the map they should prioritize or deprioritize.
Dota is a 5v5 game where each player controls a single hero unit. The playing field has three paths that connect the opposing bases, and an endless source of dumb computer-controlled units march along these paths to try to destroy the opposing base (in a perpetual stalemate). The objective of the game is to power up your hero units (by defeating enemy heroes and computer-controlled units) until you have an overwhelming advantage, by which point you can mount an attack on the enemy base.
> 1. terrible at warding and dewarding
Dota has a fog-of-war system, inherited from its real-time-strategy roots. There are two main game mechanics that interact with the fog of wr: wards and smokes. Wards are stationary units that can be placed at a spot on the map to provide vision around them. Smokes are a consumable resource that can temporarily make your team invisible, so you can sneak through your opponent's wards undetected.
In regular Dota play, map vision is highly valued, to the point that there is a hardcoded limit on how many wards and smokes each team is allowed to use.
For all intents and purposes, OpenAI seemed to have no idea what it was doing with these mechanics. It placed wards at low-value places (instead of high value map intersections), and often wasted them by placing multiple wards on the same spot.
> are terrible at using spell spam to farm.
Farm in Dota means collecting resources to power up your hero by defeating dumb computer-controlled units (as opposed to by defeating the enemy heroes). The slang is because it is a relatively peaceful and patient way of acummulating resources. Farming is all about establishing control of a region of the map so that your team can defeat the computer controlled units there but your opponents can't.
OpenAI showed that it was very good at using their heroes' abilities to fight enemy heroes head on, but it wasn't as good at playing the "economic game" and using their spells to accumulate resources when there was no fighting going on.
> don't understand bouncing spells like the lich nuke
A handful of spells in Dota "jump" to nearby enemies after the first one, like a hot potato. Usually you want t ouse them when multiple enemies are next to each other but OpenAI was more willing to use these spells een when they didn't give an extra bounce.
I think that this might be a side effect of the AI's self-play. OpenAI has superhuman reaction times and is very good at positioning and moving around to avoid getting hit by those additional "jumps", so it wouldn't value them very highly.
> Bad at non-nuky/long running ultimates
Some hero abilities in Dota can be used very often (once every few seconds) while others have a longer cooldown time (once every 2 or 3 minutes).
In regular Dota play, you only want to use these major spells when contesting an important objective against your enemy, because if you "waste" them, the enemy team can take advantage of the time when your heroes aren't at their full strength.
OpenAI seemed to do a bad job at this kind of long term thinking. It would use some of its powerful abilities against computer-controlled units and then not have them available for an important fight vs the humans.
> Bad at judging Roshan respawn.
Roshan is a powerful computer-controlled monster that appears at the middle of the map. It takes a team effort to defeat, but the team that does so receives a very high reward.
After Roshan is defeated he shows up again after 8 to 11 minutes. Good players know that they can ignore Roshan for these first 8 minutes but that after that they must pay close attention to the area around its cave to prevent the enemy team from sneaking there undetected and defeating Roshan uncontested. OpenAI didn't seem to have learned this timing behavior. They would constantly check if Roshan was present in his cave, even in times when it was mathematically impossible.
> Are bad at juking.
Juking is trying to "break your opponents ankles" when they are chasing you around the map, by exploiting the fog of war. Enemies can't see higher ground or behind trees, so you can use those opportunities to quickly change the direction you are running in an attempt to evade who is chasing you.
OpenAI seemed to do a bad job of timing these direction changes. Ideally you turn to move in a different direction as soon as you enter the fog of war, no sooner and no later.
> not good at prioritizing cores over supports
In the later portions of a typical Dota game, 3 of the five heroes in each team will be very strong at fighting (the cores) and 2 will be weaker (the supports). Usual strategy is that it is more important to prioritize defeating the strongest heroes in the enemy team. OpenAI was more willing to go for the weaker enemies first, but I am not sure that is a bad thing actually.
> terrible at dealing with split pushes.
The most direct way to attack the enemy base in Dota is to defeat the enemy team in a head on fight and use the window of time when they are incapacitated to attack the undefended base.
Split pushing is a strategy where you try to attack the base when you think the enemies are grouped up far away from it and can't defend it immediately. The humans successfully used this strategy to delay the game, by forcing the bots to abort their frontal assaults and retreat to defend their base.
> bad at properly utilizing buybacks.
In Dota, heroes are incapacitated for a brief time after they are defeated. In the later parts of the game it is around 1 or 2 minutes in the sidelines.
There is an option to spend some resources (sacrificing future strength) to immediately return your hero to action.
This is a very high-risk high-reward play that needs to be timed appropriately to be worth the investment. The bots made some questionable plays around this game feature.
I think while most of your points are valid, some of them miss the mark.
> 3. don't understand bouncing spells like the lich nuke. In fact many times lich used that spell as a poking spell.
I'm not convinced that their use of Chain Frost is necessarily bad. They are likely still used to their five man death ball strategy, where a single kill is often enough to secure a tower. In that case, using Chain Frost on a single hero may very well have better expected value with the guaranteed kill than waiting for an opportunity and potentially getting no kills, and even worse, pushing the game late where they cannot play effectively.
> 5. Bad at judging Roshan respawn.
This is technically true, but misleading. Humans determine this more accurately simply because we have been told the actual time range, but if you couldn't look it up, you would be bad at it too. You could try it in game, but imagine you weren't allowed to start a demo match, you were only allowed to do it in a live game. Furthermore, you have no access to patch notes, so you need to keep checking the Roshan pit in case they changed the respawn time. In fact, you don't even know that it respawns and then stays there. Maybe it only shows up for 30 seconds on every even game time minute, and disappears if it isn't attacked. Given the huge advantage that taking Roshan gets, maybe it is actually worth it to stick around there if you have no idea what the respawn rules are.
> 8. not good at prioritizing specific heroes in team fights (cores over supports, if both look equally chasable)
I didn't see this in the game.
> 9. terrible at dealing with split pushes. They prioritize defending their towers over everything. Also bad at split pushing, in general.
I didn't really see this in the game either. Axe was pushing, but what could they have really done about it? If they tried to wrap around and gank him, that would just split them up and leave them open to counter attack by the much stronger human team, particularly the Sniper.
In regards to the second point, in one of the caster games, the AI sacrificed Sven in order to secure the bottom T2 tower. Sounds like split pushing to me.
> 11. when they are behind, they simply don't have any coherent way of catching up other than taking huge team fights.
What would you suggest they should have done in this game then? It seemed to me like they didn't have very many options left at the end.
1. terrible at warding and dewarding
2. are terrible at using spell spam to farm. while their aoe spell usage is great in team fights, they suck at using spells to farm when they are alone. Gyro used his ult to farm a normal sized lane and still missed half the creeps.
3. don't understand bouncing spells like the lich nuke. In fact many times lich used that spell as a poking spell.
4. Bad at non-nuky/long running ultimates. DP made really bad usage of her ult several times outside of team fights.
5. Bad at judging Roshan respawn.
6. bad at using ability runes.
7. Are bad at juking. They have the right idea to start the juke when the chasing enemy is on a high ground ramp, but they can't time the highground fog jukes properly.
8. not good at prioritizing specific heroes in team fights (cores over supports, if both look equally chasable)
9. terrible at dealing with split pushes. They prioritize defending their towers over everything. Also bad at split pushing, in general.
10. bad at properly utilizing buybacks. If some heroes buyback during the defense, it is expected that the remaining heroes should ensure that the enemies can't escape, otherwise buybacks would be pretty useless.
11. when they are behind, they simply don't have any coherent way of catching up other than taking huge team fights. Which the enemy team will deny the AI, if they are smart.
For now winning lanes and having overwhelming team fighting abilities is the only meta that the AI seems to be doing.