
OpenAI Five at Dota 2 – The International [video] - nerform
https://www.youtube.com/watch?v=dYXIgOaU1zw
======
rishav_sharan
There are a lot of areas that OpenAI needs to improve on. the bots generally;

1\. terrible at warding and dewarding

2\. are terrible at using spell spam to farm. while their aoe spell usage is
great in team fights, they suck at using spells to farm when they are alone.
Gyro used his ult to farm a normal sized lane and still missed half the
creeps.

3\. don't understand bouncing spells like the lich nuke. In fact many times
lich used that spell as a poking spell.

4\. Bad at non-nuky/long running ultimates. DP made really bad usage of her
ult several times outside of team fights.

5\. Bad at judging Roshan respawn.

6\. bad at using ability runes.

7\. Are bad at juking. They have the right idea to start the juke when the
chasing enemy is on a high ground ramp, but they can't time the highground fog
jukes properly.

8\. not good at prioritizing specific heroes in team fights (cores over
supports, if both look equally chasable)

9\. terrible at dealing with split pushes. They prioritize defending their
towers over everything. Also bad at split pushing, in general.

10\. bad at properly utilizing buybacks. If some heroes buyback during the
defense, it is expected that the remaining heroes should ensure that the
enemies can't escape, otherwise buybacks would be pretty useless.

11\. when they are behind, they simply don't have any coherent way of catching
up other than taking huge team fights. Which the enemy team will deny the AI,
if they are smart.

For now winning lanes and having overwhelming team fighting abilities is the
only meta that the AI seems to be doing.

~~~
emilsedgh
If a team of humans made such horrible mistakes, they would never be able to
compete nowhere near this level.

Is OpenAI's team-fight "so good" that compensates for these huge mistakes and
allows them to compete with pro teams?

For me as a player, the team fights are quite confusing and overwhelming. The
most complex part of the game.

If the OpenAI team got the teamfights covered that well, I'm pretty much sure
they can improve their farming/warding techniques as well.

~~~
WhompingWindows
Warding is a tough one. Outside of hard-coding it, how would you suggest the
AI would learn the best locations for warding and de-warding? Can they surmise
that an opponent's gank or quick reaction to their movement means there may be
an obs ward? If so, where precisely is that?

I don't know if the OpenAI supports can play the warding mind-games, which
also for me personally involve 1000+ games' experience. I see interesting
wards from teammates occasionally and file them away in the memory bank.
Especially "hipster wards", I like to call them, just wards that "see" the
opponents but are not in the highest-probability spots like on pedestals or on
ramp edges. Just throw an obs ward near a couple medium camps, for instance,
and you can really gain some important intel w/o a sentry ruining your ward.
Can the OpenAI team learn to do this sort of behavior?

~~~
Ntrails
> Can the OpenAI team learn to do this sort of behavior?

This is the core of what I want from open ai and feel like I'm not seeing. I
want the AI to _reason_ that blocking the creepwave brings the equilibrium
back to your tower and results in an easier lane. Not just to say "this thing
works so I'll do it a lot" \- but to say "if I do this then it will have this
impact on the game state".

What we got was the devs specifically training models for goals like creep
blocking. Which, you know, just seems a bit meh

------
adverbly
Reaction time seems to be a lot more complicated than just 200ms flat. There
were a lot of superhuman reactions that game which really made it difficult
for humans to start a fight. At the same time though, increasing much more
than 200ms might give a big advantage to humans in other situations. It will
be interesting to see if they can find a better way to model it.

All in all I am very impressed by the AI. So many complex strategies that it
is employing(grouping for towers, lane swaps, etc...). But at the same time,
it really does look at the moment like this is only close because the AI has
vast advantages from mechanics and teamfighting.

~~~
glalonde
The human would probably be able to blink/cancel if they were expecting it and
therefore able to focus on a single thing prior to an event. The bot can focus
on everything simultaneously and doesn't really need to expect anything. It
gets a signal, it responds within 200ms, no problem. You could program that
analytically.

So I would say the superhuman-ness isn't in the number of actions taken, or in
the response delay, but in the massive attention bandwidth. I believe they've
attempted to even the playing field in the first two, which are easily
quantifiable, but I don't know about the latter.

~~~
IMTDb
Can't they "just" add a cost to each API call ? Wanna know the position of an
enemy on the map: 50ms. Target a hero/creep/tower: 75ms.

~~~
josefx
Just make them drop the API and require use of computer vision on the same UI
human players have to use.

~~~
conjectures
Yeah, this is the thing that I wasn't expecting when I first saw this
reporting. Preferably with some lag to cursor moves etc.

------
a_humean
So three major differences between this and the last big match for OpenAI Five
which explain the different result in favour of the Humans:

1) The Humans, Pain Gaming, is a professional team with players that regularly
play and practice together and attend major tournaments. While they are the
weakest team at this event and are arguably there only because of a regional
qualifier system (they are from South America and the strongest team in an up
and coming region), they are capable of taking games off some of the strongest
teams due to their unpredictable and dynamic play. They are substantially
better than the team of ex-pros and commentators that last played against the
AI.

2) They removed an important limitation in the game, which was the 5
invincible couriers, and replaced it with a single killable courier that the
five agents have to share (just like in a normal game of dota). The AI was
able to use the single courier effectively, but did occasionally let it die a
bit too often. What is important about this change is that it invalidated a
strategy of the AI which was to continuously ferry consumable healing items to
enable relentless aggression after gaining a slight edge in the early game.
The AI adapted to this change by playing more cautiously in light of this
standard resource/logistics constraint. It adds strength to the argument that
as the OpenAI team introduces more complexity and resource constraints back
into the game that the AI's weaknesses start to become more apparent and
exploitable by better players.

3) The heroes composition was drafted in advance to be as even a match as
possible, and a coin flip was made to decide which team got which draft. This
is important as the version of the game is a subset of the game, and the
humans evidently didn't understand the meta game in this weird subset during
the last event. From the AI's perspective this was as even a draft as possible
given the very small hero pool. In the last event the AI predicted 70-95% win
confidence before the game even started due humans not understanding the
drafting meta with a pool of 18/115 heroes and no bans.

 _Still, really impressive performance by OpenAI as at least for the early
game and some of the mid game it was a close game with lots of good plays by
both sides. The fact that OpenAI can be comparable to a pro team in what is
nearly a full game of dota is really impressive._

~~~
ollerac
> 1) The Humans, Pain Gaming, [...] are the weakest team at this event and are
> arguably there only because of a regional qualifier system...

I think it's totally disingenuous to call them the weakest team at the event.
Two other teams performed worse in the group stage and one tied Pain's score,
but still made it into the main event through sheer luck. Pain has won games
against every single one of the strongest teams there.

If you follow pro Dota, you know a lot of these games hinge on how a team
happens to be performing on a certain day. Just this past May, Pain played in
one of the top Dota tournaments of the year with 9 other teams who also happen
to be at The International with them now -- and Pain came in 3rd place, ahead
of Fnatic, OG, and Mineski.

~~~
a_humean
And you just selectively quoted me to remove all of the qualifications and
praise that I used to make it appear that I'm unduly harsh.

In that paragraph where you selectively quoted you removed the clause where I
said: "...they are capable of taking games off some of the strongest teams due
to their unpredictable and dynamic play", which is is no different to the
point you are trying to make.

Fact is, they came last, they were lucky to attend given it was a surprise
that their region was given a slot for the first time ever, but proved
themselves to be a capable team from a region that is starting to get
international exposure, and you are deliberately misrepresenting me.

------
jsheard
For context: OpenAI is playing against a professional team this time, unlike
the event a few weeks ago where the humans were skilled but not active pros or
used to playing together.

This team isn't the strongest (they placed 17-18th during this TI) but OpenAI
will play again tomorrow and the day after, presumably against progressively
better teams.

~~~
sambroner
Seems like playing one of the best 200 players in the world is a reasonable
proxy for a top tier player.

I'm unfamiliar with DOTA rules, but does anyone know if there are any
limitations on the openAI team? e.g. Things like keystrokes per minute, scroll
speed, etc

~~~
jsheard
During the last OpenAI showmatch the bots had an artificial 200ms reaction
time imposed on them, I assume it's the same here but I'm not sure.

~~~
larkeith
I don't believe this is the case, as OpenAI was able to hex (an immediate
disable) Earthshaker before he got off Echoslam (a spell with no cast delay
except the time to click the key).

~~~
def_true_false
The ES player didn't queue his abilities so there was delay between blinking
and casting echo. Someone counted the frames and it was well over the 200ms
minimum.

~~~
el_devo
Here is a post from the Dota 2 subreddit discussing the timing with proof that
OpenAI's reaction was over the 200ms minimum:

[https://www.reddit.com/r/DotA2/comments/94vdpm/openai_hex_wa...](https://www.reddit.com/r/DotA2/comments/94vdpm/openai_hex_was_within_the_200ms_response_time/)

~~~
Q6T46nT668w6i3m
Yeah. I agree. It was over the 200ms minimum but it was artificial. No human
could reasonably perform that type of action as relilably. And that has
nothing to do with learning performance.

~~~
eptcyka
You can queue abilities to instantly cast them after the previous one
finishes, so it is actually quite reasonable for a human to perform that type
of action quite reliably.

~~~
def_true_false
They probably meant the reaction, not the blink and echoslam.

------
jjcm
Overall a very good game. Definitely smart on PAIN gaming's part to delay the
game and wait until the mid game (where OpenAI traditionally excels the most)
is over in order to rely on their stronger carry. I think the lack of 5
couriers really showed where OpenAI had a crutch the last game it played.
Excited to see the improvements as this goes on, and I suspect that next year
we'll see the OpenAI team beat the #1 ranked human team.

~~~
nindalf
Loved the strategy used by Pain. They recognised that fighting OpenAI wasn’t
working so they actively avoided fights. They recognised strategic
shortcomings, like the AI’s reluctance to pressure lanes and exploited it.
What disappointed me a bit is that the AI didn’t make a concerted effort to
win the game when it was strongest.

~~~
chibg10
This type of playstyle adjustment by Pain is just standard Dota strategy tbh.
When your team is losing teamfights, you avoid teamfights unless you know you
have an imminent advantage (terrain, towers, initiation, numbers, etc.). A big
part of avoiding teamfights is splitting up and putting pressure on lanes so
that the other team has to split up to defend their towers. If they are split
up and defending their towers, they can't group up and force fights at
objectives (Roshan, towers, w/e) that your team would have to group up to
defend.

Not to take anything away from Pain--just making it to TI is enough to prove
that they're excellent Dota players.

------
tokipin
The bots were using their ultimates pretty liberally. Maybe their "bot meta"
is very group-intensive, so if they see one hero there's a good chance there
are other heroes nearby? Other possible reasons: Opportunity cost of not using
ultimates, getting a single kill late game can mean victory, bots penalized
for games going later or their games just not running long in general?

~~~
Q6T46nT668w6i3m
I think it’s becoming clear that the network is overfitting to team fights
(and other small duration action). OpenAI use of cooldowns in the last 20
minutes was inexcusable. It became obvious OpenAI had no working strategy.

~~~
roenxi
Is OpenAI's expected win chance available anywhere? Only the bot reaction time
against initiations was keeping them in the game for the last 20 minutes, so
it might have been the desperate flailing typical of bots when they are
losing.

~~~
dx87
The bots are actually calculating their expected win chance continuously
throughout the match, even while the heroes are still being picked. During the
OpenAI Five benchmark, the AI estimated its win chance at over 90 percent when
it got to pick its own heroes, but estimated a mid 20% chance of victory when
it was deliberately given bad heroes.

------
Kagerjay
DoTA has come such a long way from being just a warcraft 3 mod. All those
random pub matches I used to play with w33, analyzing all the patchnotes and
.w3g replays before youtube was a thing, to shitty garena pubmatches, to a
full fledge esports remake, to going to my first Dota TI2 international 2
years back, and now AI is being thrown in the mix. I wonder what the future of
DoTA will look like now

~~~
emilsedgh
But the popularity is a bit declining, isn't it?

~~~
nstart
Going to go off on a tangent here.

Yes and no. By raw numbers it certainly is in a state of decline. But put in
context, it may have been inevitable. It has everything going against it.

DotA is a team game.

It's incredibly complex. Not easy to pick up. Mastery requires 1000's of
hours. And it's miserable to play at the start.

The community is toxic. So random teams in a public match when you are new to
the game compounds. Even if you aren't new, one way or another random teams
can quite often be a frustrating experience that you just don't feel you have
control over.

Compunding even further is the fact that once you start a game you are locked
in for 35-55+ minutes. That can be 35 painful minutes of feeling crap.

Compare that to rising battle Royale games, and you get solo games you are
allowed to be bad at and slowly get better in. And the time you spend in it is
directly related to how much fun you have. And the games have a definite time
cap. And the mechanics are simple. This holds true for a lot of other up and
coming online casual multiplayer games.

Point is, Dota is destined to be niche as a game that is played.

What i would love to see is how the game can change externally to become more
accessible to viewers. From client changes to community changes to changes in
presenters, valve needs to step up to help people participate as spectators.
The potential for the game to generate significant money lies in their pro
circuit and it looks like valve is really stepping up their involvement here.
Hopefully this continues.

The (somewhat poor) parallel is something like rugby or squash. Fairly arcane
rules. A lot of people don't remember when they last played the game. But they
can still be heavily invested in watching.

~~~
Kagerjay
^ This

DoTA is incredibly complicated, and has more compounding interest towards
tactical strategy + gameplay + teamwork + skill execution, more so than battle
royale.

In battle royale, if you played awful you just die. Then you start a new game.
With DoTA that death impacts your game in the long run, putting your team at a
disadvantage, since you just fed their carry. But comebacks do happen though

Going to DoTA TI:2 internationals, where all the major esports teams compete
is different than anything else you've ever experienced.

Its like going to the superbowl. Except, everyone knows how to play football,
usually on an average / above-average level. But, instead of a few hours, its
8+ hours, for 7+ days. It gets EXTREMELY intense, just because you because you
can relate to how difficult manuveurs are, and it isn't dictated by things
like how much weight you can squat in football (For faster sprinting /
agility, etc). There's many little minor mental and physical gymnastics that
DoTA2 pro players do 100xs better than your average player.

Comparing this to Fortnite competitive mode, its not the same. Its more
closely tied to watching something with less strategical depth, such as
tennis. Tennis is still exciting for its own reasons, but its mostly skill
based execution, and mostly a solo game. Even if you played 2v2.

I watch both fortnite and dota2 competitive every so often

------
ggregoire
Wait… the big news here is steam.tv. I heard the rumors but didn't know it was
up?!

(it seems to work only for their own event at this time tho.)

~~~
kawsper
They announced it a couple of days ago: [http://blog.dota2.com/2018/08/the-
main-event-with-new-steam-...](http://blog.dota2.com/2018/08/the-main-event-
with-new-steam-broadcasting/)

It still have limitations and bugs, yesterday it just wouldn't load for me,
but it works now, and you are unable to watch yesterdays content.

~~~
ehsankia
The UI is very wonky, definitely looks like it was rushed last minute to
release for TI. That "Watch with friends" button is pretty messed up if you're
logged in. Also if you're logged in and "close" the stream tab, I don't see
any way to get back to it. Definitely a work in progress.

------
sama
We are playing with regular couriers for the first time! And against a pro
team.

~~~
hamhamed
Pain isn't the toughest of opponents at TI. Will they be playing vs a team
that is currently still alive in the bracket? So far it seems Pain is kind of
controlling the game at least much better vs the caster team (99th percentile)

~~~
noxvilleza
They're still the ~19th best team in the world.

~~~
rajahafify
No. TI is not a meritocracy. Pain came from the weakest region in the world.
Without the SA slot, they wouldn't even be in TI.

~~~
xfer
And you know that, how? Have you seen them fight against other region teams
that failed to qualify?

~~~
andreime
Maybe, like me, he follows the pro scene.

People complain a lot that Europe is stacked (meaning stronger teams) compared
to regions like NA, and they get the same amount of slots a TI. South America
is probably the weakest region.

I think a lot of people following the competitive scene would agree with what
he was saying - that Pain is not top 20 in the world.

~~~
xfer
I follow the pro scene too, nobody sensible would say they are not top 20, can
you name a team that deserved to be in TI other than pain here from any
region?

> People complain a lot that Europe is stacked (meaning stronger teams)
> compared to regions like NA

Really? Go ahead and name an EU team that is not playing in TI that should be
here. EU is top heavy and they get to TI easily, other than that not much. SA
is weaker region for sure, that's why they have 1 team from their region and
they deserve to be there.

------
21
How can I tell looking at the screen who is winning? Is there a score
somewhere? Maybe the left side sliders?

~~~
noxvilleza
Net Worth Difference is the best single indicator, but there are additional
factors like tower difference, item purchasing decisions, hero composition,
hero aliveness & buyback status, aegis, etc. Some specific heroes also wildly
differ from the norm for each one of these factors. The game is very complex
:()

~~~
nindalf
Real noxville?

------
Leary
Outfought the humans in teamfights, other than that not sure how much long
term strategic thought was shown.

~~~
nindalf
It was frustrating to see it do so many things well but make strange decisions
otherwise

\- poor warding. Some wards were simply wasted. (Wards are small, invisible,
immovable units that grant vision) \- using powerful long cool down spells to
earn gold instead of keeping it in reserve for fights \- not recognising that
one of the lanes needed to be pushed out. Eventually this bit them in the ass.
\- using buybacks where none were required

~~~
dx87
I don't know if it's still the case, but the OpenAI developers said that the
reason why it sometimes wastes wards and smokes is because it doesn't know how
to drop items, so it'll just use them to free up inventory space if it wants
to buy something new.

~~~
ajuc
I played dota for half a year not knowing you can doubleclick to switch which
ward is on the top.

I always disabled joining wards, but when I forgot - I warded like openai,
because I had 3 obs wards on top of a sentry and needed to put a sentry
somewhere :)

------
alexedw
Looks like OpenAI just lost Match #1.

------
Analemma_
Hooray, humans not obsolete quite yet!

Very good play from OpenAI though, even though it ultimately lost. It looked
very scary how fast it could switch between dominating 5-man teamfights to
splitting up and ganking or pushing down towers.

------
sh4z
Updated the ruleset with regular couriers! Will be a lot different from the
earlier matches. I'm guessing humans can win if they find "cheese" plays that
the bot has not found during it's practice.

------
abakus
At the current state, openai five stands no chance against stronger teams (eg
liquid lgd) of TI8.

------
symlinkk
Their reaction time seemed way too fast. It was entertaining to watch though.

~~~
noxvilleza
It's actually internationally delayed reactions to make it more fair (200ms of
delay added).

~~~
makoz
Watching the match there seemed to be a crazy amount of instant Euls and what
not.

I doubt a player is about to react to a blink initiation and click Euls/Hex in
the same amount of time. It'd be a lot more fair for them to calibrate against
the reaction time of pro-players across the same scenarios. (I doubt pros can
hit 200ms consistently)

~~~
noxvilleza
That or there's something like 200ms "windows" in which API sync occurs - so
if someone Blinks next to you you can react within 1 tick (33ms) if the timing
is right.

------
greenmountin
Interesting that the draft was predetermined (unlike the previous exhibition,
which was live), allowing OpenAI to train ahead of time on a restricted set of
matches. I wonder how long that was?

~~~
wnevets
Since Saturday they said.

------
jlebar
Youtube link:
[https://www.youtube.com/watch?v=Y2EQCE9LRXE](https://www.youtube.com/watch?v=Y2EQCE9LRXE)

------
white-flame
> _OpenAI is a non-profit artificial intelligence research company that aims
> to promote and develop _FRIENDLY AI_ in such a way as to benefit humanity as
> a whole._

I find it amusing & ironic that they're pursuing games of efficiently &
strategically killing enemies as examples of their successful progress. ;-)

~~~
hsrada
> I find it amusing & ironic that they're pursuing games of efficiently &
> strategically killing enemies as examples of their successful progress. ;-)

Except that the amusement and irony would cease to exist once you have some
idea of what's happening behind the scenes.

The AI doesn't know it's "killing enemies". For it, it's just something that
results in the increase of a numerical reward signal.

~~~
tree_of_item
Is that supposed to remove the amusement and irony? "We're creating friendly
AI! Well, it _thinks_ it's friendly, since to it, everything is just a
numerical reward signal."

------
czardoz
Is this five different OpenAI bots cooperating in the game, or a single engine
controlling five boots?

~~~
pure-awesome
It's five different instances:

[https://openai.com/five/#how-openai-five-works](https://openai.com/five/#how-
openai-five-works)

------
sama
rigged IMO

~~~
ISL
Citation needed?

~~~
minimaxir
Sam Altman is a jokester.

------
bxio
This time, OpenAI is playing a normal game without restrictions. Team Human
got thoroughly trashed last time vs OpenAI in a game with hero restrictions.

Go OpenAI! I For One welcome our new robot overlords.

~~~
Analemma_
Not _quite_ a normal game with no restrictions because it's a pre-selected
hero pick from a limited pool instead of a full draft, but it's still very
cool.

Interesting that OpenAI seems to prefer deathballing, but it makes sense: its
main advantage over humans is probably tactical and in teamfights, and 5-man
maximizes your options. The human strategy should probably be to split push,
but one of the commentators (who is also a pro who played against OpenAI
earlier) says that is very difficult because OpenAI can apply pressure
everywhere.

~~~
balls187
I recall reading somewhere that the real advantage AI had was in it's ability
to come up with bizarre strategies that confuse human opponents.

~~~
ionforce
I'm skeptical that you read this somewhere. AI doesn't "come up" with
strategies. It's likely something that's been discovered in training and then
mechanically repeated. But it doesn't "come up" with strategies out of
nowhere.

~~~
tree_of_item
Why doesn't it come up with strategies? The program is doing a massive search
over an action space, of course it will find things there.

~~~
balls187
It's a dumb semantics argument about using the phrase "come up."

------
branevoid
Can the bots learn from the previously played international matches?

------
tachikomagenius
for a bot, they sure are doing pretty well versus the top 5% of the skilled
dota players, assuming that only the top players around the world can enter
the main international event.

~~~
unrealhoang
It’s 0.05%. But also the game OpenAI plays is very limited. It’s a very small
game compare to the real Dota. They just got rid of a small limitation of
having 5 invulnerable couriers and already being punished hard (since their
push timing being delayed). And Dota is much more complicated than just team
fighting and death ball.

------
AmitJS
But this bot can defeat a bunch of heralds and guardian players.

------
scns
If you want better quality watch the viedo by dota 2 rapier

------
yskchu
Since maybe others will be looking for it too: the link above is for the live
show - for the recording of the match (which is over) there's a link on
Youtube:
[https://www.youtube.com/watch?v=Y2EQCE9LRXE](https://www.youtube.com/watch?v=Y2EQCE9LRXE)

