
OpenAI at the Dota 2 World Championships - frisco
https://openai.com/the-international/
======
darrenkopp
Really was quite interesting to watch
([https://www.twitch.tv/videos/166172514?t=7h3m10s](https://www.twitch.tv/videos/166172514?t=7h3m10s)).
Honestly, the bot played _extremely_ well, but I think the biggest advantage
was how much faster it's reaction time was and it's movements were likely much
more precise than a human is with a mouse.

I'm pretty interested in seeing their 5v5 results as well. It seems like that
will have similar results as the bots can coordinate, but it's still a bit up
in the air.

I'm really not sure how well a bot like this would do with 4 human teammates
though. I would guess the bot would be a strong laner, but fairly weak overall
due to it's inability to communicate, though if it's really good at learning
how it's current teammates are playing as it goes, it may do alright.

I'm also pretty curious about the match limitations: no shrines (regen), no
soul ring (mana regen at expense of some health), no raindrops (fixed amount
of magic damage block)

~~~
dirtyaura
It would be a fantastic side project to teach bots to learn to speak "Dota".
Learning to communicate efficiently is likely much easier as the vocabulary
and intentions behind them are constrained.

~~~
quadcore
Side question: if we consider a team of 5 bots, would they need to
communicate?

~~~
sbarre
If you wanted them to play "within the rules", they would need some way to
communicate via the game.. Text chat would make the most sense since adding in
TTS / STT would seem unnecessary.. They could even communicate in some kind of
shorthand language that only the bots understand..

~~~
Cyphase
I think the grandparent is asking whether the bots would need to communicate
at all, as opposed to just "knowing" what the other bots are "thinking".

------
dvt
Saw this live and as someone that's played Dota for many years, it's not as
impressive as people make it out to be. A Shadowfiend 1v1 is primarily a test
of mechanical skill and _not_ judgment.

A similar comparison is if a Counter-Strike or Quake bot would instantly snap
to your head getting a headshot. Like, yeah, cool I guess, and it would beat
100% of human players 100% of the time, but it's not even remotely as
impressive as Chess or Go. The razes in particular are "skill shots" that are
sometimes hard for humans to estimate whereas for a robot it's just simple
math.

I will say that the one impressive aspect of the bot was raze faking, but I
think that it was one of those cases where it was "coached" (same as the creep
blocking).

~~~
jlebar
> but I think that it was one of those cases where it was "coached" (same as
> the creep blocking).

What makes you think that? In the interview, they explicitly disclaim this.

~~~
dvt
In the interview, they literally said they coached it to have certain
behaviors.

~~~
keerthiko
They said "yes, it does that"/"it's been known to do that", not "it was
coached to do that". Just as a "We've seen it do that before."

~~~
dvt
Yeah, just re-watched the clip, I think you're right. My mistake (the word
"coached" confused me).

------
ahh
The developer interviewed claimed that there was no domain-specific knowledge
and implied (though didn't explicitly state) there wasn't any training against
non-OpenAI bots or players. (I'd love to know the reward function they used
for whatever Q-learning variation they ran with.)

If this is accurate, one of the things I'm most impressed with is that the bot
figured out creep-blocking. (I can't find a good GIF, but this is walking in a
wiggly path in front of the first wave of neutrals on your side, delaying
their progress and pushing the lane towards you, which is good for ~reasons.)

Creep blocking isn't all that hard in dexterity--I am a terrible dota player
and I can more or less do it. And it's one of the most common pieces of dota
knowledge; every pro player does it and since it's relatively easy compared to
a lot of pro micro, everyone else rapidly learns they should.

But nevertheless--the bot had enough games that it could randomly jump in
front of the wave enough times that it _noticed_ a win rate improvement for
that slight wave push, and begin to do it intentionally? (And then get good at
it?) Damn.

One thing I don't really know about Q-learning and the typical nets used for
it: I am guessing it is likely that internally to the bot's evaluation
functions, there is some learned feature whose activation correlates well to
the location of the wave equilibrium (since that's a feature that correlates
well with winning!) At that point, is it likely that the bot can learn in
smaller increments--that is, it knows that pushing equilibrium towards itself
is good, and thus randomly creep blocking a little becomes reinforced (rather
than having to notice the creep block's effect on _game_ wins?)

~~~
thefreeman
I don't know that they specifically said there was no domain-specific
knowledge. iirc they said they didn't "teach it the rules of dota" but they
also said the training involved "coaching". I interpret that to include
showing the bot useful techniques (like creep blocking) which the AI then
learned leads to higher win rates, etc.

~~~
sondr3
They also mentioned that in the beginning the bot figured out that the best
way to win was to not play the game (aka hiding in it's own base). Then it
started running around wildly, dying to enemy towers in the wrong lanes at the
map. So it definitely took some nudging getting it to do something more than
being AFK in base.

~~~
StavrosK
How did it figure that? There's pretty much no way to win if you're just
staying in base.

~~~
Shaanie
Barring faction imbalance, there's a 50% chance to win by staying in base.
However, I'm not sure how running around and dying could negatively impact
your win percentage unless the other bot is also outside of the base.

~~~
StavrosK
I don't understand what you mean. If you stay in base, the opponent won't, and
will quickly win the game.

~~~
tinco
This is during the training phase, the opponent will be a similarily trained
AI.

~~~
StavrosK
Ah, I see, thanks.

------
habitue
So an interesting thing you could do as a game company with an "unbeatable"
bot is to use it to balance a metagame. Let the bots all play each other and
tweak character stats etc until they win a proportional amount of the time.
(This presupposes that the bot learns the game the way OpenAI claims it is,
without needing to learn from player replays or from playing in a very
different way than human players do with tree search etc)

~~~
DannyDaemonic
I think balance is a bigger problem than people realize. It's much harder than
you'd think, and the most common way of balancing things is to simply make
them more "samey".

I was part of the Warcraft 3 beta. When it started all the races felt so
unique. It's a bit foggy now but I think, for example, all the elves buildings
- which appeared tree like - would automatically regenerate. Once they started
balancing the game, all these special traits fell away. All the races were
homogenized and despite still being different, all played much more similarly.
Honestly, it ruined the game for me. I feel if I had just picked it up upon
release I would have enjoyed it, but watching all these unique traits and play
styles fade away made me realize what could have been.

~~~
sondr3
This is luckily a problem Dota2 doesn't suffer from, we've been incredibly
lucky to have Icefrog be the main force behind it for over a decade. It's an
incredibly well balanced game, at this International (World Championships for
Dota2) 112 of 113 heroes has been picked and/or banned. The amount of
diversity you have in strategy is amazing, there is a meta obviously, but
teams have very individual styles and heroes they prioritize.

As far as I'm concerned Dota2 is probably the most well balanced competitive
video game out there.

~~~
NegatioN
It's also interesting to note that icefrogs balance strategy revolves around
strengthening strengths, and making initial weaknesses more pronounced. I
think this helps a lot with diversity.

------
UnpossibleJim
These types of MOBA games are a good precursor of military unit management. To
be honest, I'm not totally sure whether to be happy or sad about this sort of
thing. The strategic control of drone units on a combat field, without the
loss of personel (on the countries with this technology, anyways) should make
me happy, and does, to a point. BUT the potential for abuse (and, no, I
haven't even begun to extrapolate towards the Terminator, nightmare scenarios)
is rampant. Fewer people with a conscience on the battlefield or controlling
the apparatuses of war may or may not be worth the cost of the lives of young
men and women..... but I think wars should be fought by old men and women with
swords, anyway. If you're old and can look i to the face of your enemy while
you kill them, there's a better chance it's worth killing and dying for (by
the numbers, anyways).... plus, the President, Congress and the Senate have to
serve in combat positions, in my little fantasy scenerio =)

~~~
0xdada
DotA is not a MOBA and it's not a very good precursor for military unit
management. I'd say Starcraft is a better comparison, and they are working on
applying DeepMind to that right now [0].

[0] [https://techcrunch.com/2017/08/09/blizzard-and-deepmind-
turn...](https://techcrunch.com/2017/08/09/blizzard-and-deepmind-turn-
starcraft-ii-into-an-ai-research-lab/)

~~~
ylor
Dota definitely is a MOBA. Dota is _the_ MOBA from which all the others are
devised [0].

[0]:
[https://en.wikipedia.org/wiki/Multiplayer_online_battle_aren...](https://en.wikipedia.org/wiki/Multiplayer_online_battle_arena)

~~~
CrystalLangUser
Some people dislike that term. Many people in the dota community prefer ARTS
over MOBA. Though yes, technically it’s a MOBA; many other games could be
MOBAs because it’s fairly vague.

------
obastani
I think one of the biggest challenges in moving from 1v1 to 5v5 is the
substantially increased possibility of creative strategies. In a 1v1, there
isn't that much room for innovation, making it a perfect target for existing
AI techniques, which are good at learning what is "in the data" but have not
yet been shown to be capable of thinking outside the box. In contrast, in a
5v5 matchup, a much wider range of strategies is available. I'd imagine that
in this more flexible setting, AIs would be highly vulnerable to cheese
strategies, or in general strategies they have never seen before. Furthermore,
if there are any exploitable quirks in the AI (which seems quite likely, given
the established lack of robustness of neural nets), I have no doubt that
clever human players will be quick to exploit them. On the flip side, I think
that makes 5v5 a far more challenging and interesting goal that would be very
exciting if achieved!

As a side note: I think the most impressive aspect of this AI is that it was
trained without watching _any_ human games. In contrast, AlphaGo was
bootstrapped from a bunch of existing human games, which I imagine was crucial
to its training. Being able to learn how to fake out the human player without
having seen this action before is quite impressive.

~~~
electrograv
Without even getting into misconceptions and inaccuracies here, the following
quotes illustrate "the moving goalposts of AI": The almost-metaphysical
"mystique" that many attribute (perhaps subconsciously) to human intelligence
vs machine intelligence:

    
    
      > AI techniques, which are good at learning what is "in the data" 
      but have not yet been shown to be capable of thinking outside the box
    
      > AIs would be highly vulnerable
    
      > exploitable quirks in the AI
    
      > established lack of robustness of neural nets
    
      > clever human players will be quick to exploit them
    

Of course, it is important to identify pitfalls, so as to address them. But it
is sad that no matter how good AI gets, there will always be pessimistic
reasoning downplaying its success. With every step forward, there are the
predictions of impassible future failures -- "just around the corner"!

Due to this mystical "cleverness" attributed exclusively to humans, I'm not
convinced a majority of humans will ever allow machines to be considered
'intelligent', no matter how far the technology advances. I'd love to be
proven wrong, though.

~~~
flunhat
I will consider an A.I. truly _intelligent_ when it passes the Turing test
with flying colors. And as a native English speaker, too (not just barely
passing by posing as a 13 year old Ukrainian boy with a poor grasp of English,
which is what happened last time).

As far I know, no A.I. developed has ever met this bar. It seems that the
nuances, inconsistencies, and total freedom of language make it a totally
different kind of problem to solve than a simple game-playing algorithm.

Or maybe I'm wrong and getting an A.I. to pass the Turing test is just another
kind of game-playing algorithm with different parameters. At any rate, we're
quite a way off.

~~~
electrograv
Again, _you 're moving the goalposts forward whenever AI surpasses them_. This
time you move the goalposts forward by requiring a proof of native English
proficiency. Next time it will be something else; that goalpost will be
achieved, and when it is, you will move the goalpost forward yet again.

Perhaps you'll require the Turing+++ test be conducted with audio rather than
text. Then it will be video. Then a physical android. Then it will need to
prove itself better than the human in an IQ test. Then it will need to pass
various real world success tests, because IQ tests are too synthetic. But
those won't be good enough either, so it will have to live 80+ years as a
human in the real world without being discovered. But that's not enough,
because maybe its life was too easy -- the real human experience is
characterized by pain and suffering as well as joy, so we'll also need it to
somehow prove its experience is more real. And on and on and on. It's always
something else -- with every step forward, the success will be dismissed and
the goalpost will move forward.

I should say this isn't intrinsically a bad thing, since it pushes us to
advance the field of AI more and more. But IMO this is too often promoted as a
pessimistic criticism of the AI field, rather than an encouraging push of
optimism.

Why must this notion of binary "true intelligence" exist, anyway? Can't we see
that intelligence is a spectrum (most likely with multiple dimensions of
proficiency), and AI is continuously advancing upwards?

~~~
flunhat
The Turing test is hardly a new measure - it's been around for ~70 years and
hasn't been passed yet. But otherwise I agree with what you said - this is
just my personal measure of when we'll be in society-shaking territory.

~~~
pcnix
It _has_ been passed, but the definition of passed changes to make the test
tougher each time. I think that is what the other poster is talking about.

~~~
ralfd
Has it been passed? Turings original paper assumes adults conversing
intelligently. That is the spirit of the test. Not a clever hack of the rules,
like posing as a child which doesn't speak english.

This is the $20000 long bet from 2002 between Kapor and Kurzweil:

[http://longbets.org/1/#terms](http://longbets.org/1/#terms)

> Ray Kurzweil maintains that a computer (i.e., a machine intelligence) will
> pass the Turing test by 2029. Mitchell Kapor believes this will not happen.

------
kibwen
In the interest of a "fairer" comparison, I wonder how much of a difference it
would make to force the AI to simulate mouse/keyboard input and interpret the
raw screen buffer output, rather than using direct APIs into the game's guts,
to more faithfully emulate its human opponent. I'm guessing the peripheral
inputs wouldn't be much of a hurdle, but the image processing step could be
very interesting.

~~~
147
How do you know it's not doing that already? I was under the assumption that
open ai agents interacted with the environments through vnc.

~~~
Ambrosia
DOTA has a bot API
([https://developer.valvesoftware.com/wiki/Dota_Bot_Scripting](https://developer.valvesoftware.com/wiki/Dota_Bot_Scripting))

------
justicezyx
To be clear 1v1 mid lane is harder than 1 unit v 1 unit in sc. But it is at
least 3 levels behind 5v5 human play.

So first it's laining, which is showing in the game.

Then is the ganking basically 2-3 player working together.

Above that that's 5 men team fight.

Then you have the strategy planning in the band pick phase, and in game
movement coordination.

There are other things like item choices, in game communication etc.

I guess bots never tilt...

This 1v1 match is definitely a proof of the strength and maturity of modern
AI.

It will be extremely interesting to see if they can train a bot to achieve the
above intelligence. If so, I guess the game is pretty much losing a lot of its
appeal.

~~~
ronald_raygun
I'm not sure about DOTA, but in LOL there is also champion select which
happens before the game starts, where you try to pick/ban/steal champions to
get a good team comp and get the other team to have a bad team comp. It's lots
of know overall strategy, what your enemy likes, and some game theory.

You can lose the game before your summoner even hits the field.

~~~
deivid
Main difference is all Dota heroes are viable in competitive. Also, there's no
mirror match

~~~
needz
Only 3 heroes not picked in TI, iirc. It hasn't always been this diverse
though. This patch is particularly well balanced.

~~~
deivid
Picked heroes have been >80% since TI3. Compare that to LoL's ~50%

[https://www.reddit.com/r/leagueoflegends/comments/4n5og5/com...](https://www.reddit.com/r/leagueoflegends/comments/4n5og5/comparing_professional_lol_champion_pick/)

------
joefkelley
This will be HUGE for competitive Dota, even if they never take it further.

In much the same ways top chess players have learned from engines, I have to
think top Dota players will practice against and study the hell out of this
bot if OpenAI makes it available.

Not everything is replicable by humans... for instance I noticed it constantly
animation-canceling razes and only finishing it if it was going to hit; a
human will definitely mess this up. But other things can definitely be used.
It was positioning somewhat strangely, for example.

------
personjerry
It was streaming and it's over, so here's the VOD:
[https://www.youtube.com/watch?v=ac1getNs2P8](https://www.youtube.com/watch?v=ac1getNs2P8)

------
sondr3
Wow, that was not even close either. And it only takes it two weeks to reach
that level of play? OpenAI should create a team of five (as they said they
would) and let it play in the qualifiers next year.

~~~
hmate9
5v5 is much more complex than 1v1 gameplay, it's not as simple as taking the
current AI and making 5 of it.

~~~
ronald_raygun
I don't want to be "that guy", but I used to play a lot of LOL, and I feel
like 5v5 is in a different ballpark in terms of difficulty than a 1v1. Even
champion select is strategically intensive. TSM (one of the best US teams),
lost to an amateur team because they got out done in team selection.

[https://www.youtube.com/watch?v=g7l30-4A6WQ](https://www.youtube.com/watch?v=g7l30-4A6WQ)

~~~
hmate9
It is extremely difficult. 1v1 is like beating humans at chess. 5v5 is like
beating humans at Go. That's the kind of jump it is.

~~~
unrealhoang
I would even bring that to one more level of difficulty. There are so many
choices can be made in-game to gain advantages over enemy: item build,
gank/farm distribution, resource sharing, vision fight, choose to defense or
let go/trade of tower/objectives...

If we can have AI that beat Pro at 5v5 in Dota, it would very likely be an
AGI.

~~~
Tomminn
No. The value function for the AI will be way too specialized to look anything
like AGI.

------
jmkapz
I wonder if this a good move for Valve. How many people will be loose interest
from the game if next year OpenAI manages to pull off the 5v5 challenge, thus
bringing DotA 2 to the same level as GO and chess for AI, i.e. beating the
best human(s). Has there been any study on the popularity of chess after AI
moved in? How about prize pools, as a large driving force of DotA 2 is the
competitive scene? I suppose Valve will find out soon enough and it may be a
significant event to study of the impact of AI, if only by the scale (> 10
millions players [1]) and the traceability of the metrics. What if the numbers
do indeed plunge? Maybe such public display of AI might be given second
thoughts. And what of outside the gaming industry?

[1] [http://www.criticalhit.net/gaming/dota-2-vs-league-
legends-u...](http://www.criticalhit.net/gaming/dota-2-vs-league-legends-
updating-numbers/)

~~~
hmate9
Of course it's a good move. Millions watched AlphaGo, many of whom have never
played Go before. Millions, almost every single gamer will watch AI vs Dota
that will be amazing marketing for Valve.

~~~
jmkapz
I'm talking mid to long term here. To rephrase, is there a sizeable amount of
players who will stop playing, knowing that they cannot ever be the best?
Because when AI beats one, I don't believe there is a coming back. For the
concrete AlphaGo example, of course it was hugely popular but I would be
interested in the evolution on the number of players, how many of those
viewers started playing balanced by how many existing players lost interest
following the matches (excluding GO researchers :) ).

~~~
vecter
I don't think it will matter at all. Literally, zero impact.

Chess computers crush the world's best humans, yet here we are in meatspace,
still vying to win the human world chess championship.

Also, 99.99999% of players realize they'll never ever be close to being the
best in the world, yet they still play the game because it is inherently fun.

~~~
Noos
the problem is that players will use those bots against other players, just
like they do in online chess. The burden of reporting them will be too much
for online play probably. It's the ultimate cheat.

------
Tangokat
It is so weird reading all these comments. Almost half of them start out as
"it's impressive.. but". Is it human nature or are HN commenters just so
sceptical/negative of all the new tech?

One of the OpenAI guys mentioned that they could potentially use the same
technique in real life applications like surgery. Surgery is not "just" run on
a computer it has a physical component too. Is that really the next step or
was he just throwing out a random example people could understand?

~~~
thinkfurther
Why not ask them directly, if you actually want to _ask_ rather than make up?

For me it makes as much sense as comparing someone sewing pieces of clothes
together with someone pulling a zipper. Did people also have races against
cars all the time? Is anyone getting excited over some plastic not changing
texture when submerged in water for days or even years, while humans get
elephant skin rather quickly? I _do_ find all of that highly interesting, but
it's more a morbid curiosity than being amazed.

> One of the OpenAI guys mentioned that they could potentially use the same
> technique in real life applications like surgery.

Oh yeah, it will _all_ be for benefiting the elderly and the poor, I'm sure.
At some point, ever around the corner. It won't benefit the people who need to
have surgery in the first place because the so called civilized world can't
even deal with warmongers and power mad cops, can't reign in sheer greed and
sociopathy -- but nobody who has excuses today will still have them when faced
with fully automatic enforcement systems.

Generally, at least allow for the possibility that someone who is not utterly
fascinated by something you like might not be _less_ curious and progressive
than you, but the opposite, and that what you think is the bigger picture
being a fraction of what they see. At least until you actually asked the
people whose comments you don't like.

~~~
burkaman
> Why not ask them directly, if you actually want to ask rather than make up?

It would be annoying to ask the same question 10 different times in a thread,
I think a top level comment is better. Also, I didn't see any made up
explanation in the parent comment, it looks like a straightforward question to
me.

> the so called civilized world can't even deal with warmongers and power mad
> cops, can't reign in sheer greed and sociopathy

Can't this be used to shut down literally any conversation? It's obviously not
possible for any one innovation or idea to solve all these problems, so it's
not really that useful to point out. Would you dismiss any story that doesn't
say "war solved"?

> allow for the possibility that someone who is not utterly fascinated by
> something you like might not be less curious and progressive than you, but
> the opposite, and that what you think is the bigger picture being a fraction
> of what they see

Where are you getting this from? Nobody said this, who are you attacking? This
comment is so absurdly negative, and it seems completely unprovoked.

------
terda12
As a longtime DotA player and someone who's following the pro scene, this is
very impressive. Especially considering how it's beaten Sumail, widely
regarded as one of the best 1v1 players in the world. Can't wait to see what
OpenAI have in store a year from now for 5v5.

~~~
perishabledave
Sumail won once until they gave the bot insane creep blocking skills.

[https://twitter.com/Phillip_Aram/status/896162260455800832](https://twitter.com/Phillip_Aram/status/896162260455800832)

~~~
0x00000000
>The bot didn't recognize items on ground so he expended Mana picked up mango
then killed. So yes, he won, but it was more gimping the bot.

That's insane that he figured out how to beat it so quickly. I feel there are
other ways to cheese it too. Like maybe survive until 6 -> rush shadow amulet
-> smoke -> activate and walk into lane during fade time -> ult when the
wave/bot is on top of you. I bet the bot has never seen invisibility and
wouldn't know what to do

------
Analemma_
Not that this isn’t very cool, but 1v1 Dota isn’t anything like the full game,
it’s mostly a competition of who has better micro. If it can beat a team of
pros at 5v5– which is where the imperfect information, short-vs-long term
strategy and inter-agent communication challenges come into play— then I’ll be
impressed.

~~~
popinman322
Honestly, you don't even have to make all the AI heroes separate. We
automatically assume that each would be controlled by a separate virtual
player, but one virtual player could very well control all controllable units
on a team. That'd be interesting to watch.

------
taion
Is the "two weeks" of time in the Dota environment, or two weeks of training
time running in parallel as with A3C or something?

~~~
hhmc
From what the devs indicated on stream it sounded like processing hours (with
a ratio of around 300 in-game hours to 1 processing hour).

~~~
nopinsight
Interesting. That means it accumulates the equivalent of 24 * 14 * 300 =
100,800 hours of experience. That's about double the amount of practice one
gets for playing 10 hours a day for 14 years.

------
brandonhsiao
Is it taking in raw pixels or reading from game memory? How did they gather
data to train for this? (Did Valve give them an API?)

~~~
mIREdeRMedeFLaO
[https://developer.valvesoftware.com/wiki/Dota_Bot_Scripting](https://developer.valvesoftware.com/wiki/Dota_Bot_Scripting)

> Bot scripting in Dota is done via lua scripting. This is done at the server
> level, so there's no need to do things like examine screen pixels or
> simulate mouse clicks; instead scripts can query the game state and issue
> orders directly to units. Scripts have full access to all the entity
> locations, cooldowns, mana values, etc that a player on that team would
> expect to. The API is restricted such that scripts can't cheat -- units in
> FoW can't be queried, commands can't be issued to units the script doesn't
> control, etc.

~~~
hhmc
I wouldn't be surprised if they had access to a less restricted AI. Until we
see more concrete information I remain sceptical that they are only using the
official API.

~~~
mAritz
Why? Everything it needed for that 1v1 was available. There was nothing in the
FoW that needed to be known, it can keep timers about enemy cooldowns, gold
and xp (approximations at least since it's somewhat random).

~~~
hhmc
I'm not suggesting they had information that a human player wouldn't (i.e.
cheating), I just suspect that they may have access to a richer api.

~~~
jsmthrowaway
Which would give them what? The second clause of your sentence contradicts the
first.

~~~
hhmc
The statements are not contradictory - they can have a superior api which
doesn't permit 'cheating'. For example they could expose c++ api rather than a
lua api for performance concerns. Furthermore some of the api calls (e.g.
GetNearbyCreeps) have arbitrary restrictions.

------
nacs
Looks like its also being hosted on Twitch.tv in addition to Youtube:

[https://www.twitch.tv/dota2ti](https://www.twitch.tv/dota2ti)

~~~
justicezyx
Twitch is the official streaming partner for 7 years.

------
bhntr3
This was fun. It's our intern's last day on the ML infra team here. And he
happens to be a competitive collegiate DOTA player. We were all crowded around
watching the screen, shouting. Couldn't have planned a better send off. Nice
job, OpenAI!

~~~
popcorncolonel
As a ML researcher and an avid dota fan, I'm jealous! And it must have been
great with Dendi the legend there too.

------
Hroble
If I remember correctly, Deep Mind recently releasead a kit to train bots to
play Starcraft 2. Would anyone be able to compare the differences and
difficulty of playing dota and starcraft for a bot?

------
evc123
Elon posted these AI fear/regulation tweets right after he retweeted the
opeanai dota 2 blog post:

[https://twitter.com/elonmusk/status/896166762361704450](https://twitter.com/elonmusk/status/896166762361704450)

[https://twitter.com/elonmusk/status/896169801277517824](https://twitter.com/elonmusk/status/896169801277517824)

------
rafinha
All these fancy stuff and valve still can't detect people feeding chicken...

------
ted12345
I wonder if some of the things the guy says are misleading.

In the video, we see the bot "creep blocking." For those unfamiliar with dota,
players can use the model of the unit they control to obstruct the movement of
allied computer controlled units in order to gain a favorable position.

I suppose it's possible that over millions and millions of matches played
against itself, the OpenAI bot "invented" this behavior for itself. But it
seems more likely to me that the programmers "built that behavior in."

~~~
popcorncolonel
It would be pretty much impossible for the programmers to "build the behavior
in" to the neural network, unless you mean training on supervised data or
something.

~~~
visarga
It's not impossible, it's called inverse reinforcement learning, where they
learn a value function from an external demonstration. Then they use this
value function for teaching the bot an action policy. Intuitively, the idea is
to learn first what are a good state and a bad state, based on external
demonstrations, then use that to teach the bot how to act.

This kind of learning is similar to GANs, where the discriminator learns from
real data and the generator learns from the discriminator.

~~~
popcorncolonel
Very interesting! Thanks for sharing -- I'll look more into this.

------
demonshalo
MEH. I am not impressed. Restricting the number of parameters and variables in
order to produce a bot that can do 1 sub-set of tasks really well is nothing
special imo.

Complexity and simulations of what you have not yet encountered is something
humans can do with ease. This is not something a bot can do in a complex
environment like DOTA.

I'll be the first to admit that I was wrong and that AI is truly a thing once
I see a bot like this one beat a pro-team in a 5v5. Until then, meh...

~~~
criloz2
is an improvement against the other bots, it can be really good for training
players, I would love to test it my self, but yeah dota is biggest that this,
not only 5 v5, also 100+ heroes, and professional player can play a variety of
heroes on mid with different matchups and its nuisances, the bot still need it
a lot to even dominate 1v1.

------
kahlonel
Even with a lot of restrictions, I can't imagine how much variables this bot
has to take into account; and then generate the output within a sub-
millisecond time period. The most interesting part was aggressive positioning
of the bot, and faking the spells to scare the shit out of Dendi (the pro
player competing against it). Will definitely follow the progress of this
project.

------
bitmapbrother
I really don't think this is a big accomplishment. Dota 2 is a team game where
5 players all work together. I went in thinking I would see a 5v5 against
bots. They promised a 5v5 against bots next year so we'll see how that plays
out.

------
Asdfbla
I wonder what the input for the bot was. Just a screengrab like in other
reinforcement learning examples (like Pong or GTA etc.) or did they use some
Dota API to make their lives a bit more easy?

Impressive nonetheless, like others said, knowing the reward function would be
interesting.

------
kmnc
How do things like reaction time, and actions per second work with something
like this? Is it just an assumed advantage the ai gets, or does it simulate
the limitations of a human? If it doesn't, how big of an advantage is it in an
ai versus human competition?

~~~
mAritz
That was pretty much the entire reason it won. It did some standard high-level
1v1 laning techniques, but if human "conditions"[1] were implemented it would
look a lot different.

Just the blocking of the lane creeps at the start is already super-human and
gives the bot a huge advantage.

[1] like lower actions per minute, latency, occasionally missclicking and not
being 100% certain about distances

------
antouank
Seems like the AI was beaten at least 50 times.

[https://twitter.com/riningear/status/896297256550252545](https://twitter.com/riningear/status/896297256550252545)

------
eduren
This is really cool outreach on OpenAI's part. So many young people watch The
International and I'm guessing that at least a few of them are more interested
in CS and STEM from seeing this.

------
VHRanger
Where's the paper describing the exact methods used?

------
thatsadude
I love both Dota and ML, this is awesome. I would love to know whether they
release the source code for us to play around.

~~~
popcorncolonel
It is _Open_ AI, after all.

------
exabrial
OT: Is backdooring prevented by the game engine yet? Annoyed me that a useful
tactic was "against the rules"

~~~
banhfun
It's always been prevented in Dota 2. Buildings have backdoor protection,
which makes them regenerate lost HP from recent attacks unless there are
creeps nearby.

~~~
ryanlol
Not really "prevented", backdoor protection only provides 90HP/s.

~~~
Smaug123
Backdoor protection also gives a substantial damage reduction - 25%, and much
more against illusions.

------
bronz
this whole emphasis that openai and deepmind put on human/robot collaboration
is a paper thin pr move in my opinion. the robots will be better than any
human, humans will not be able to contribute a single thing soon. but they try
to make us all feel safe by making it look like they benefit from our brains.

~~~
cglouch
I know it's not the same type of AI, but in chess there's a whole scene for
computer + human play. A chess engine on its own can have trouble seeing
strategic ideas that humans can recognize (e.g. opposite-colored bishop
endgames, certain closed positions, and fortresses) so an engine on its own
will lose to that engine being assisted by a skilled human player. In other
words, humans are still capable for contributing at least a little. That said,
it's not much - I think a chess GM paired with an engine will probably only be
able to beat an engine rated ~100 or so points higher than their own.

It will be interesting for deep learning, though, where the ideas are a bit
more abstract. Perhaps humans will be useful for a while longer.

------
xfer
Very impressive, even if there are some limitations. I look forward to more
progress for a team of bots.

------
savethefuture
Well the presentation was a little meh but very impressive tech they built,
excited to see the 5v5.

------
ematvey
To OpenAI folks: are you planning to publish a paper with implementation
details?

------
justicezyx
When will the match happen or did it already happen? I did not find any vods.

~~~
justicezyx
OK it's happening right now, I saw machine the host's announcement

~~~
BLanen
Weird website... No indication that a match is happening/happened/ or going to
happen or with times or not.

I guess I missed it and the stream link is now just a general link to the TI
stream, which confused me for a bit.

~~~
justicezyx
They probably dont expect the dota tournament to have such a large influence
scope...

------
cissou
It looks like it's over…? Someone has a replay of the relevant part(s)?

~~~
cissou
[https://www.youtube.com/watch?v=ac1getNs2P8](https://www.youtube.com/watch?v=ac1getNs2P8)

------
wnevets
As a long time dota2 player, it was simply absurd just how good it was.

------
csomar
Is this the start of the end of online gaming? If bots outperform humans, what
to prevent someone from a running bots for online games (even Poker and Chess)
and being sure that everyone loses?

~~~
simonebrunozzi
Or, quite the opposite.

I played DotA a few dozen hours, mostly against the computer, to learn. Then I
played only a few matches against people, but I generally didn't like them.

Why? Because there were expectations that I would know certain things, or that
I would perform at a certain level, and the other players were, on average, a
bunch of young kids with tons of time to play DotA and their own weird jargon
and acronyms for me.

I would strongly prefer to play against bots only, and be able to adjust the
difficulty, to make the game always challenging and interesting as I progress
in skill level. It could also make the "competitor" in me happy by knowing
what my (objective) ranking would be.

------
vfistri2
Is there a research paper on this?

------
donovanm
this is pretty awesome, hopefully they'll release some more details about how
it was implemented

------
Hekatron
Figuring teamplay out is going to be a lot more complicated but a strong 1v1
bot is already very impressive. Props to openAI, that universe didn't take off
was a bummer, it was a pretty cool project as well.

------
Macrosmatic
Anyone else interested in seeing this bot Vs itself?

------
nether
No!

