
Hype or Not? Some Perspective on OpenAI’s DotA 2 Bot - dennybritz
http://www.wildml.com/2017/08/hype-or-not-some-perspective-on-openais-dota-2-bot/
======
dvt
Pretty much agree with everything in here. As I said in my earlier posting
(and this blog post reiterates), a 1v1 Shadowfiend mid is highly technical and
does not require a huge search space (like in Go or Chess) or any judgment;
all it takes is a few tactics (e.g. creep blocking) and good aim for the
razes.

Also, the bot was already beaten 50+ times[1]. There are at least 3 strategies
that work. It just goes to show how primitive AI is, as it took the AI team
thousands of generations to get it to this stage, but a few determined gamers
outsmarted it (using a few cheap meta-strategies) it in less than 6 hours
after release.

[1]
[https://www.reddit.com/r/DotA2/comments/6t8qvs/openai_bots_w...](https://www.reddit.com/r/DotA2/comments/6t8qvs/openai_bots_were_defeated_atleast_50_times/)

~~~
teej
What hero do you think would be a better test?

~~~
dvt
Either, (a) high risk/reward heroes: Pudge, Huskar, maybe Weaver or Puck, or
(b) highly complex heroes: Invoker, Morphling, Earth Spirit, Nature's Prophet,
Techies, maybe Meepo.

~~~
unrealhoang
Would love to see courier snipe from lv 1 by NP or even man fight in the base

------
AndrewKemendo
_We did not make sudden progress in AI because our algorithms are so smart –
it worked because our researchers are smart about setting up the problem in
just the right way to work around the limitations of current techniques._

This statement is like putting wheels and a motor at the base of the
goalposts.

Everyone who practices ML knows the reality that while we're not going to see
AGI for a while, and these systems are massively hard to build and do very
narrow bounded things, they are also making massive progress in "intelligent"
outputs at a pace we've never seen.

Yes, there is hype, but there are pretty solid reasons to be hyped.

We'll keep seeing people saying oh well it's not _that impressive_ probably
until AGI has clearly taken everyone's job in 2100 and we're all just
providing training data for it.

~~~
dennybritz
I agree. The point I was trying to make is not whether it is impressive or not
(it is impressive!), but that the general press hype about "AI breakthroughs
that will soon kill us all and need regulation" is over the top and
misleading. The probably massive engineering effort that went into this is, to
me, more impressive than the algorithmic innovation, if any.

Progress is made with small incremental improvements, including this one, and
there have been few real algorithmic "breakthroughts" over the past few years.
That's why I think it is important to give some perspective to the hype.

~~~
sweden
I feel that you are missing the whole point of "AI breakthroughs that will
soon kill us all and need regulation". It's not about AI becoming self-aware
and start to proactively taking over humanity starting from a Dota game, it's
how we are finally capable of putting AI anywhere.

The point Elon Musk was trying to show is that nowadays, we have the
technology and the research to replace humans by AI for making judgement
calls, no matter how difficult it is. And this was proven with a "simple" game
of Dota. And if we are able to build a system to play Dota, we are also able
to build AIs for anything at all.

You claim that there had not been any real breakthroughs over the past years
but truth to be told, today we have AI playing Go, AI managing self driving
systems, AI playing games of Dota against the top players. All of this
happened over the last few years, 5 years ago this all felt like a distant
future.

~~~
debatem1
"nowadays, we have the technology and the research to replace humans by AI for
making judgement calls, no matter how difficult it is"

"if we are able to build a system to play Dota, we are also able to build AIs
for anything at all"

These are stated like facts, but are not facts.

There are decisions made by humans which no AI system today can make well, and
there are problems for which we have no idea how to build even mediocre AIs.
The claim that someday we will have AGI is plausible but not yet certain to be
true.

~~~
sweden
"There are decisions made by humans which no AI system today can make well,
(...)"

Exactly, that is precisely the point. Companies and business will just rush to
implement the next big neural network to boost their business, no matter how
immature the technology is.

And the claim is not about AGI, it's more about the little things. For some
reason people like to think really big and exaggerated scenarios when it comes
to AI.

Let's take web apps as an example. We can all agree that the state of the art
of the current web programming is very poor: JavaScript, NodeJS, Electron,
CSS, etc. The technologies are bloated, they are slow, full of hacks and
workarounds, and so on. And yet... people use them for everything, it's like
the Atwood's law described: "any application that can be written in
JavaScript, will eventually be written in JavaScript"

I imagine that a similar scenario will happen with AI and neural networks. Can
you imagine, for example, start dealing with an AI instead of an human when it
comes to customer service? Maybe it is already happening, if you look for
stories about Google's customer service in the internet, you would think
everything is run by some sort of AI there.

Another example, look at Microsoft is doing with Visual Studio's telemetry
data: [https://blogs.msdn.microsoft.com/dotnet/2017/07/21/what-
weve...](https://blogs.msdn.microsoft.com/dotnet/2017/07/21/what-weve-learned-
from-net-core-sdk-telemetry/)

I wouldn't be surprised if they built a neural network to feed all that data
to and to have it to produce a UX "optimized" for mass consumption.

We are getting into a time in which everything will be powered by "AI" and I
think that's what OpenAI is trying to regulate before we get every single
business pestered with an half-assed implementation of neural network and data
analytics.

------
nbkvjones
there was a "discussion" on nadota.com about the bot and the semipro player
that openai used to test chimed in.

apparently the set of items the bot chose to purchase from was limited[1] and
recommended by the semipro tester. As someone who knows next to nothing about
ai, my question is this: the bot was announced on stage as blank slate, dumped
into dota, and built entirely from grinding countless games against itself; is
it reasonable to pitch it this way while having this item constraint from an
outside source? I also wonder what else was recommended by the tester, and
then constrained.

the "discussion" is linked below and the tester is the user sammyboy. Here's a
warning though: nadota is 99% trolling, hate, idiocy, and garbage.

[1] [http://nadota.com/showthread.php?41718-terrifying-1v1-mid-
AI...](http://nadota.com/showthread.php?41718-terrifying-1v1-mid-AI-dumpsters-
suma1l-arteezy-with-ease&p=1807415&viewfull=1#post1807415)

~~~
gdb
(I work at OpenAI.)

We'll have another blog post coming in the next few days. But as a sneak peek:
we use self-play to learn everything that depends on an interaction with the
opponent. Didn't need to with those that don't (e.g. fixed item builds,
separately learned creep block).

~~~
hayd
So it's not exclusively unsupervised self-play? This contradicts how it was
portrayed during the game/afterwards.

~~~
gdb
See the opening video
([https://www.youtube.com/watch?v=wiOopO9jTZw&t=268](https://www.youtube.com/watch?v=wiOopO9jTZw&t=268)),
where I mention that we "coach" it on what we think is good and bad. All the
heavy lifting is self-play (which I would characterize as supervised).

------
candiodari
> Nobody likes being regulated, but everything (cars, planes, food, drugs,
> etc) that's a danger to the public is regulated. AI should be too.

I hate that people actually see things this way. Regulation to prevent AIs
from taking over the world will never happen, because nation states won't
cooperate on such rules [1]. Additionally you can't catch people using AIs to
determine their actions.

BUT what regulation can do is prevent people from competing with a few of
Larry Page's and Elon Musk's businesses.

[1] [https://www.rt.com/news/395375-kalashnikov-automated-
neural-...](https://www.rt.com/news/395375-kalashnikov-automated-neural-
network-gun/)

~~~
sweden
The "taking of the world" scenario is a too narrow view of the whole
complexity of AI regulation.

There are other more realistic and important scenarios that need to be
addressed, for example: AI's default behaviour when challenged with a
possibility of a car accident on a self driving system. Should the car try to
avoid the accident by jumping of a cliff, sacrificing the life of the driver
and of the people inside the car, or should it hit the next car making the
accident worse for the other person but potentially saving the life of the
people inside the car?

~~~
mizzao
You must be thinking of
[http://moralmachine.mit.edu/](http://moralmachine.mit.edu/)

But in practice, this is more a study of people's ethical tendencies around
the world than a practical way to address AI decision-making in bad
situations.

------
Funnnny
It worths noting that, shortly after they offer SF arcana (extremely rare
item), 50 people did go and beat the bot on the spot.

~~~
yskchu
The strategy that was used, seems to be surprise the AI and mess up it's
decision tree:

The general strategy is to win by claiming first tower. At 0:00, you aggro the
enemy creep wave so that they start following you. Then you walk around in a
circle around the jungle, and the enemy wave will start to form a congo line
that will follow you around. You then path around the jungle so that on the
next wave spawn, you can aggro the wave again and continue to walk around in
circles. The AI will burn glyph when your creep wave hits the tower, and for
some reason it can't really decide between chasing you or defending the tower.
So after about 5 minutes of doing this, your creep waves will eventually
destroy the tower and you win the 1v1.

From
[https://www.reddit.com/r/DotA2/comments/6t8qvs/openai_bots_w...](https://www.reddit.com/r/DotA2/comments/6t8qvs/openai_bots_were_defeated_atleast_50_times/dliundl/)

~~~
duskwuff
This is, admittedly, a weird enough strategy that a human player would
probably get flustered and respond with suboptimal choices, just like the AI
did. A human would probably adapt to the situation more quickly, though.

Heck, I wouldn't be surprised if a few people are trying this strategy in pub
games now, just for shits and giggles. :)

------
itchyjunk
Is no one upset that they claimed `learned from self play only` when clearly
it isn't? Creep blocking ? really? it learned a left over feature in original
DOTA from warcraft in a new stand alone DOTA 2 client strictly on self play?
And that distinct animation canceling the bot does? Look at amateur players
and pro players (pro players do it more often then actually needed to `warm
up` the muscles, similar to extra key strokes in SC). I wouldn't be surprised
if the bot was trained in tons of pro openings. :/

Why not be clear about what has been done? Deepmind has said they do
supervised learning first and other stuff on top of that. My guess is
something similar to that happened.

------
jarsin
All the bots have to do is spew tons of toxic crap in chat.

Then they will be like any other real life DOTA player.

~~~
perishabledave
I think the root of that is the high reliance on teammates and the level of
commitment each game takes. Unlike League there is no early surrender, so if
you’re in a rough game you have to endure the full duration which can last
around an hour. If you feel like it’s your teammate, who is probably randomly
paired with you, that is failing it’s easy to get frustrated. This is one of
the many things that make Dota either extremely frustrating or extremely
satisfying.

------
oldstrangers
Surely the author knows that neither Chess nor Go have been "solved". Qoutes
or no qoutes, it's still very inaccurate.

I'd also argue that chess and go are both vastly more difficult problem sets.
We literally do not have the computational power to solve a game of chess and
it's projected that we won't for another 50-100 years.

------
darod
While the author is probably right and this is no huge breakthrough in AI/ML,
it is yet another example of AI/ML being able to do an activity that surpasses
a human's ability. I am still waiting for an example of how AI/ML will
complement a human's life as opposed to demonstrating an area where a human
can be replaced.

~~~
JSONwebtoken
How can you not see applications where replacing humans complements another
human's life? I'm sure if you thought about it, you could see how self driving
cars, real time translation, image to text for visually impaired, could be of
some benefit to peoples lives? These applications might be automating a human
out of a job, but they are also far superior at their narrow task.

You can argue the total societal value net the societal cost of putting some
people out of jobs, but saying you can't think of any application where AI can
complement peoples lives is being intentionally hyperbolic and is a bad start
to a discussion.

~~~
darod
In this application who does this AI serve? A self driving car doesn't help me
to become a better driver. It removes me from the driving equation all
together.

------
musashizak
There is top hype in ai but also in neuroscience. Actually there aren't
scientific evidences that mind and coscience are materiale and born from the
brain. Also the emotions are really important in the logic and thinking
process. So without coscience and enotions we can't have real think on a
machine.

------
craigsmansion
I hope someone can clarify.

What are the definitions of AI and game complexity in this field?

These all sound like very exiting developments. As I read about them a lot of
times games such as Dota and Starcraft are touted as more complex than Chess
or Go, but--at least with Starcraft, the AIs are limited in their number of
actions to level the playing field. Isn't that like claiming humans can run
faster than greyhounds, provided that the greyhounds only get to use two legs?
Or maybe claiming that humans are better at chess when computers are
restricted to the maximum human ply depth?

I also noticed a claim--again, in a Starcraft related article--that the AIs
previously couldn't beat the build-in AIs (the computer players). What type of
AIs are considered as challengers here? Only blank-slate self learning AIs?

~~~
Tangokat
The reason why the AI is limited in number of actions is because the strategy
is interesting, not so much the mechanical skill (think aimbots in FPS games).
We know the AI will be better at clicking and moving units and would be
capable of moving every unit optimally at all times. But is it also capable of
coming up with creative strategies that humans could use as well?

I think both situations would be interesting to be honest. Have an
unrestricted AI and a restricted AI - using ML they would probably develop
vastly different ways of playing.

------
DSrcl
A lot of dota's mechanics is designed with the assumption that the player is
human -- e.g. skills that can be programmed to be released perfectly but are
hard for a human (even pro) to do so reliably (Shadow Fiend's raze is one of
them).

------
aorth
I'm still just trying to figure out what "Dota" stands for. Is it an acronym?
Neither the Valve website or Wikipedia clarify this!

~~~
natural219
Funny story. The original term "DOTA" was an acronym, for "Defense of the
Ancients", which was the original Warcraft 3 mod of which the later game,
"Dota 2", was based. Since the original DOTA was a community mod using assets
from a commercial (IE, trademarked) Blizzard property, it was in sort of a
gray area in regards to re-commercialization of what was essentially the same
gameplay and characters. To avoid further standing for Blizzard to reclaim its
trademarked properties, Valve registered the trademark "Dota" specifically as
a word that doesn't refer to anything, officially eliminating the acronym from
the name and somehow distancing the new engine from trouble with the
trademark.

------
JonathanLIabc
I see Elon Musk tweets as a warning about the _potential_ of AI, not a hype of
AI nor the current stage of AI.

The most impressive part to me is that the bots are _self-learned_. On the
other hand, AlphaGo is supervisored. They are different (not to say which one
is better).

------
colordrops
Big assumptions were made by the author of this post, the biggest being that
they used an API to get access to game data rather than pixels. If the AI were
limited to pixels then the achievement is much greater.

~~~
dennybritz
Author here. I agree this is an assumption, but based on my experience it is
_very_ unlikely that this is trained on pixels. Training would've been orders
of magnitude more expensive. If it really is trained on pixel input I would be
shocked and extremely impressed, and parts of post would not apply.

------
velobro
The bot did not have their creep blocks hard coded. In the event the player
purposefully messed up his block to see if the bot would respond and it did

------
zaroth
Where do bots go to fight other bots in millions of games, and algorithms
compete for superiority? I assume there must be an ongoing "marketplace" to
match bots and run the simulations.

~~~
astrojams
They play themselves. No need to play other bots.

~~~
zaroth
Is that a fact? You don't ever see better performance by training in a
heterogeneous environment? Or just, no one does it this way?

