
Human-level performance in 3D multiplayer games with population-based RL - ajay-d
https://science.sciencemag.org/content/364/6443/859
======
d--b
Do they handicap the AI to have sub-perfect aiming?

3D shooters and Quake in particular are aiming games above. If you aim
perfectly, you will always win against humans.

Adding randomness to aiming is the way to provide fair comparison to give any
conclusion regarding true AI features.

~~~
HanayamaTriplet
The ML-based AI (or "agent" as the authors referred to it) in the article
never achieved perfect aiming, only up to 80% at close range and 0.5% at long
range. After artificially delaying reaction time, a team of one strong human
plus one delayed agent was only able to win against a team of two delayed
agents only 21% of the time. While this only controlled for reaction time and
not accuracy, the authors note that the human and delayed agents had similar
numbers of hits per game. Meanwhile, the agents had 4-5x more flag captures
than humans, leading them to believe that it had developed superior strategy
which carried the games even without god-like aim.

~~~
de_watcher
> it had developed superior strategy

The problem is that when it starts to play against humans the humans develop
an anti-strategy after just a several matches. Feeding that back into AI
doesn't immediately give enough data points for training.

------
isseu
I feel problem definition is underappreciated in current ml world and the
effort is put into the ml methods. People has been able to tackle almost any
problem well defined but these are usually simple in comparison to true ai so
we still get "stupid" agents. What problem is the test to true ai?

~~~
yodon
This paper would seem to be a fairly broad based and complex set of
capabilities to learn and accomplish, including both collaboration and
exceeding the performance of high skill humans at a highly skill based
activity. Is there something you feel is missing from the task in this paper?

~~~
avsteele
How much would the agent's skill level decline, and how much training would be
required to recover if, for example, the game were reskinned? How does the
skill in this one game mode translate into others?

Stating the critique more directly: in what way does the expertise
demonstrated mirror the kind of expertise possessed by a similar level human
ELO player?

If would seem a really shallow kind of expertise, if it didn't translate into
some competence in very similar games.

~~~
yodon
The article discusses a wide range of what constitute significant conceptual
mappings discovered autonomously. This work was trained on a single game with
a single art direction. The AI can generalize to new maps it has not seen
before within that style of art direction, including reliably beating expert
human players on maps that are new to both human and AI. That is a significant
generalization. The technique should be generalizable to other styles of
skinning or different art direction in other games, but there is no way for
the the AI to have learned that yet in this experiment because the AI was only
ever exposed to a single game here. Over time this technique undoubtedly will
be extended to multiple games with different styles of art direction, the
challenge there being less on the AI/machine learning side and more on the
effort required to hook up multiple games for use in this type of training
environment.

------
dontreact
What is deepmind going to do once they solve all video games? It seems like
they are leveraging the massive economic demand for entertaining simulations
of the world. But once they hit the limit of that they will have to make their
own simulations.

~~~
jimmy_dean
Deepmind can always transfer learning to the robotics industry, if they
haven't already. There's bound to be a space they can fill with the progress
they've made in these video game reinforcement learning tasks. Maybe aiding in
medical discoveries?

~~~
dontreact
The sim to real gap is very large and it honestly seems to me like maybe most
of the work is not in the RL part of it but rather instrumenting the robot
properly, setting up the right simulation, figuring out how to scale up the
real world learning. Haven't seen much robotics stuff come out of Deepmind
yet, maybe for these reasons.

~~~
sjg007
You need algorithms that can learn complex tasks and also reason under
uncertainty. Not just uncertainty of sensor inputs but also uncertainty of
state. The goal of all of these video games is to develop the algorithms over
a wide range of difficulty. So algorithms that have memory, learn / plan and
are reasonably robust etc...

