
Pac-Man recreated with a GAN trained on 50k game episodes - doener
https://blogs.nvidia.com/blog/2020/05/22/gamegan-research-pacman-anniversary/
======
lumost
I've observed an increasing trend with modern AI research that

A) It's no longer practical to incorporate into real products given the
hardware costs for training _and_ inference. In NLP, large transformer based
models can easily occupy a 5 thousand dollar GPU at a paltry 50-60 words per
second processing pace.

B) The Research often demonstrates things that existing non-ML toolchains were
already reasonable at doing with a better result, and a faster iteration time.

This demo looks like it's replicated the core mechanics of pac-man with
_worse_ graphics, 3-4 orders of magnitude higher resource consumption, and
undoubtably many _odd_ bugs hidden away from the pretty demo. It even needed
the game to exist in the first place! While this is fascinating from the
technical and academic perspective, it's hard to be convinced that any gains
we see in the field won't be offset by increased hardware/latency costs from a
product standpoint.

~~~
YeGoblynQueenne
Or, to paraphrase: great paper, no application.

~~~
SQueeeeeL
It really is a great paper, the problem is when your entire field has few real
applications but everyone is making too much money to admit that. Cool tech
demos like this is awesome as a blog post, but not sure why it's academic...

~~~
esoterica
I don't know where you got the idea that the purpose of an academic paper is
to produce the schematics for a working commercial product. The fact that it's
novel and interesting is sufficient grounds for producing a paper about it.
Why do you think projects like these are solely the domain of blog posts?

~~~
YeGoblynQueenne
That is not true for academic papers in computer science and particularly in
machine learning were contributions are expected to have some sort of
practical application that is at least possible to foresee.

Indeed, a common criticism against papers in machine learning is "what is a
real-world application of this approach?". If you can't answer that
convincingly then it's unlikely your paper gets published, particularly if it
has rather mediocre results otherwise, as is the case with this paper (see my
earlier comment about the comparison to the two other systems which is a bit
of an anticlimax).

In fact, the paper (well, the preprint version) makes a clear attempt to
motivate the work with a practical application, saying that it aims to "learn
a simulator [for robotic agents] by simply watching an agent interact with an
environment".

~~~
esoterica
> That is not true for academic papers in computer science and particularly in
> machine learning were contributions are expected to have some sort of
> practical application that is at least possible to foresee.

This is 100% not true for computer science and I don't know why you think it's
true. There's are entire branches of computer science (e.g. complexity theory)
that are highly theoretical and are certainly not geared towards immediate
practical application.

Even in more applied branches of computer science like ML there are many
papers that are geared more towards the fundamental research side of things.

> saying that it aims to "learn a simulator [for robotic agents] by simply
> watching an agent interact with an environment".

Saying that a paper could lead to a useful application down the line is not
the same that saying that the paper is intended to describe the schematics for
a specific useful application.

~~~
YeGoblynQueenne
Well of course, it's not 100% true- who said anything about 100%? There are
theoretical branches of computer science; but most of computer science is not
purely theoretical. Certainly machine learning (and more broadly speaking,
much of AI) is like I say.

I'm not sure who brought up a schematic? The bit I quoted is taken from the
paper and it's a clear attempt to motivate the paper's approach as having a
practical application. To clarify, do you disagree with that?

------
QuixoticQuibit
This research reminds me of one of their April Fool’s videos [1] with a gaming
USB stick that you plug into your computer to offer various gaming
enhancements. One of which is a “Ghostplay” feature where an AI program can
mimic you after training on your gameplay.

I remember that video because it always struck me as not being far off from
reality. It’ll make the current set of online game cheats/hacks seem like
child’s play if it can be done effectively (read: human-like and
undetectable). Maybe then a server-running AI will be needed to detect AI-like
gameplay mimicry.

[1] [https://youtu.be/smM-Wdk2RLQ](https://youtu.be/smM-Wdk2RLQ)

~~~
Ajedi32
But if the AI is functionally indistinguishable from a human player (human-
like and undetectable), does it even matter really?
[https://xkcd.com/810/](https://xkcd.com/810/)

(Conversely, if they're _not_ functionally indistinguishable from a human
player, then that means it's possible to write an AI or other algorithm to
detect them.)

~~~
sandworm101
>> does it even matter really?

Because, even if acting like a human, AI can play/fight 24/7\. Many online
games still have leader boards that can be climbed through steady play rather
than short-term bursts of skill.

~~~
indeyets
One more reason to fix those games, I guess.

------
CShorten
I made a video explaining the details of this paper if you are interested!
[https://youtu.be/H8F6J7mYyz0](https://youtu.be/H8F6J7mYyz0)

~~~
bjourne
Very interesting video! How does the GameGAN image discriminator work? I don't
know so much about GANs but it seem to me that classifying PacMan images as
real or fake would be a very hard problem because of the low resolution.

Suppose PacMan is in a corridor and there is a dot to the left of him and
three to the right: ".P..." That is plausible because PacMan hasn't eaten any
dots yet. But if the second dot to the right is missing: ".P._." something is
wrong because the first dot to the right is still there and PacMan can't jump
over dots. It must be very hard to get the discriminator to understand that
the second image is fake.

~~~
CShorten
Thanks for watching bjourne! There are three different discriminators, the
first does realism for the game, the second does action-conditioned
discrimination (making sure the generator takes in the action for its next
frame prediction), and the third is a temporal realism with a 3D CNN.

Interesting scenario, it would be a cool experiment to walk through the latent
space and try to find that kind of frame in the GAN latent space. I'm sure you
could find it, but the question might be how far off the generated data's
manifold is it.

------
itchyjunk
Thought experiment: Is my brain also generating reality without a physics
engine and is filling in details that it thinks should be there? If true, what
are the advantages/disadvantages of it?

~~~
coldpie
Yes, absolutely. You can catch a baseball because your brain predicts where it
will go before your eyes see its actual movement. This may be a good place to
start learning more:
[https://en.wikipedia.org/wiki/Direct_and_indirect_realism](https://en.wikipedia.org/wiki/Direct_and_indirect_realism)

~~~
lonelappde
How is that "not a physics engine"?

~~~
jcims
It's more general than a physics engine in that it's predicting an expected
outcome...it just so happens that the outcome here is largely dictated by
physics. (At some level they all are but some more obviously than others)

------
andr
It would be super interesting to see if this knowledge generalizes with
transfer learning. For example, after seeing 50,000 episodes of PacMan, would
the GAN be able to recreate Space Invaders with just 5,000 extra episodes?

------
flixic
Interesting to explore how this can be used. You still need to develop a game
with an actual engine first, to generate all the samples for training the AI
how to generate frames without an engine.

But maybe if training sample size can be considerably reduced, game making can
open up to far less technical people. Instead of developing with an engine,
game makers could show the AI "when I press this button, change the scene like
this", and the AI could try to figure out the rules from given samples.

~~~
jcims
Seems like it's basically simulating an environment which it has agency to
affect. To the extent that games are simulating some universe, it seems like
this could also be applicable for simulating at least one universe that has a
game engine we can't quite reproduce yet.

------
dougabug
The point is this brings us closer to demonstrating AI which is in some sense
grounded in reality, as opposed to some kind of “Clever Hans” or Searle’s
“Chinese Box.” The network has learned how to “imagine playing out a scenario”
much like a human player might use his own mental model of the game to
consider various strategies to attempt, perhaps with the real game.

That the network apparently has a mental model which is rich enough to
simulate actual game play implies that it hasn’t just fit a static
caricurature of the game which immediately breaks on an unseen move. That is,
it has a non-trivial “conceptualization” of its environment which goes beyond
simple pattern matching or nearest neighbor matching.

------
ekianjo
> Trained on 50,000 episodes of the game

Meanwhile an actual human will probably need only about 5 games to fully
understand how the game works and reacts.

~~~
ttul
Fundamentally, this is because we have already been trained on years of living
in and reacting to the real world, which, to some abstract degree, the game
mimicks. The Nvidia model starts with nothing.

~~~
amelius
So you are saying that if you first trained it on a different game with
similar real-world mechanics, then training it on this game would require far
fewer iterations?

~~~
chongli
Pac-Man isn't a purely abstract game, it's loaded with human cultural
references. Pac-Man himself is a creature who loves eating food: mostly white
pellets but also various kinds of fruit and of course the iconic power
pellets. Fruit of course is something humans immediately identify as something
good to eat and in the game they're rewarded with points which are denoted by
numbers on the screen that grow larger.

It almost needn't be said that human culture puts a great deal of emphasis on
making numbers grow larger, particularly those associated with personal wealth
and economic growth. To a computer without this context, why should there be
any reason to prefer larger numbers over smaller ones? Heck, if you're going
all the way and denying the AI direct access to the game's memory, then it's
got to learn to recognize the digits first before it can even begin to decide
whether it's good to increase them.

Even more abstractly, the game relies on the human concepts of good and bad.
It's good to eat the fruit and pellets to increase your score counter. It's
bad to let a ghost catch pac-man, causing him to die and lose a life. Without
being taught, human children can identify all of these elements but the
computer would seem to have a more difficult task.

------
caust1c
For those confused, this is just a recreation of an animated image of pacman,
not a playable recreation of the actual game.

Impressive still, but not exactly what's lead to be believed by the title.

~~~
CShorten
It's not quite as playable as the full pacman game, but it is more than an
animation since it's conditioned on the action taken

------
YeGoblynQueenne
Full article is here:

[https://cdn.arstechnica.net/wp-
content/uploads/2020/05/Nvidi...](https://cdn.arstechnica.net/wp-
content/uploads/2020/05/Nvidia_GameGAN_Research.pdf)

------
ginko
So how large is the generated model compared to the Pac-Man binary? (edit:
which was 16KB)

------
gallerdude
I wonder what the copyright implications of this are. Obviously it's Pacman,
but none of the source is the same.

~~~
lonelappde
Game rules aren't copyrightable. The visuals are a copy but obviously fair
use, and it's obviously trademark use, and Nvidia probably has a marketing
partnership with NAMCO for this

~~~
jerf
"“We were blown away when we saw the results, in disbelief that AI could
recreate the iconic PAC-MAN experience without a game engine,” said Koichiro
Tsutsumi from BANDAI NAMCO Research Inc., the research development company of
the game’s publisher BANDAI NAMCO Entertainment Inc., _which provided the PAC-
MAN data to train GameGAN_. “This research presents exciting possibilities to
help game developers accelerate the creative process of developing new level
layouts, characters and even games.”"

Emphasis mine.

Also, while I disagree that this would be real "fair use" (rather than
Internet Pundit fair use, which is virtually indistinguishable from "I want
this, so it must be legal") without permission, it wouldn't matter anyhow. If
nVidia did this without permission and tried to distribute exactly what we
saw, it would also be a trademark violation. Pacman is almost certainly in the
top 1% of defended trademarks, if not .1% or .01%... not a great thing to try
to use without permission.

------
drcode
The design of their GAN looks super complex, and Pacman is pretty simple... I
kinda wonder if they meta-overfit the game (ie the Pacman game engine which
they say is missing is sorta hidden in the structure of the GAN architecture
that they start off with)

------
spacephysics
How long until this method evolves to solve business requirements? (Argument
could be made that the game _is_ a set of business requirements)

Imagine visual programming but instead of connecting systems and dealing with
high level schemas, the stake holder just defines the data flow, and the model
creates a (hopefully) more extendable system than current visual programming
systems.

But the complexities/black box nature of some neural nets (and the CPU
designed by AI that was enormously complex) may transcend to this use case.
Meaning we can create systems that are more efficient, more easily, but we
ourselves can’t interpret the entire system, let alone each piece
sufficiently.

~~~
nradov
In most real world business situations we can't even define the requirements
and constraints in a manner amenable to AI solutions. It only works for the
most trivial issues.

------
arkanciscan
I've never had a harder time explaining something this impressive to my gf.

------
danans
So AI can learn to play the games, but can it learn to create them? That is,
can it create compelling cultural novelty?

The most successful examples I've heard are computer generated classical
music, but generating abstract art like classical music is simpler than
generating art that relies on coherent language and narrative, like opera, or
the blues, or a good novel.

~~~
mettamage
There has been research on it. I forgot the source and can’t find it easily,
but if you email Julian Togelius, he might know. He doesn’t focus on game AI
research and is the only name I can remember at the moment.

Edit: oh, Togelius also seemed to have researched it.

[https://www.google.com/amp/s/www.technologyreview.com/2017/1...](https://www.google.com/amp/s/www.technologyreview.com/2017/11/29/147421/ai-
is-dreaming-up-new-kinds-of-video-games/amp/)

~~~
danans
Cool. Thanks for that link!

------
ccvannorman
This seems less impressive than the Mario recreation from a few years ago[0]
-- I must be missing something because this is a simpler game that was trained
on significantly more data, yet the result is comparatively worse.

[0] [https://www.theverge.com/2017/9/10/16276528/ai-video-
games-g...](https://www.theverge.com/2017/9/10/16276528/ai-video-games-game-
engine)

------
pbhjpbhj
What're the total number of possible states in Pacman?

~~~
simias
The number of possible pellet configurations on the grid is astronomical on
its own, then you have pacman positions and ghost positions at least. I don't
know the exact value but my estimate is "a very big number".

------
vagab0nd
Does anyone know if we can combine knowledge from different domains when
training to play games? Example: have a language model read pieces of text
about the game, then feed the output of that into a model that's doing normal
reinforcement learning. Maybe train them both at the same time?

The hope is that the model can combine inputs from different domains and
generalize better.

------
dontreact
I predict methods like this will be used to create a simulation environment
that approximates the real world. Reinforcement learning folks are running out
of game engines win inside of. I believe the best way to transcend this coming
limitation will be to take a learning based approach similar to this.

~~~
justicezyx
I mean, it's pretty _clear_ that we are already living in such simulation,
right?

~~~
eru
What evidence do you have? How would the world look different, if we weren't
in a simulation?

------
tripzilch
I wonder, does it also reproduce the four unique behaviours of the ghosts?
(their movement patterns are all programmed differently, one is random, one
goes away from you, and I'm not sure about the rest)

------
polivka2
For anyone interested on how the ghosts AI works:
[https://youtu.be/ataGotQ7ir8](https://youtu.be/ataGotQ7ir8) I doubt the AI
actually recreated this.

------
ngneer
Please help me understand the point. Is it to show that nVidia chips can be
used to create game AI using GANs? Or is it to aid in game design? Wouldn't a
human be in a better position to do both?

~~~
Teknoman117
If I'm understanding correctly, the machine learning model was able to create
a game executable from information about a game.

~~~
Ajedi32
From the article, it sounds to me more like the AI itself is rendering
individual frames of the game. It takes the last frame and the current state
of the gamepad as input, and outputs the next frame. They've effectively
replaced the entire game code with a neural network.

------
kabes
How do they know it's fully functional without trying out how it reacts in
every possible situation (which isn't really feasible)?

~~~
SirYandi
I guess they should train an AI to play it too

------
huffmsa
So wait, is this playable, or is it just capable of reproducing video of
gameplay?

------
tokai
If the old patterns don't work its not really pacman.

------
ookblah
hmm, train a bunch of UI elements and interactions and feed it underlying
codebases and see what comes back... lol.

~~~
dougabug
That’s actually been done at least a couple years back.

[https://towardsdatascience.com/code2pix-deep-learning-
compil...](https://towardsdatascience.com/code2pix-deep-learning-compiler-for-
graphical-user-interfaces-1256c346950b)

------
ge96
dumb question but could ML be made to copy an OS? since this has a
learning/input ability with rules.

------
Kenji
I'd love to see more gameplay. In fact, I'd love to play it myself. I just
can't believe it replicates the original gameplay exactly.

