
Artificial Intelligence Uses Videogame Footage to Recreate Game Engine - ingve
https://gvu.gatech.edu/ai-uses-less-two-minutes-videogame-footage-recreate-game-engine
======
stephengillie
This is reminiscent of the Tribes 2 development, when the developers revealed
they lost the original Tribes source code, and thus the physics calculations.

Beta testers constantly complained that falling didn't feel right, jetting and
skiing (the game's main movement) was slow and soupy. Players were leaking
videos and photos showing the differences in motion.

At one point, Dynamix hired a top player to playtest, as he "remembered the
best" how it felt. All to reclaim the physics calculations of some game from
the Leisure Suit Larry company.

~~~
ramses0
As a matter of fact, there is still a regular group of Tribes2 players online.
This past Memorial Day weekend, I booted up a VM, d/l'd the ISO's, etc. and
played a capture the flag match for a few hours... eventually with 16+ on the
same server.

The game is 16 years old... old enough to drive a car and it's still being
played!

~~~
52-6F-62
REALLY?? I wonder if there's a Tribes 1 revival group...

Tribes 1 was an absolute favourite in my household growing up. It was the
first real viable, extendable MMO fighter I'd played. I don't know if I've had
as much fun in a similar format (and I enjoy [like the odd ice cream treat]
some modern ones like BF1). Maybe it's just sentimentalism. It was the first
game I got my younger siblings into, and they became obsessed. More than me...
haha...

SHAZBOT!

~~~
khedoros1
Tribes 1 was the first FPS I played after about...Star Wars Dark Forces, I
think. It's the first game that I had the opportunity to play on a LAN, and
it's the first game that I remember using mouselook in.

We had a community center near home, with about 10 gaming PCs. Tribes was a
huge favorite, especially with all the different mods that were available.

We switched almost completely to AVP when that came out, though.

~~~
52-6F-62
>We had a community center near home, with about 10 gaming PCs. Tribes was a
huge favorite, especially with all the different mods that were available.

You just reminded me. I had a real slack-off sort of teacher(s) in my 9th and
10th grade communications labs, and Tribes would run on the old Dell boxes
they had in there for whatever work. We had a whole system worked out with a
watcher and everybody playing over LAN in the lab whenever the teacher stepped
out of the room. I'd forgotten all about it.

------
egypturnash
It sounds like the only input is video. Video plus the user's controls might
be much more interesting; there's a huge archive of input recordings available
at tasvideos.org that could conceivably be used as a source, rather than
making people actually play the damn things.

Or you could just take some of the other AIs designed to play games based on
video and wire them up. Then just let your system learn to play AND run Mega
Metroid Brothers.

~~~
zardo
>Then just let your system learn to play AND run Mega Metroid Brothers.

One of the many games on the triangle with vertices Mega Man, Super Metroid,
and Mario Brothers?

~~~
ticviking
I'd play something in that space. At least to see if it is good.

------
ZirconCode
I don't have any experience in the field, but reading the paper, it seems
impossibly weak and almost useless. This should only work for a very limited
type of game, in which case it will never help anyone speed up game
construction more than a simple sprite engine. It seems more an exercise in
using openCV.

What am I missing here, because I'm positive that I am missing something?

~~~
tree_of_item
An artificial intelligence observed a game being played and recreated it and
you aren't impressed at all? Man, the future must be boring for people living
in it.

Yes, the technique it uses only works for a certain space of possible games.
That means there is an obvious path to increasing the size of that space.

~~~
AstralStorm
"Observed" after being fed lots of sprites and actual ways on how to play it
and actually win at it in the objective function. And it "played" only one
kind of game. "Obvious path" riiiight.

In addition, this is wrong to having been said to be new, such attempts have
been made before and even stronger in results and generality. For example this
(relatively dumb) approach from 2013 generalized kinda well, much better than
I've seen a silly even deep network generalize:
[http://www.cs.cmu.edu/~tom7/mario/](http://www.cs.cmu.edu/~tom7/mario/)

So yes, they are overselling it a lot. I am 100% not impressed by this paper
as it lacks critical detail. That it can parse stuff from 2D frames is not
interesting, it is basic motion analysis which can be done even by a supremely
stupid algorithm, not even a CNN.

I mean, Google best AI can play 15 rooms of a simple game...

~~~
elliottcarlson
You are comparing a system that learned to play a game (which indeed was very
impressive), to a system that learned to make the game by observing the
behavior from video. None of your points actually relate to the system
described.

~~~
AstralStorm
By "make" you meant "match some sort of a simple function approximation after
hardcoding lots of knowledge about the system and the general function" right?
Which is essentially what the neural networks and all the other optimization
algorithms were made for?

(The algorithm as described will require a huge database for a game that is
even slightly more complicated than Infinite Mario. And we don't even have the
sources to try that.)

Even the object motion tracker part will choke in 3D environment. (It is a
greedy matcher as they described it.)

Speaking of impressed, Google DeepMind paper is way more actually feasible to
improve upon and rich in detail:
[https://arxiv.org/pdf/1606.01868v1.pdf](https://arxiv.org/pdf/1606.01868v1.pdf)
Compare the two papers in straight quality. I understand why you'd publish any
worthless junk in the current academic culture and do not agree we should
actually do it.

------
jordache
the examples had the character move precisely like in the reference video.
This just looks like the AI recorded the original video and played it back.

I thought the AI created a playable game engine from the reference video? If
so, why did it need to replicate the exact movement of the game character? Why
not come up with its unique set of movements in a fully flexible game engine?

~~~
kilceem
Maybe that was just to show the recreation. An engine is pure code and a lot
of times closed source. Just creating animation and similitude in that ways
does not recreate a engine but could create those aspects. Engines have many
aspects of game mechanics many you do not really see directly but as a
byproduct.

~~~
jordache
But the impressive sell of this technique was the AI discerned enough patterns
and physics from the reference video to auto generate a game engine that
replicated/simulated the original game. This alludes that it understood the
player character and now can control the sprite representing the player
character via unique paths through the game map.

It's waaaayy less impressive if it's just programmatically processing video
frames, tracking pixels to generate coalescence, generating a library of
sprites based on pixel coalescence, and then playing back the same sequence of
sprites programmatically...

------
Upvoter33
more AI theater. there will be a moment in about five years when people are
like "what happened to all that stuff about auto-driving cars, evil AI, etc.?"

~~~
red75prime
Siri, what happened to all that AI nonsense from 5 years ago?

I'm sorry, Dave, I'm afraid I can't answer that.

------
Phait
Seriously, tech journalism needs to get it's shit together because it's
becoming annoying. If I see another article about a mediocre algorithm
presented as AGI, I'm gonna start posting on my blog about how every failed
experiments is really Skynet being lazy in its teenager years

~~~
Bartweiss
Agreed, although I think this case shows a different problem - it's not a tech
news site but a university press release.

University press has a horrible tendency to oversell research in the name of
getting news coverage, often completely burying possible flaws or limitations
of the result. The University of Maryland infamously put out a major release
on concussion treatment based on a study that _didn 't exist_ (1). Similar but
lesser abuses appear to be almost constant. It seems like worthwhile-but-
unspectacular thesis result gets spun as a groundbreaking insight in its
field.

1: [http://nymag.com/scienceofus/2016/01/chocolate-milk-
concussi...](http://nymag.com/scienceofus/2016/01/chocolate-milk-concussion-
scandal.html)

------
daralthus
They wrote a blog post a couple of month ago with more details:
[https://medium.com/@mark_riedl/automated-game-
understanding-...](https://medium.com/@mark_riedl/automated-game-
understanding-e8c9107c5ac2)

------
bennysince86
"The technique relies on a relatively simple search algorithm that searches
through possible sets of rules that can best predict a set of frame
transitions"

so, their script/app doesn't reproduce a game engine at all, it instead
analyzes pixel arrays from video frames and maintains rules about how the
pixel arrays typically transform from one state to another. this sounds more
like a useful video analytics tool than an "AI that makes game engines". if i
was responsible for the headline/marketing, i would've gone with "Artificial
Intelligence Can Predict the Ending of a Movie" or something (as long as the
movie is 8-bit and only has 256 possible colors!)

in other words i dont think Unreal or Unity are worried about this tech.

------
bllguo
The algo learned about the underlying rules of its environment - the physics -
by watching video. Imagine applying this to more complex games, and then to
the real world (the ultimate game!). Cool approach; to me it seems to really
resemble how people learn in real life.

