
Teaching Your Computer To Play Super Mario Bros - hivebrain
http://www.ehrenbrav.com/2016/08/teaching-your-computer-to-play-super-mario-bros-a-fork-of-the-google-deepmind-atari-machine-learning-project/
======
rubenfiszel
I believe that mario bros is deterministic and doesn't involve any randomness.
Did you implement random start for fair evaluations to make sure that he found
patterns instead of learning what to do by heart (which is a lot simpler).
Random evaluations is a chain of random input at the start to randomize the
environment a bit.

~~~
ehrenbrav79
No - I actually didn't use a random start. But when Mario dies far enough
along in a level, he starts from near that point rather than from all the way
back at the beginning, which kind of serves that purpose. I agree that what's
most interesting is truly learning to generalize rather than just learning a
sequence of moves that happen to work...

~~~
rubenfiszel
Well not really since the fact that he dies and respawn at that point is still
deterministic. Cool project though

------
smegger001
I wonder if you were to train a single NN on multiple games (say maybe Mario,
Commander Keen, Duke Nukem 2, Battle Toads, etc) if it would perform better
against any one game than say a NN trained against just the one same game? I
know I have gotten better on any particular game when I have played more games
of the genre. If that;s the case could we use this approach to make a General
Game Playing AI.

~~~
cttet
The original DQN was created to do that.

------
twblalock
This is a similar project written in Lua, with a different Mario game:
[https://www.youtube.com/watch?v=qv6UVOQ0F44](https://www.youtube.com/watch?v=qv6UVOQ0F44)

~~~
qwertyuiop924
Referenced in the article as one of the two predecessors to this project.

------
CGamesPlay
Neat! I also did a similar project but haven't invested the time in letting it
run for very long. 4 million frames is about 37 game play hours, right? What
were your results? Can Mario clear the first level?

When I implemented this, I scored the q-learning using real score and live
count. I didn't need to add in a bonus for running right because it would
randomly grab the first coin or randomly jump on the goomba quite frequently.

Also, I implemented mine using tensorflow, so it's possible I just sidestepped
some bug.

~~~
cissou
if you check the video you can see it clear the first level and fail a couple
of times on the second one

~~~
wolfram74
Fail a couple of times is generous, it get's a game over around half way
through the second level.

------
soared
I've been collecting these videos. So far I have Mario, an RC car, Montezuma's
Revenge, and a NLP for grammar.

[https://www.reddit.com/r/watchmachineslearn](https://www.reddit.com/r/watchmachineslearn)

------
nojvek
This was a great explanation. Thank you. I wonder if similar technique can be
used for lane following robot that would understand stop signs, traffic lights
and avoid collisions.

I've been thinking of creating a simulator game for driving, however have
absolutely no idea how to program the AI side of things.

~~~
ehrenbrav79
Yeah, you'd need to "punish" it somehow for running stop signs and lights :)

If there's a good driving sim game on Nintendo, it should in theory take
minimal modification of my existing code to make it work...

------
yzh
I really wish someone could try deep neural networks on my favorite mario
doujin game that combines mario with portal:
[http://stabyourself.net/mari0/](http://stabyourself.net/mari0/)

------
seccess
Also, check out
[https://www.youtube.com/watch?v=qv6UVOQ0F44](https://www.youtube.com/watch?v=qv6UVOQ0F44)
for another cool ML-Mario implementation.

------
orsenthil
How long did the training take? I am still training the default breakout game
and it still has not learnt yet. I have been training it for a week on a 4 CPU
machine. Should I go for GPUs?

~~~
ehrenbrav79
Definitely use a GPU if you can - it's just so much faster. You could also try
training it on a powerful machine in the cloud if you don't mind spending the
money for it...

I'm running a 8 core machine, 12+ GB RAM, and a 980 GPU. It takes roughly 24
hours for 1 million steps in Mario...

------
SilasX
Sorry for content-free comment, but this is _really_ interesting, and I wish
it had gotten more discussion. Starred and saved for later.

~~~
ehrenbrav79
Glad you think so! It was a lot of fun to put together and I'm contemplating
working on something like Zelda next now that I have a computer-controllable
NES emulator working...

------
durkie
missing the best video about teaching computers to play video games:
[https://www.youtube.com/watch?v=xOCurBYI_gY](https://www.youtube.com/watch?v=xOCurBYI_gY)

