
Using Keras and Deep Deterministic Policy Gradient to play TORCS - wei_jok
https://yanpanlau.github.io/2016/10/11/Torcs-Keras.html
======
modeless
This is cool, but unlike some other recent neural game players [1] [2], it
doesn't look at the screen at all. It also has an objective function which is
tightly coupled to this game. Therefore this work is not easily generalizable
to other video games, nor would it be useful for a real-world self-driving
car.

[1] [http://vizdoom.cs.put.edu.pl/](http://vizdoom.cs.put.edu.pl/) [2]
[https://deepmind.com/research/dqn/](https://deepmind.com/research/dqn/)

~~~
yanpanlau
It is quite easy to change the input features as pixels and fit into convnet
under Keras (That's why I love Keras so much). However, gym_torcs only support
64x64 pixels and it is hard to see by human eyes, IMHO.

[https://github.com/ugo-nama-kun/gym_torcs/issues/4](https://github.com/ugo-
nama-kun/gym_torcs/issues/4)

~~~
modeless
Interesting, yeah that seems a bit small. Would be a fun project to fix!

------
hardmaru
Projects like these reminds me of why I got into AI research in the first
place.

------
bluetwo
Very interesting.

Given enough training, would the car learn to find the apexes of turns?

~~~
sp332
No, the function rewards staying in the middle of the track.

~~~
yanpanlau
Staying in the middle of the track is not a necessary requirement in the
reward function. The reason I include it is to speed up the learning time in
the beginning. You can remove it once you learn a reasonable policy and see it
the agent can find the optimal apex path. I will do a test tonight.

Just like in human world : You first learn how to drive before you learn how
to drift the car.

~~~
bluetwo
Any news on the test? I'm curious.

~~~
yanpanlau
Please find the result below. I modified the reward function such that staying
in the middle of the track is no longer required.

[https://youtu.be/Tb5gASEJIRM](https://youtu.be/Tb5gASEJIRM)

