With build I mean, how do you structure inputs and objects so that the goal is met? The goal for racing is both steering, braking, trottle...
You can get some pretty interesting behavior if you start with a barebones model of the world like that. At one point my racers started teleporting around parts of the track by taking advantage of the fact that I wasn't doing continuous collision detection - they would detect a turn and then output a speed and direction that would cause them to instantly appear on the other side of the corner without navigating the curve on the way. It was interesting to watch them balance the ability to teleport against the range of their sensors.
Imagine having a wall, and a certain portion of it is sticky (since it's covered in invisible glue), but you don't know which portion, and you certainly don't want to touch it to find out. So you throw spaghetti at the wall, and see which portions stick. Next, you throw more spaghetti at the part that seems to stick, and repeat until you've got the entire sticky portion mapped out.
There's an old genetic algorithms demo with smart rockets and path-finding here: http://www.blprnt.com/smartrockets/ [warning: requires flash].
If you're actually interested in building one of these, Daniel Shiffman wrote The Nature of Code: http://natureofcode.com/, which is an introductory level book for this kind of thing. He uses Processing in the book itself for modeling, but he also goes over the theories pretty well.
If you want some formal literature, I took a brain and behavioral processing class in college, and the professor still has his slides up here: http://www.life.illinois.edu/mcb/419/sessions/index.php, along with links to papers and homework scaffolding.
(Don't worry if not, As much as I'd love to browse some of whats on there, I don't want to make extra work for you or get anyone in trouble!)
You can find here all of the code.
Everything here is written (by me) from a scratch, beggining with game engine, and ending on ML algorithms. It's taken to the level where I even multiply matrixes for graphics transformation, operating on pure HTML5 canvas. So everyone should find there something interesting for him. Only editors are a great plugin for ace C9.
When I create projects of my own I like to face some challenges that help me improve my skills. On the other hand there is work, where I use libraries and other ready stuff, as the software has to be usually deployed as fast as possible and in the highest quality. But with project like this I can take my time and just have fun :) And by fun I mean comming up with an arhitecture, code optimization, data structures (like quad tree in this example) implementation and many others.
It took slightly more generations to learn how to race, but not that many more (~80 generations vs ~90 generations). In addition, I thought I observed a small difference in behavior, but nothing I could explicitly measure.
But I'm testing this scenario right now, and all I can get are cars going on different curves. Are you sure, that you have restarted the process after making changes to the script?
And I figured out what the behavior difference was: before they finally "figure it out", they spend most of their time spinning in place or moving in "epicycles", for lack of a better term.
Thanks for the project; it's been great fun!
Also, changining the track from one interation to another I don't think it helps. The training should be done on the same track until at least one of the cars is able to complete it. Than it can be changed.
Sometimes some of the cars get stuck and the round doesn't finish, without having to click the "End round" button. There should be some mechanism that detects that cars are stuck and end the round automatically.
I actually have had inactivity detection implemented at some point, but I've resigned of this in favor for user-defined stop condition. But you are right, I will bring that back.
It's actually expected that in some cases it might take many generations to learn, as the population is rather small. In other attempts they can do this in like 5 generations, so please, tray to restart the whole process.
Keeping one track untill at least one car is able to finish it is not as good idea as it seems due to ovefritting. The will simply memorize all the moves depending on some insigificant factors and they won't be albe to finish any other track. I have tested that :D
For research purposes :)
0.5 mutation rate is way too much..
On the other hand 0.5 is not so large, when you look at the mutation procedure. If a speciman is going to be mutated, only 20% of it's weights are changed and only within a range [-0.1,...,0.1]. But fill free to experiment :)
While I agree that exposing the NN to diverse tracks each iteration asures a thorough learning, I was kind of curios how susceptible the algorithm is to exploit irelevant particularities of a track instead of finding the general rule.
I wish the creator will add the posiblity to chose if the track should be changed each iteration or only after at least one car completed it.
value => value >= 0 ? value : 0.5 * value;
It should be noted that the genetic algorithm used is not the same as reinforcement learning, in which the entire history of decisions leading up to a success is fed back to the NN as training examples.
EDIT: Sorry, I've understood you wrong, what you are describing is, of course, not a tabu search, but still I believe that this GA is a reinforcement learning.
EDIT2: After some research I can say, that opinions are divided :D
There isn't much documentation though - is the genetic algorithm being used to evolve the NN somehow?
Yup, I (as a developer of this thing) haven't created a documentation yet, but it's mostly due to the fact, that I have deployed this just yesterday and there are still many things that I'd like to change in it. But I will work on that :)