

Show HN: AI QLearning Robot Simulation in JavaScript and WebGL - javierluraschi
http://qlearning.4ck5.com/

======
akyu
Very Cool.

I did a similar project using a Neural Network instead of a look up table,
similar to how deepmind did their Atari game system. The neural network adds
some overhead to the training time because of backprop, so for small state
spaces which the table can handle easily it is a bit overkill. But the neural
network can scale up to much more complex state spaces that would be
impractical to use a table for.

~~~
javierluraschi
Do you have a library for your project? It sounds really interesting. I think
this space is interesting enough that would benefit from a GutHub project to
collaborate.

------
nrox
It looks based on this repo:
[https://github.com/nrox/q-learning.js](https://github.com/nrox/q-learning.js)
and this example:
[http://nrox.github.io/q-learning.js/test2.html](http://nrox.github.io/q-learning.js/test2.html)

~~~
nrox
that makes me proud :) do you have suggestions to improve this library ?

------
dlsym
Food / Poison break even was at 88. A chart would be interesting.

However - since R2D2 seems like a slow learner it would be nice to speed up
the time a little bit.

~~~
doorhammer
Warning: I only glanced through the code real quickly, so I haven't checked to
see if this has unintended side effects to the game logic, or to make sure
it's a complete solution.

It looks like there's a variable that controls how quickly the board itself is
updated:

    
    
      var boardGameUpdateInterval = 10;
    

And a function deciding how quickly the visual representation gets updated:

    
    
      var render = function () {
          setTimeout( function() {          
              requestAnimationFrame(render);
          }, 1000 / 15);
    
          if (gameTicks % boardGameUpdateInterval == 0) {
              updateBoard();
          }
      
          animateBoard();
      
          gameTicks++;
          renderer.render(scene, camera);
      };
      

They're global variables so it's pretty easy to just drop into the developer
console and overwrite them.

I just changed the denominator of the setTimeout to 30 instead of 15, and the
boardGameUpdateInterval to 5 instead of 10. Of course, you don't have to
change the render speed to get the logic speedup, but I figured I'd do both.

This appears to have sped the simulation speed up by a factor of two.

ymmv

------
hmate9
Looks very nice, I have one question:

How does R2D2 "see" the blocks? Is it by pixels or is the game hard-coded in
this sense?

~~~
javierluraschi
Currently, it sees a board of 3x3 states, as in:

_ _ G

G B B

_ _ G

A next step would be to use real computer vision over a 3D camera in the
environment aligned to R2D@ and a deep neural network to interpret pixels
instead of states. DeepMind has a paper describing this approach:
[http://arxiv.org/pdf/1312.5602.pdf](http://arxiv.org/pdf/1312.5602.pdf)

