

Eyetracking Jetpack Joyride, Smash Hit and Dungeon Keeper - chewxy
http://blog.chewxy.com/2014/04/08/eyetracking-jetpack-joyride-smash-hit-and-dungeon-keeper/?hn=1

======
xyzzy123
Using a tablet's built-in camera for gaze tracking is a pretty neat hack :)

I wonder if you could make a game which incorporates looking (or not looking
at things) in some way?

~~~
chewxy
It's not going to be terribly accurate. I have accuracy of about 80-85%. It
loses accuracy towards the edges, and towards the bottom of the screen

~~~
xyzzy123
Any idea much more accuracy might be achievable given hardware limitations?

~~~
chewxy
well, it would take a lot of tuning of the code. But even with a lot of
careful tuning, there will be limitation in form of the framerate with
JavaCameraView or even NativeCameraView.

Better algorithms exist. I've tested them too, but they're too slow to work on
the onboard hardware.

I've worked out an algorithm where a portion of the processing is offloaded to
a different machine. Assuming a fast enough internet connection, (or even
local LAN), and assuming that the offloaded processing is done with say,
graphics cards or something like that, the accuracy will be largely improved
to about 90-ish %.

It took me quite a loooong time to figure out how to profile code in Android,
but the processing bottlenecks are these: finding the center of the eye, and
calculating the probable location and shape of the iris. So these are the
things that are to be offloaded to be processed.

Naturally network latency would be then an issue, but I have some ideas on how
to overcome those

TL;DR - if it's just relying on onboard hardware limitation, more accurate
algorithms cannot be used because it would slow to a crawl.

EDIT: also, there is the issue with memory use - my app would consistently
crash after about 1 min of use (I used this to my advantage) as it was somehow
somewhere leaking memory.

~~~
xyzzy123
I see, thanks. So a game would probably be hard, since games require low
latency. Something might be possible if one could find mechanics which don't
require high accuracy...

