
PointerPointer: points to your cursor - nthitz
http://www.pointerpointer.com/
======
paulirish
I made a quick video explaining how the webapp works:
<http://youtu.be/Z2ZXW2HBLPM>

Essentially, using the gridPositions.json to populate a Voronoi Grid (toggle
display of the <canvas> in your browser). Then your cursor position is used
against the grid to find which shape encloses it which maps back to the source
JPEG which is then displayed.

~~~
snowwrestler
I watched your video. It does not seem to explain how the fingertip position
in each photo was calculated, which it seems to me is really the heart of the
magic.

~~~
Flam
He says how it's basically a json file with every image mapped to coords of
each voronoi "tile". I guess it was handmade in advance

~~~
snowwrestler
Not a bad guess but I'd be interested to actually know the details. Making it
all by hand would be problematic for reasons beyond the sheer volume. How
would one find a deep, diverse set of images of people pointing? How did the
creators ensure they had images to cover all possible pixel positions within a
certain proximity? It seems like it would take reviewing many more than 900
images to produce a final set of 900 that includes even coverage.

~~~
olalonde
Another guess here, but it could have been crowdsourced using Amazon
Mechanical Turk. Assuming a conservative 0.05$ by picture, the total cost
would be 901 * 0.05 = 45.05$.

~~~
nmb
Yeah, this seems likely. Here's how I'd do it: create a webapp similar to this
one to show to your Turkers. The difference would be that it would show an
image at random, and the Turker would decide whether or not someone is
pointing in the image, and if so they would click where the person was
pointing. For each image, take the mean position of mouseclicks. This becomes
a seed for the Voronoi diagram.

You can tell he/she "cheated" a bit for some the parts of the canvas that
didn't get many seeds. For example, if you move your mouse within one of the
large cells in the bottom left, the image just moves to keep up with the
pointer. :P

------
runako
Interesting example of how a hard problem (image recognition/understanding)
can be faked for some cases by solving an easier problem (lookup table, a few
hours of manual intervention.)

~~~
taylorbuley
Seeing <http://www.pointerpointer.com/gridPositions.json> really took away
some of the magic for me. Still, very clever and shows that web magic can be
in the eye of the mouse-holder!

~~~
ma2rten
Erm.. even there were some computer vision algorithm involved, he would
probably want to run it offline.

------
Inufu
doesn't really work for me with Chrome on Windows 7 - an image flashes for a
split-second, then it's back to "Finding pointer" (I'm not even touching my
mouse)

~~~
AsylumWarden
Same here. Just flashes the picture really fast in the latest stable release
of Chrome under Windows 7. Works beautifully in the latest Firefox though.
Windows IE isn't supported at all apparently (I tried IE 9).

Are the developers willing to share some insight into how this works? The
picture selection is right on and there seems to be a good number of pictures
at that.

------
sporksmith
Fun :). I did something similar with a bunch of photos of a stuffed monkey. I
made a music video instead of an interactive web app, though the latter would
probably not be too difficult since I wrote the software using Processing.
<http://sporksmith.wordpress.com/travels-of-code-monkey/>

------
shashashasha
A faster (older) version of this by the same people, with camera flashes:
<http://flitser.org/>

~~~
timmaxw
Warning: at least one of the images is not "family-friendly". Are the images
automatically collected?

~~~
hoprocker
It's probably easier to automate the picture harvesting here via an intensity
filter. I'm still stumped about how the finger-recognition was accomplished in
the OP.

------
Groxx
Definitely good for a chuckle. It helps a _lot_ that the photos are _very_
well aimed, really makes it feel magical :)

~~~
pyre
The images are adjusted. I was able to get the same picture 3 times in a row
by moving cursor slightly, but the finger was always aligned exactly the same
on the cursor, even though I had moved it.

------
hackNightly
Just wasted about 20 minutes of my life playing with this. I would really like
to know the technology behind. Is there some sort of ML going on, or is this
human trained? Either way, very entertaining and scary accurate.

~~~
xd
You wasted 20 minutes ... what makes you think someone didn't waste 20 days
pulling the image collection together. Occam's razor ;)

~~~
almost
All the images are listed here:

<http://www.pointerpointer.com/gridPositions.json>

There are around 900 of them! I'd have used Amazon's Mechanical Turk...

Very cool effect, I also wasted way to much time playing with it :)

~~~
hoprocker
Good "point" -- the right question posted to mturk/crowdflower/etc. could have
knocked out that json file for relatively minimum time/cost. Oh, the things we
humans engage in.

------
alpb
There are serious problems around the upper right corner. I did not get any
results from there. I'm wondering whether the pictures are originally pointing
that coordinate or are they resized+translated to some offset?

~~~
Geee
You'll have to move the cursor left a bit, to get the same image, but with
visible finger. The images are indeed offset so that in the upper right corner
the finger actually doesn't fit in the frame.

------
ionforce
I don't see the point of this. The images load far too slowly.

------
kephra
One suggestion is to move the jquery from jquery.com to your domain for two
reasons:

1: It will be one click less, for those who use noscript. 2: It will allow
your site to survive the case that jquery.com changes anything, or goes down.

Besides that its a nice toy.

~~~
jdpage
Alternately, use the Google CDN.

------
Andi
What is the point about this beyond showing pictures where the finger is next
to my cursor? (Why is this popular?) It's not more interesting than a
circumference search. You could implement that with a simple rtree.

------
chousho
Haha, very clever. Although I seem to be running into problems whenever I put
the mouse into one of the 4 corners of the screen. It can find anything more
centrally located, though (apparently?)

------
hanbam
I was actually expecting that somebody did some machine learning / matching
over a large set of pictures and found all those images / coordinates.

------
CountHackulus
This is pretty neat, I wonder if it has to be human trained or if there's some
sort of computer vision algorithm determining pointing.

~~~
siim
After looking into to the code I found out that this app is made by using
Voronoi diagrams. [1]

The actual positions are saved in a json file. [2]

[1] <http://en.wikipedia.org/wiki/Voronoi_diagram>

[2]<http://www.pointerpointer.com/gridPositions.json>

~~~
slig
Yes, but how did the author create that .json file? Manually or using some ML
algorithm?

~~~
saraid216
What would be interesting is if new images were periodically added and the
grid adjusted in response.

------
AznHisoka
How can I get a pointer to my cursor in Javascript? I might find this useful
in automating certain processes.

------
DiabloD3
On Firefox, never goes past "Pointer found, pointing". No image loads, works
on Chrome though.

------
edwinyzh
What's this? I saw some pictures flashed after some loading indicators are
shown...

------
PidGin128
This is technically interesting, but voyeur-istically creepy.

------
gouranga
Doesn't work in IE9. That was the limit of my experience :(

------
Inetgate
Is this a some kind of concentration game?

------
joshu
Why does it take so long to load?

~~~
apu
I was wondering the same thing. With a precomputed voronoi diagram and only a
few hundred cells, lookups should be almost instantaneous.

------
cocoflunchy
Doesn't load for me, on Chrome/FF/IE9...

------
jsilence
awesome!

