
Show HN: An Augmented Reality Hand Puppet App - hwoolery
https://medium.com/yopuppet/bringing-puppets-to-mixed-reality-8ea5d9771c4f?r=1
======
A-minus
Awesome idea - tracking hand movements can't be easy! I like the dino puppet.

------
Stanleyc23
cool work! i was also working in a similar game/computer vision space its very
fun

and also

> hand gesture recognition is far more complicated than facial feature
> detection

the good news is we may not be that far off anymore
[https://news.ycombinator.com/item?id=14263678](https://news.ycombinator.com/item?id=14263678)

~~~
hwoolery
thanks! I saw that page as well a while ago on HN, although I'm not sure how
feasible it is for such a network to run smoothly on mobile.

~~~
Stanleyc23
I'm no expert in anything deep learning, but I think getting it to work on
mobile is just a matter of importing your trained file like the dlib facial
landmark detector. which definitely runs in real time - albeit maybe not at 60
FPS.

~~~
hwoolery
yup, I actually use the dlib facial detector in this app for a ventriloquist
type feature (not shown in video). The facial landmarks detector assume you've
framed an ROI for a face already. Locating faces in an image is trivial...
locating hands in an image and their orientation, not so much : ) That's not
to say a CNN couldn't be trained to do it, it's just going to take some work!

