
Show HN: Touchable augmented reality on your phone, 3 years in the making - lazyjeff
https://portalble.cs.brown.edu
======
Phillips126
Looks quite interesting. I remember when the Leap Motion was first being
released but I haven't seen a great deal in the wild that use it.

I signed up to get an early dev kit of their technology when I was working at
an old job. We were a non-profit agency that helped educate individuals with
disabilities - often having difficulty with their motor skills (using a
typical mouse/keyboard was impossible for most of them). The idea was we'd use
Unity3D or a similar game engine to create minigames that used the leap motion
as the input device. The games would educate the individuals on many things
from normal daily hygiene to cooking, etc.

In the end, our tiny team (of 3-4 people) bit off more than we could chew and
nothing substantial was ever materialized. The leader of the team moved on to
another company and the project dissolved. It was definitely one of the more
interesting projects I worked on at that company.

------
akrymski
Cool. But you shouldn't need Leap for this these days, you can do hand
tracking with ML: [https://ai.googleblog.com/2019/08/on-device-real-time-
hand-t...](https://ai.googleblog.com/2019/08/on-device-real-time-hand-
tracking-with.html)

~~~
lazyjeff
We have directly tested the MediaPipe one and it's less accurate than the
Leap. Like it's completely unusable in practice, especially when the palm is
not facing the camera. Also there's an HCI problem, which is that people often
go offscreen when interacting with objects (your brain assumes things can
happen outside the field of view), but even a moment outside the field of view
breaks the interaction.

------
villgax
Try checking out [https://ai.googleblog.com/2019/08/on-device-real-time-
hand-t...](https://ai.googleblog.com/2019/08/on-device-real-time-hand-
tracking-with.html) this might remove the need for any hardware apart from the
phone & its vibration motor.

~~~
lazyjeff
Yup we've looked into that. It's not accurate enough to feel good to use, and
often you go off screen for a bit which breaks the interaction. The infrared
sensor actually has a field of view wider than the camera and helps maintain
the connection in that brief offscreen moment.

