
HAMR – 3D Hand Shape and Pose Estimation from a Single RGB Image - posnererez
https://heartbeat.fritz.ai/hamr-3d-hand-shape-and-pose-estimation-from-a-single-rgb-image-225cba26ce2
======
mncharity
In other news, there's a Google MediaPipe hand tracking example.[1][2] It's
still documented as iOS/Android only, but there's now a hand_tracking
directory under linux desktop examples![3] Results have been mixed.[4]

[1] [https://ai.googleblog.com/2019/08/on-device-real-time-
hand-t...](https://ai.googleblog.com/2019/08/on-device-real-time-hand-
tracking-with.html) [2]
[https://github.com/google/mediapipe/blob/master/mediapipe/do...](https://github.com/google/mediapipe/blob/master/mediapipe/docs/hand_tracking_mobile_gpu.md)
[3]
[https://github.com/google/mediapipe/tree/master/mediapipe/ex...](https://github.com/google/mediapipe/tree/master/mediapipe/examples/desktop)
[4]
[https://www.youtube.com/watch?v=ZwgjgT9hu6A](https://www.youtube.com/watch?v=ZwgjgT9hu6A)
(Aug 31)

------
alok99
I didn't see any mention of how long it takes to recover a mesh from an image.
I imagine it's a significant amount of time, not including training.

What I wonder is, if this technology was fast enough, could it be used to
caption sign language?

The full publication:
[https://arxiv.org/abs/1902.09305](https://arxiv.org/abs/1902.09305)

------
thedaemon
This looks amazing and really useful for real-time sign language translation!

~~~
amayne
It’s worth paying attention to the deaf community’s thoughts on any tech that
comes along, gloves etc., for translating sign language. Short version, ASL is
more than hand gestures and past efforts at solving this tend to focus more on
assisting the hearing than the deaf.

This is a really good write up on the issue of signing gloves and tech:
[https://www.theatlantic.com/technology/archive/2017/11/why-s...](https://www.theatlantic.com/technology/archive/2017/11/why-
sign-language-gloves-dont-help-deaf-people/545441/)

~~~
112233
Thank you for the constructive reply and the link, it will certainly be
useful. It is hard for me to properly respond, when this topic (sign language
machine translation) comes up. You cannot encode sign language as text! You
generally cannot understand what two people are talking about informally, if
you don't know them. Oh well...

~~~
amayne
My high school had a fairly large deaf program. I took Latin with a majority
of the students being deaf/hearing impaired. Best part was watching the
translator relay my smart-ass jokes to the other students. I’d say something
and a half minute later get people grinning at me. It was a fun experience and
helped prepare me for performing in other countries.

------
mncharity
Other work: [https://github.com/xinghaochen/awesome-hand-pose-
estimation](https://github.com/xinghaochen/awesome-hand-pose-estimation)

------
etaioinshrdlu
Maybe the soli radar is just unnecessary, if you have a low light camera or
just illuminate with IR.

------
romwell
This is one problem where getting the results in software is very impressive,
but the problem becomes much simpler with just a modicum of extra hardware.

LeapMotion[1] devices accomplish this with nothing more than a pair of cameras
in a matchbox, in real time. And this kind of hardware is already becoming
standard on cell phones and laptops.

Still, the killer obstacle for the applications I was trying to make was not
precision, but _latency_.

Amazing work on HAMR - I wish they had the latency numbers in the article too!

[1][https://www.leapmotion.com/](https://www.leapmotion.com/)

~~~
Tade0
Interesting. I didn't notice any significant latency when I was using it.

Or were you trying to make musical instruments?

~~~
romwell
>Or were you trying to make musical instruments?

Bingo :) The latency was pretty low, and good for UI, but you'd want <10ms
total latency for musical instruments.

~~~
Tade0
I can relate. My biggest disappointment in this regard were Android devices -
such flexibility and potential! But still, they couldn't deliver.

This goes to show that consumer-grade hardware(and associated software) is not
up to par with the requirements of making music.

