
Microsoft Research turns Kinect into sign language reader - tellarin
http://blogs.technet.com/b/inside_microsoft_research/archive/2013/07/16/digital-assistance-for-sign-language-users.aspx
======
lukifer
I'd be very curious to learn about the speed difference between a competent
typist and a fluent signer, especially if visual cues can be used to
accelerate the latter (ie, an equivalent of auto-complete).

When writing prose (as opposed to code), I could actually see signing being
very speed-competitive, and possibly a more natural way to communicate text
than a keyboard. (Other advantages might be use with Glass, or the ability to
pace around a room while "typing".) Maybe someday, kids will learn signing
instead of keyboarding.

~~~
seiji
_When writing prose (as opposed to code), I could actually see signing being
very speed-competitive_

How so? Signing isn't as "complete" as written or spoken language. You would
say/write "The dog ran in the park yesterday" but the signing would be more
like "YESTERDAY, DOG RUN [THROUGH] PARK."

You're not going to get David Mitchell out of ASL, but then again you can't
get motive music videos in Dejima either. They each have their place and
meaning, but they aren't quite interchangeable.

~~~
lukifer
Right, as a non-signer it didn't even occur that syntax would be different,
implied, or occasionally non-existent. In theory AI could try to auto-
generate, but yeah, maybe not as rosy as I hoped.

------
mabbo
Having studied ASL at University (almost got a minor in it) I have a bit of a
hard time believing this is fully possible at all. Signed languages aren't
simply a series of words where each word has a given sign. It's a very complex
and nuances language, and the regional dialects alone would make it tough to
really decide what sign means what noun or verb.

~~~
ericd
Probabilistic graphical models like Markov networks can yield surprisingly
complex behaviors, if they're complex enough. I don't have much trouble
believing that it's possible.

~~~
Argorak
Considering that sign languages regularly invent signs, e.g. for names, I
doubt that. The interpreter might find out that there is a name at that place,
but not know what the written equivalent of that name is.

~~~
seiji
Nobody is arguing complete all-knowing recognition of human movement. It's
just recognition of a sign for "table" being table and a sign for "hat" being
hat, etc.

~~~
mabbo
Except that is the very problem: drive 100 miles, and the signs for table and
hat are different. Change the conversation, and the signs may be different. I
can put an arm low and flat, and it means a general flat surface. It doesn't
mean table until I mime setting a cup down on it, for example.

Like I said, complex and nuanced. You have some incorrect assumptions in how
signed languages work.

~~~
seiji
Everything can be deciphered completely contextually based on location though.
None of these are insurmountable problems.

I wouldn't claim an ASL recognizer would suddenly identify BSL or anything
wacky like that. Systems can even be trained (or generalized) down to a per-
signer level. You could even have "Joe Pikachu's Generic Gesture Translator"
too.

Think solutions, not problems.

------
jcater
I wonder how it would distinguish between the many dialects in use in the US.
Granted, a lot of the adult population does use ASL [1], but lots of times
children grow up using one of the Signed English[2] variants, which vary from
state to state.

[1]
[http://en.wikipedia.org/wiki/American_Sign_Language](http://en.wikipedia.org/wiki/American_Sign_Language)

[2]
[http://en.wikipedia.org/wiki/Signing_Exact_English](http://en.wikipedia.org/wiki/Signing_Exact_English)

[Edit: links]

~~~
stephengillie
Regional Language Packs?

------
lucb1e
Well that's gotta be the first useful application of the hardware that I've
heard of! I've tried Kinect before and it might be just me, but I find it to
be horrible for gaming. Give me physical keys and I'm all happy. But of course
that doesn't appeal to the bigger audience of casual gamers. Nintendo got that
right with their, now dust-covered, Wiis.

~~~
kvb
There have been a few other interesting applications, such as use by surgeons
where physically using a computer would be unhygienic (e.g. see
[http://www.zdnet.com/blog/health/xbox-kinect-helps-
surgeons-...](http://www.zdnet.com/blog/health/xbox-kinect-helps-surgeons-in-
the-operating-room/277)).

------
tn13
During one coruse project we had used Kinect as a Semaphore (the flag
signalling) reader.

~~~
steve19
I got to play with a prototype simulator at a Navy base where a Kinect was
used to allow cross training of helicopter pilots and those signaling guys who
direct the helicopter from the deck of a ship.

The signaling guy stands in front of a large wrap-around projector screen with
a Kinect to capture his arm movements, which were sent over the network to a
central server which ran the simulation. The pilot in the heli cockpit
simulator would see the guy on the ship. The pilot and signal guy could be
training on different sides of the country (or world).

------
volandovengo
It is a shame that things like this aren't made into shipping products.

Maybe at some point they will integrate research into the product groups so
that we can still really cool (self driving car cool) projects coming out of
Microsoft

------
spellboots
Another win for the NSA against the menace of deaf terrorists!

