
Students Build a Glove to Translate Sign Language into English - cpeterso
http://www.fastcompany.com/3059616/these-students-built-a-glove-that-translates-sign-language-into-english
======
DenisM
The headline is wildly inaccurate, and the article papers over _huge_
problems. They did not build a translation device.

For one thing, ASL has certain morphemes that are expressed through use of
external objects, facial expressions, etc. Things other than hands.

Another, bigger problem is that ASL is not English. You can't just take a
word, translate it into a different language and assume you have a valid
sentence in the target language.

There is a very detailed explanation written in this letter [1], by the
linguistics department of UW. That is in contrast to the press office of UW.
Among other things: [...] Deaf signers who have viewed the demonstration video
of SignAloud do not recognize what is being demonstrated as ASL: [...]

Unfortunately (IMHO) this letter[1] got mixed with a lot of politics in it,
but the technical aspect is solid.

[1]
[https://catalyst.uw.edu/workspace/file/download/5c706e255e89...](https://catalyst.uw.edu/workspace/file/download/5c706e255e89d3eea9cc0a7d6ac1fb3e89c44cfdc54774630e7637ef0eb47f94?inline=1)

~~~
Jun8
"Unfortunately (IMHO) this letter[1] got mixed with a lot of politics in it,
but the technical aspect is solid."

Totally agree, question is why didn't the Linguistics Department finish the
letter after that section, i.e. why the convoluted discussions about audism,
appropriation and such? Not having a deaf person on your team or checking the
results with the Deaf Community is certainly to be criticized with respect to
the naivete of developing an accurate translation system but the letter treats
this as a colonial assault.

If I want to develop my own French to English translation system (I don't know
anything about French) should this be interpreted as my (perhaps) subconscious
feelings of superiority? Evidently so.

~~~
teraflop
If you built a system that translated word-by-word from a pidgin French into
English, and then tried to sell it as a tool that would unlock "the
fundamental right of communication" for those poor French people, then yeah, I
think you should expect some push-back.

~~~
notahacker
Sure, but if the linguistics department wrote a response which spent half a
page explaining why the app didn't actually interpret French very well and
might have benefited from a native speaker on the team and devoted
considerably more space to attacking the creators' motivation with specious
drivel about how it was "cultural appropriation" to take their language
"without permission", I'd think they were the ones displaying the arrogant
colonialist mentality.

~~~
teraflop
Well, I don't agree but that's probably because I don't think "cultural
appropriation" is an inherently specious concept, as you seem to.

Whatever else you might think about that letter, I thought the authors made it
very explicit that they were _not_ attacking the SignAloud creators'
motivation.

~~~
notahacker
I realise the letter starts with a disclaimer about not attacking the
creators, who seem to have been browbeaten into actually endorsing it, but it
also spends two pages bashing them with phrases like "a feather in the cap—for
members of the non-marginalized culture—hearing people", "reaping the
rewards...of cultural appropriation" and "this is exploitation" which clearly
apply to them more than to the PR office; internal consistency is not one of
its strongpoints.

FWIW, I don't think the concept of cultural appropriation is _inherently_
specious, but I do think many if not most accusations of cultural
appropriation are, and suggesting that the idea to demo a tech product as a
_potential translation gadget_ is "exploitation" is stupid and
counterproductive beyond belief. I mean, if a student at this university in
future has a choice between pursuing a neat idea for an app they think might
help deaf people or a different project, which way is this letter going to
nudge their decision? They could have made the same point about the
disadvantages of not engaging the deaf community with the project at its
inception and the resulting practical shortcomings by structuring the letter
as an offer of help...

------
jasimq
Something similar was done by some Pakistani students a few years ago. Here's
a video:
[https://www.youtube.com/watch?v=-CfaxbVvi80](https://www.youtube.com/watch?v=-CfaxbVvi80)
(sorry, no english translation)

------
franciscop
In my University a team built one using a Leap Motion in 24h, winning the Hack
For Good. Links (in Spanish):

\- Project site: [http://www.showleap.com/](http://www.showleap.com/)

\- Article: [http://www.ibertronica.es/blog/actualidad/show-
leap/](http://www.ibertronica.es/blog/actualidad/show-leap/)

------
AdamSC1
As a Sign student I've commented on this project a few times as I wanted to
give some relevant examples.

As others pointed out ASL is not simply English replaced with symbols but is
its own nuanced language. Many Sign speakers learn to read and write English
as well and so can use a bridge called PSE (pidgin sign english) which is
commonly used in the workplace, but it is certainly not ASL.

In ASL facial expression not only accounts for ones grammar but can
signifigantly change the meaning of the sentence, for example:

-I pay him on a regular basis -I always pay him diligently -I always pay with ease -I always manage to pay

These four sentences have very different meaning but can all be made with the
exact same ASL signs and only a different facial expression.

Beyond that the grammar, syntax and even word choice of ASL is not the same as
English. For an example, just the other day my ASL teacher who is deaf came
away with the sentence:

"There[directional], swim swim finish drive shop food room [#CM] eat burger
champ I"

In English this would be interpreted:

"I was swimming there (pointed at building down the street), then I drove to
the food court at the mall and ate the best burger."

There are a lot of things we don't use in English such as directional
pointers, or the fact that he finger-spelled CM as the initials of our local
mall to clarify what he meant. Then there are different meanings such as using
"finish" to list of a series of events, or "champ" to represent something is
the best he's had.

If you take a pair of gloves and stick them on an ASL speaker and simply spit
out English with no understanding of the language and no note of facial
expression - you're going to have a bad time. And you are likely unable to
easily build a system to translate ASL syntax to English as there isn't an
easy to use large corpus of samples.

Lastly, in the article the founders are saying how communication is a
fundamental human right and this is supposed to empower the ASL community. The
ASL community is a vibrant community, who can communicate (even surprisingly
well with non-ASL speakers). You'll certainly face a lot of backlash if you
suggest they are disadvantaged or lack a voice. Many deaf individuals will be
sure to tell you that they are not disadvantaged they just communicate
differently.

All in all, it's an interesting technology but I think if you move forward
with a technology like this for people to use it needs to be with the
involvement of that people and an understanding of their needs, culture and
language. For example, using this as input device for a computer may be a more
natural use case for a lot of ASL users.

~~~
DenisM
What are your thoughts on the concept of cultural appropriation?

