

Convert American sign language to Text - zedzan

I am working on an app to help children in need to communicate with their caregivers. Most of these caregivers don&#x27;t understand sign language.<p>I want to convert sign language to text and text to sign language. I know that this is an area of hot active research. I don&#x27;t want to re-invent the wheel, I wanna use this as feature in my app where users enter input-text or an audio (using the speech to text api) then translate the text into a sign language corresponding to the text entered.<p>I am looking for an API, engine, open source or research project to embedded it, or a database in a form that could be easily translated to animation in 3D (DB of word to sign language 3D animation )<p>Can you please recommend any helpful tool&#x2F;resource ?
======
lsiebert
How much do you know about ASL? I'm going to assume somebody reading this
doesn't know anything. I know I'm far from an expert in ASL or in linguistics.

In short, no I can't recommend, but I can suggest what some of the challenges
are.

First of all, you have to know ASL has a way to represent english characters
and numbers. This lets ASL speakers who know how to read English communicate
with people who know how to fingerspell. But it's no more ASL then the
alphabet is English.

ASL (and pidgins of English ADL such as SEE and PSE) is a language, and
language translation is difficult all on it's own. ASL has some ways to code
it to text, but it doesn't strictly have, AFAIK, a written form, which means
there isn't a way to serialize it.

There are also ASL conventions that make it difficult to do translation. For
example, in ASL I may establish a location, give it a description or a name,
and then use it as a shorthand when communicating.

ASL also has a variety ways to communicate what is often communicated by tone
or word emphasis. This can be in speed, size of movements, etc.

Also facial movements are important to ASL. They can be modifiers to signs, or
signs all on their own. Also signs flow into one another... it's visual
calligraphy. Beautiful, but it means that a translation system may need such
transitions handled.

So when you read about people converting sign to text, it's generally some
method of converting fingerspelling to letters with either a gesture capturing
glove, or a vision system. I know there was a MIT project
([http://dspace.mit.edu/bitstream/handle/1721.1/80082/43496321...](http://dspace.mit.edu/bitstream/handle/1721.1/80082/43496321.pdf?sequence=1))
that did sign to text more in depth, and maybe you could reverse engineer it.
I can't speak to text to sign, I know of no system other then websites with
short gifs/video clips.

I'm sorry I can't be more helpful. I know there are some deaf hackers on here,
you might see if they know about something, as I'm not hooked into the
ASL/Deaf community.

I am not convinced, even if you have "good enough" text to sign representation
of some sort, that you could generate movement. But good luck anyway.

