
Transitioning entirely to neural machine translation - darwhy
https://code.facebook.com/posts/289921871474277/transitioning-entirely-to-neural-machine-translation/?utm_source=codedot_rss_feed&utm_medium=rss&utm_campaign=RSS+Feed
======
GuiA
Will be interesting to see how it performs in the wild.

Here's a sample sentence that I've never seen any automatic translator get
right:

 _" My cousin and her wife"_

Any human would infer from the context that my cousin is a woman, in a same
sex marriage. Yet Google, who also uses fancy deep learning, gives us, for
Spanish:

 _" Mi primo y su esposa"_

This should be _" Mi prima y su esposa"_. And that's just for Spanish, a
language relatively close to English. With more convoluted examples and more
distant languages, it still breaks down pretty fast.

See also: examples of gender bias when using deep learning to translate from
languages with gender neutral pronouns:

[https://twitter.com/amuellerml/status/799658925326995456](https://twitter.com/amuellerml/status/799658925326995456)

[https://twitter.com/nobody_indepth/status/799700696572526592](https://twitter.com/nobody_indepth/status/799700696572526592)

~~~
hasenj
Actually as a human I did not immediately infer the lesbian relationship part.

------
AndrewKemendo
This is much more impressive than I think it will get credit for.

I mean really, having an always on, immediate access massive NLP DNN system
translating every piece of text from any language on the worlds largest
platform is a staggering feat.

Facebook has the most impressive applications of ML right now in my opinion.
They have Yann Lecun to thank for that (and Mark for recruiting him).

~~~
whatrusmoking
??? Google's been doing this at scale for a while.

~~~
dmcy22
Google only supports Neural Machine Translation for a subset of languages.
Whereas as Facebook is doing it for 2,000 translation directions.

~~~
Retric
45 languages * 44 other langues ~= 2000 translation directions. There are
~4,500 languages with more than 1,000 speakers ~4,500 * 4,499 ~= 20,254,500
language pairs so that's a very long way from every language.

~~~
hood_syntax
Frankly though, I imagine the vast majority of those languages are unappealing
targets in terms of return on investment.

~~~
flamedoge
and much smaller training data..

------
peterburkimsher
I made Pingtype, a site to translate Chinese to English word-for-word.

[https://pingtype.github.io](https://pingtype.github.io)

I also made an English-to-Chinese version,
[https://pingtype.github.io/english.html](https://pingtype.github.io/english.html)

One thing that struck me is that even the largest online dictionaries (Oxford,
Wiktionary) are totally inadequate. I'm continually adding new words. Another
thing I noticed is that spaces are not always between words! Chinese doesn't
have spaces at all, so I wrote a word spacing tool. When I rewrote the app for
English, I thought I could use the space character, but I can't.

Many verbs wrap around nouns e.g. "put [the phone] down". The source
dictionary data has a definition for "put something down", and my program has
support for a special word: "something", which causes it to look ahead for the
second half of the phrase.

Pingtype doesn't reorder the sentence, because it's intended for education.
But it's easy to train, unlike machine learning models.

~~~
toomanybeersies
I don't think that even on a trivial level it's possible to word-for-word
translate most languages, except the most closely related ones, since they
have very different language structures.

~~~
rpmcmurphy
And then there is the German word backpfeifengesicht, a personal favorite
along with schadenfreude.

~~~
visarga
You can still solve those with a dictionary of translations and for the
missing words, a sub-word embedding system that gleans the meaning from the
phonetic form.

------
toomanybeersies
It's always interesting to compare machine translation of French to my (albeit
limited) translation of French. I tend to prefer cognates, even if they're a
bit obscure, to more colloquial translations, since I think it preserves a bit
more of the original text.

For instance, "Drôle" in French is usually translated to "Funny", but I would
translate it to "Droll". I also try to keep the sentence structure closer to
the original, which does make my translations look like they were written by
Shakespeare.

Something that I really like in regards to the implementation of FB
translation is that I can select languages that I don't want to be
automatically translated. For French, German, and Spanish, I'd prefer to at
least have a crack at it before translating it, but for other languages like
Turkish, there's no point.

One thing I don't like so much though is that for automatically translated
languages, they replace the original text and don't make it super obvious that
it's a translation. I do feel like there's a bit of language colonialism going
on with that.

------
therein
Native Turkish speaker here. Very impressive, especially since the sentence
they display in the post is actually quite colloquial in its use of the
language.

------
otto_ortega
Is there a more detailed example of how such neural systems work?

Like, what are the inputs? how the architecture of the network looks like?

I ask because sentences could be of any length, so I'm not sure what's feed to
the model.

My only experience with NLP is setting a toy text classifier using naive-
bayes.

~~~
juanpino
You can read more about it in
[https://arxiv.org/abs/1409.0473](https://arxiv.org/abs/1409.0473)

------
make3
I'm surprised they weren't already fully neural. NTM (neural machine
translation) has been much better than everything else for a while now.

~~~
landon32
I'd imagine part of it is just having to test all the edge cases-- for every
language is their full neural better?

That, and also building a good pipeline for it to work at scale.

