
Google Team Refines GPU Powered Neural Machine Translation - Katydid
https://www.nextplatform.com/2017/03/20/google-team-refines-gpu-powered-neural-machine-translation/
======
visarga
TL;DR - It's the code from "Massive Exploration of Neural Machine Translation
Architectures", where they run extensive hyperparameter search using 250K
hours of GPU time.

Direct GitHub link:
[https://github.com/google/seq2seq](https://github.com/google/seq2seq)

Arxiv: [https://arxiv.org/abs/1703.03906](https://arxiv.org/abs/1703.03906)

------
rawoke083600
I'm surprised they used k80 and not something like 1080*. Surely they don't
need the double precision-performance ? Am I missing something?

~~~
tcwc
Using a Geforce card in a datacenter or "GPU cluster" setup voids the
warranty, which might be a factor for them at scale.

~~~
kuschku
That's invalid in the EU — maybe Google should move their GPU clusters to the
EU instead.

------
microcolonel
Seems it mishandles the difference between Premiers and Prime Ministers.

~~~
ilaksh
In what language pair going which way

~~~
microcolonel
In English, Justin Trudeau is not the premier of Canada, he is the Prime
Minister. The problem with the translation is that the machine translator
would need to know specifically about Prime Ministers as a linguistic
category, and know that Justin Trudeau is in that category. The Chinese side
of the translation does not encode the difference.

~~~
HappyTypist
With enough data the Chinese side can encode <head of state> [Justin Trudeau]
-> <head of state/premier> [Justin Trudeau]

~~~
yorwba
In fact, the Chinese text says 加拿大+總理+杜鲁多 which is Canada+<head of
state>+Trudeau. That's apparently enough context for the machine translation
to pick up on, producing the correct "Prime Minister" title.

The only flaw in the GNMT version I can find is that 此行, "this visit" is left
untranslated. (But I'm not a native Chinese speaker, so there might be more.)

------
braindead_in
Why not use DGX-1? Surely Google can afford a bunch of these.

~~~
Sephr
Because the DGX-1 is insanely overpriced. Just because you can afford
something doesn't make it a reasonable purchase.

~~~
mtgx
Last I checked (around the time Nvidia launched it), the DGX-1 seemed
"perfectly priced" against Intel's systems. As in they seemed to have priced
it in terms of how many Xeon chips you'd have to get to have equivalent
performance.

I guess that's one way to go if you value profits above anything else, but I
think Nvidia is making a strategic mistake. If it doesn't _have to_ price them
that high, then it shouldn't. Because otherwise it's just leaving a bigger
opening for Intel to enter the market in a big way.

I think a better long-term strategy would be to make it as hard as possible
for Intel to enter the market, and one way to do that is to have _reasonable_
but yet quite aggressive pricing for its GPUs.

Unlike Nvidia, Intel is going to be pressured to recoup the $31 billion it put
into Altera and Mobileye, so even if Intel will match Nvidia's new prices,
Intel will struggle to recoup that money, while Nvidia can do just fine
continuing to sell its GPUs.

~~~
kortex
And yet, (from what I've read) DGX-1's are getting snapped up faster than
Nvidia can make them. If you have a >6 month wait list, and people still want
them at that price, then supply & demand suggests you may even be _under_
-priced (of course Nvidia wants that market saturation).

~~~
braindead_in
1\. Clone DGX-1

2\. Rent it out.

3\. Profit!!!

I would pay for cloud DGX-1 servers.

