
A Neural Network for Machine Translation, at Production Scale - sudoscript
https://research.googleblog.com/2016/09/a-neural-network-for-machine.html
======
runesoerensen
Previous discussion:
[https://news.ycombinator.com/item?id=12589377](https://news.ycombinator.com/item?id=12589377)

------
mastazi
ArXiv paper:
[https://arxiv.org/abs/1609.08144](https://arxiv.org/abs/1609.08144)

(Not yet on Arxiv Sanity Preserver, but should be added soon I guess)

~~~
mastazi
Update: it is now on ASP

------
iraphael
> In theory, both of the accumulators are unbounded, but in practice, we
> noticed their values remain quite small

This empirical fact is used to their advantage, but it is not proven that this
is generally the case (tbh, it might be nearly impossible to prove it). This
is okay, but it makes the model seem kinda yucky to me.

> All of the models use 1024 LSTM nodes per encoder and decoder layers.

Ah, as I was reading the paper I kept looking for how they generated a
variable-sized attention vector using a simple feed-forward network. It seems
that the LSTM is actually capped at 1024 nodes per layer, so the attention
vector only can be a fixed size of 1024 and truncated to the number of
unrolled steps.

------
Omnipresent
Are there any open source machine translation solutions that are remotely as
good? Only one I came across is
[http://www.statmt.org/moses/](http://www.statmt.org/moses/)

~~~
michael_h
It's more about the models than the framework. Moses is really great, and has
a few example models, but to get usable results you will have to build up your
own models.

Kaldi is another. I have a model around here getting a WER of about 5%.

------
philipkglass
This is a nice improvement. It looks like the translation backend for Chrome
hasn't been switched yet.

[https://www.chinadialogue.org.cn/blog/9258-The-debate-
over-t...](https://www.chinadialogue.org.cn/blog/9258-The-debate-over-the-
future-of-nuclear-power-in-the-UK-continues/ch)

One paragraph translated through the Google Translate site:

 _The Hinckley Point C nuclear power plant project has been controversial in
the UK, largely because of its high costs and unproven technology, and the
progress made in other low-carbon technologies means that nuclear power plants
may become redundant before completion. Energy demand in the UK is declining,
and as UK and European power grid connectivity continues to improve and
battery technology advances rapidly, it is becoming increasingly obsolete for
the UK to require new nuclear power plants to meet basic load requirements._

And from using the translate option inside Chrome:

 _Hinckley angle C nuclear power plant project in the UK has been highly
controversial, mainly due to their high cost and unproven technology; In
addition, the progress made in other low-carbon technologies, means that
nuclear power plants could be completed before it becomes redundant. Britain
's energy demand is falling, and as the United Kingdom and the European grid
Unicom continuous improvement and rapid advances in battery technology,
Britain needs new nuclear power stations in order to ensure that the basic
needs of the load argument has become increasingly obsolete._

Finally, you can see a human-translated English version here:
[https://www.chinadialogue.org.cn/blog/9258-The-debate-
over-t...](https://www.chinadialogue.org.cn/blog/9258-The-debate-over-the-
future-of-nuclear-power-in-the-UK-continues/en)

 _Hinkley Point C has been the subject of fierce controversy in the UK because
of its cost, its unproven technology, and advances in other low carbon
technologies that may make it superfluous even before it is completed. The
UK’s energy demand is falling, and, with improved interconnection with
European grids and the rapid advance of battery technologies, the argument
that Britain needs new nuclear for base load energy is increasingly criticized
as out of date._

Original for posterity:
欣克利角C核电站项目在英国一直饱受争议，主要原因在于其高昂的成本以及未经验证的技术；另外，其他低碳技术领域取得的进展，意味着核电站可能在完工之前就变得多余。英国的能源需求正在下降，并且随着英国与欧洲电网的联通不断完善以及电池技术的快速进步，英国需要新的核电站以保证基本负载需求的论点也显得越来越过时。

Note especially how the older translation essentially reverses the meaning of
one important phrase -- "nuclear power plants could be completed before it
becomes redundant" vs. "nuclear power plants may become redundant before
completion."

~~~
yborg
Impressive. I'd like to see what they can do with Japanese, Google Translate
remains notably bad there and essentially hasn't improved at all in the last
3-4 years.

