Hacker News new | comments | ask | show | jobs | submit login

Nice; it only took them 7 months to catch up to amazon:

http://www.nikkostrom.com/publications/interspeech2015/strom...




For others who may be interested in the details despite the uninformative tone of this comment: The Amazon paper is about a specific tweak to learning rates for better scalability when doing distributed training. The core principles of distributed DNN training are much older - for example, Dean et al. 2012: https://papers.nips.cc/paper/4687-large-scale-distributed-de... trained a model for Imagenet on 2000 cores, using the DistBelief framework that is the predecessor to TensorFlow.

The question of how to improve the multiple-replica scaling of distributed DNN training is very important, as is the question of creating usable, flexible, and high-performance abstractions in which to implement one. They're also fairly orthogonal. TensorFlow as an architecture focuses on the latter. One could imagine implementing the Amazon tweak within either TF or any other framework.


Funny comment! Except this is actually usable by developers with non-expert knowledge of neural nets




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: