
GPU Platforms Set to Lengthen Deep Learning Reach - jonbaer
http://www.nextplatform.com/2015/12/07/gpu-platforms-emerge-for-longer-deep-learning-reach/
======
p1esk
It makes no sense to make "x10", or "x40" speedup claims, without mentioning a
specific benchmark. I'm guessing the code that scales well with 200 or even 12
GPUs is fairly rare. For example, problems like ImageNet classification with
convolutional NNs don't scale well beyond 2-4 GPUs. At some point, you will
actually get a slowdown, because of the communication overhead.

------
amelius
But why do we (they) keep on calling them GPUs?

And why not improve their form factor? Currently, a GPU hardly fits into a
server slot. And a typical desktop can host at most 2-3 of them.

As a radical idea: why not design this type of auxiliary computing power as
separate boxes that can be attached to a desktop or server computer by a
cable?

~~~
mrb
We call them GPUs because they are built from GPU chips. The Tesla M40 for
example is based on a GM200 chip which is the same as in a GeForce GTX Titan X
GPU.

Also Nvidia and other assemblers have done exactly that: separate boxes that
you attach to servers by a PCIe cable. See for example the Tesla S2050 server:
[http://www.nvidia.com/docs/io/43395/nv-ds-
tesla-s2050-june10...](http://www.nvidia.com/docs/io/43395/nv-ds-
tesla-s2050-june10-final-lores.pdf)

A good list of Tesla systems is at
[https://en.wikipedia.org/wiki/List_of_Nvidia_graphics_proces...](https://en.wikipedia.org/wiki/List_of_Nvidia_graphics_processing_units#Tesla)
(which, coincidentally, I have been heavily updating these last few days).

