
Facebook to open-source AI hardware design - kungfudoi
https://code.facebook.com/posts/1687861518126048/facebook-to-open-source-ai-hardware-design/
======
vonnik
I'm glad they're open-sourcing this, but I have to say that making 8 GPUs work
together is not that big of a deal. Companies like Cirrascale are making up to
16 GPUs scale linearly with a blade.

~~~
dgacmu
It's irrelevant if you can't balance the cost, CPU, shared DRAM, and -
possibly most importantly - PCIe and network connectivity in and out of it. A
typical mid-range Intel Xeon has 40 PCIe lanes. A high-end GPU requires 16.
You can do the math. Facebook has decided upon a balance that works for them
and their workloads; it's quite likely that going to 16 GPUs in the chassis
resulted in overall worse utilization because of PCIe bandwidth limits, or
socket and QPI count, etc. What's a big deal is someone having gone and done a
lot of work to calculate and experiment with finding a balanced design that's
also cost-effective and easy to maintain.

~~~
spiantino
Actually the PCI express topology is configurable, which is one of the
innovations. You can put all 8 gpus on a single CPU bus or have 4 on each CPU
(but have to use QPI between them)

------
scott_s
_Big Sur is twice as fast as our previous generation, which means we can train
twice as fast and explore networks twice as large._

Can a machine learning expert comment on this claim? There are many domains in
computing where doubling your compute power does _not_ halve your execution
time, or allow you to double the size of your input. (For example, your
algorithms are worse than O(n), or you have hardware communication
bottlenecks.)

~~~
spiantino
Author here - in this case we are talking mostly about ~halving the time of a
single minibatch in training a variety of standard neural networks. That type
of training tends to be very hard to distribute. The servers also have 8 gpus
in a single host, which is an additional performance boost and isn't included
in the 2x above.

~~~
hiddencost
I'm curious why this claim (distributed minibatch is hard) continues to be
repeated. Am I misreading the state-of-the-art? There was a paper at
Interspeech this year [0] where Amazon showed excellent scaling (54X speed up
when distributed across 80 nodes) without loss of accuracy.

[0]
[http://www.nikkostrom.com/publications/interspeech2015/strom...](http://www.nikkostrom.com/publications/interspeech2015/strom_interspeech2015.pdf)

------
daveguy
Can we please not call open sourcing a build plan of off the shelf components,
open-source "hardware design". I don't think backblaze called their case
designs open source hardware. It just doesn't fit when we have open source
circuit board designs. How about open-source build specification instead?

~~~
nl
I think you are completely underestimating (or misunderstanding) what this is.

The Open Compute designs (which is where this will end up) aren't just a list
of parts. They include things like blueprints for custom chassis, circuit
boards for power supplies and even designs for solid state storage
devices[1][2][3]. The also include the source code for custom management
software[4].

While it is true that they don't generally design their own logic boards, it
is quite a long way from a list of parts.

 _I don 't think backblaze called their case designs open source hardware._

You'd be wrong about that: _This new Storage Pod performs four times faster,
is simpler to assemble, and delivers our lowest cost per gigabyte of data
storage yet. And, once again, it’s open source._ [5]

I think "open source" is completely appropriate here, and to be honest I don't
quite understand your objection.

[1]
[http://www.opencompute.org/wiki/Server/SpecsAndDesigns](http://www.opencompute.org/wiki/Server/SpecsAndDesigns)

[2]
[http://www.opencompute.org/projects/chassis/](http://www.opencompute.org/projects/chassis/)

[3] [http://www.opencompute.org/projects/power-
supply/](http://www.opencompute.org/projects/power-supply/)

[4] [http://www.opencompute.org/projects/hardware-
management/](http://www.opencompute.org/projects/hardware-management/)

[5] [https://www.backblaze.com/blog/backblaze-storage-
pod-4/](https://www.backblaze.com/blog/backblaze-storage-pod-4/)

~~~
daveguy
I wasn't saying it shouldn't be called "open source". I was saying it
shouldn't be called open source "hardware design".

~~~
wyldfire
Hardware design isn't limited to gates, PCB traces and ASICs. This seems
remarkably like hardware design to me.

As much as you might not agree, plans for building a birdhouse constitute
hardware design too.

~~~
avar
Plans for building a birdhouse are completely incomparable to what the parent
is talking about. This would be like me having an "open source" birdhouse
design where the birdhouse is made of glued-together Intel™ processors.

Sure I've opened the schematics, but unlike a birdhouse made from wood you
have zero chance of being able to build one from scratch on your own, or
sourcing the Intel™ processors from another vendor.

This is the same reason for why if I release the binary of a program produced
with a secret compiler as "open source" I'll be laughed at, it completely goes
against the spirit of the thing, which is that anyone should be able to change
any part of the thing I've released.

It's good that they're doing this. But it's completely incomparable to actual
fully open source hardware.

~~~
nl
Open source gives you the ability to fork the design. An Open Source birdhouse
made of Intel processors could be forked and made from AMD ones.

 _Sure I 've opened the schematics, but unlike a birdhouse made from wood you
have zero chance of being able to build one from scratch on your own, or
sourcing the Intel™ processors from another vendor._

This is some kind of confluence of the values of FSF captial-F "Free" style
software with the more pragmatic goals associated with open source.

~~~
avar
We've eclipsed the usefulness of this birdhouse analogy, but suffice to say
being able to replace one proprietary commercial vendor with another doesn't
make something open source.

    
    
        > This is some kind of confluence of the values of
        > FSF captial-F "Free" style software with the more
        > pragmatic goals associated with open source.
    

Well, pragmatically, if you needed to alter the hardware of this "open source"
hardware what percentage of the total electronic complexity would you be
working with? I'm going to aim for a conservative one-digit percentile.

There's a real and unambiguous line to be drawn in the sand here. If you get
"open source" plans for hardware that you can't replicate with third-party
fabrication & your own materials it's not open source. It's just a recipe for
assembling commercial proprietary components.

It's awesome that people publish those recipes, but don't call it open source
hardware and dilute the meaning of hardware that is genuinely open.

~~~
nl
Don't conflate 'electonic' with being the only valuable thing here.

Some of the most active areas of research in data center design are around the
mechanical design to maximize cooling.

------
dang
More at
[https://news.ycombinator.com/item?id=10712195](https://news.ycombinator.com/item?id=10712195).

------
sawwit
How come they aim for GPU technology rather than for neuromorphic computing
ASICs?

~~~
modeless
Contrary to what you might read in puff pieces from IBM PR, all existing
"neuromorphic" hardware is terrible. The neural nets that work don't look
anything like the brain, and neural nets that look like the brain don't work.

People constructing neural nets that look like the brain are engaged in cargo
cult science, because the truth is we have no idea yet how the brain works and
attempting to imitate it without knowing that is doomed to failure.

~~~
dangirsh
I'm curious to hear your opinions on Numenta's work on emulating parts of the
neocortex.

~~~
chimtim
A good measure of any of the techniques are the results. We see record object
recognition and speech recognition results. How many of these results are from
Numenta or IBM neomorphic chips? Most successes have been on deep learning
architectures over GPUs (and large datasets). While these fancy architectures
may have their applications, they have nothing to do with recent advancements
in last 3 years.

~~~
emcq
To be fair, the goal of those new computing architectures is not to advance
algorithms. That at best is a side benefit.

Those algorithmic performance improvements seen in DNN have come from improved
datasets and training. Numenta and IBM arent focusing on training AFAIK.
Google's Quantum Annealing [0] is the only hardware I'm aware of focused on
training, although there are rumors Nervana Systems may produce something [1].
I'm sure there are others; accelerating training of DNN isnt a particularly
new idea.

The goal of these other computing architectures is typically to provide lower
power, higher frequency/lower latency, or smaller form factor execution of
trained models, but there is a question of how much value they can provide
over more conventional chips to be worth the chip design costs.

However without these architectures becoming as mainstream as say a GPU, I
think we will continue to see advances come from the typical everyday
computer. The ML community seems to be much more democratic than others.

[0] [http://googleresearch.blogspot.com/2015/12/when-can-
quantum-...](http://googleresearch.blogspot.com/2015/12/when-can-quantum-
annealing-win.html)

[1] [http://www.nervanasys.com/about/](http://www.nervanasys.com/about/)

------
chimtim
Is this a machine with GPUs? You can order a similar looking design from
acmemicro since last 4 years atleast?

~~~
mryan
Do you think there might be a reason that Facebook invested time and money in
to producing this design, rather than buying from acmemicro?

------
nickpsecurity
Good work FB on another great deliverable for open HW configurations and
improved datacenter design! :)

------
guelo
I was hoping it was the AI that was designing the hardware.

