
Real-Time AI: Microsoft Announces Preview of Project Brainwave - benryon
http://blogs.microsoft.com/ai/build-2018-project-brainwave/
======
daxfohl
I understand that the big problem with FPGA and big data is that the memory
bus ends up being the bottleneck anyway. So you're just left with a slow soft
processor and a couple microoptimizations on the processing side. How to
overcome that?

~~~
aseipp
Brainwave FPGAs sit directly on the NIC of the server; every FPGA essentially
acts as a 40GB/s passthru NIC to a host machine, and each one can talk to the
Microsoft SDN completely on its own independent of the host CPU, in order to
bind multiple devices together to create aggregate compute units. FPGAs in a
server are partitioned into multiple RTL blocks (so 1 FPGA can host/execute
multiple workloads concurrently, not unlike a cloud server), and the "outer"
RTL simply routes messages, shuffling packets between RTL blocks, or RTL
blocks on different hosts. Thus the CPU is more-or-less cut out of the fast
path in a lot of cases, and acts more like a control plane.

Brainwave can deploy to hybrid FPGA/CPU topologies, however (where part of the
design is on an FPGA, and part is on a CPU, and they coordinate over the SDN),
so the choice of underlying accelerator is flexible and they can scale
independent of available capacity.

Microsoft has effectively created a hardware microservice architecture for
their large scale FPGA deployments, because the capacity is essentially
elastic, and the hardware exposes miniature services that directly talk on the
network and coordinate to execute large-scale jobs.

~~~
daxfohl
Okay so speaking of FPGA as the next gen of AI is a bit misleading? Because
it's really just a networking optimization? (I ask because I am intrigued by
FPGAs and a bit disheartened if that is all there is to it, not because I'm
bashing anything.)

~~~
aseipp
Well, any "next-gen, hyperscale AI system" at the level of Microsoft is going
to be part hardware, part software, part infrastructure; the scale is simply
too large these days to short-change any one of these and remain competitive.

Microsoft's network and the decoupling of the FPGA from the host machine are
major aspects of Brainwave's success for them, but any successful project like
this, even with different specific choices -- is going to largely be sustained
off "boring" things like a solid networking backbone. Plus there are other
concerns at this scale, like power usage, so just thinking of it in terms
networking is misleading...

Microsoft has had major success with Brainwave and they seem committed to it,
but overall the field is advancing pretty rapidly and there will be lots of
competitors (not just FPGAs) in this space. And their needs and use cases are
unique, so drawing generalized conclusions is a little tough.

(As a disclosure, I work on DNNs-on-FPGAs for $WORK, but am speaking
generally.)

------
polskibus
I hope Intel will offer similar capability in the future without Azure (or any
other cloud provider) lock-in, with lower latencies, because you don't need to
send data to outside your facility.

~~~
aseipp
Dell is now selling rack unit servers with Intel FPGAs installed (Arria 10
line). But you're not going to get the software stack that backs Brainwave
without serious work on your own...

------
killjoywashere
Google I/O starts tomorrow. Coincidence?

~~~
manacit
Microsoft Build is this week as well (the keynote is going on as we speak).
Facebook F8 was last week. I guess this is developer conference season?

~~~
killjoywashere
Ah, thanks. I'm actually not in that world, so the general conference schedule
didn't occur to me.

