
Baidu Takes FPGA Approach to Accelerating SQL at Scale - okket
http://www.nextplatform.com/2016/08/24/baidu-takes-fpga-approach-accelerating-big-sql/
======
sixdimensional
This is kind of amazing. I have often really wondered what would happen if you
basically created a directed acyclic graph/dataflow for data processing (for
example, how Apache Spark distributes processing/operations), and then
accelerated the operations using physical implementations in FPGAs. After all,
a SQL query, when optimized, is essentially a graph of operators that data
flows through.

You do have to pass your data through the accelerator to get the processing...
which potentially means huge volumes of data moving into this physical
processing layer (probably can be done in parallel over a network at high
speed) - I would assume this is why shared memory bandwidth was a problem.

This would provide some really interesting options though - imagine feeding
data from two disparate databases (say, Oracle and SQL server) in a data flow
into this thing - now you have accelerated cross database joins (as long as
you can handle the bandwidth and processing on the way in).

There was this post before on HN previously lamenting the state of tools for
working with FPGAs, and my related comment wondering if what Baidu has done
here was possible:

\-
[https://news.ycombinator.com/item?id=9408881](https://news.ycombinator.com/item?id=9408881)

\-
[https://news.ycombinator.com/item?id=9410160](https://news.ycombinator.com/item?id=9410160)

~~~
anonetal
There is quite a bit of work on this topic in Databases, and stream processing
is one of the primary use cases they identified (in particular, if you can put
the FPGA device between the network and the CPU, it works out quite well).
This is one of the early papers:

[http://dl.acm.org/citation.cfm?id=1687730](http://dl.acm.org/citation.cfm?id=1687730)

This one appears to be a more recent survey paper by some of those authors:

[http://www.morganclaypool.com/doi/abs/10.2200/S00514ED1V01Y2...](http://www.morganclaypool.com/doi/abs/10.2200/S00514ED1V01Y201306DTM035)

As someone else pointed out, this was also one of Netezza's differentiation.

~~~
sixdimensional
That book likes like an excellent resource. Thank you for sharing.

------
smilliken
For GPUs, not FPGAs, but PostgreSQL has PGStrom[1] for offloading large table
scans. Still a work in progress.

[1]
[https://wiki.postgresql.org/wiki/PGStrom](https://wiki.postgresql.org/wiki/PGStrom)

~~~
ksec
CUDA Only. Is PGStrom in Active development? Or is it more of an proof of
concept stage?

~~~
snaky
Looks active at github, [https://github.com/pg-
strom/devel](https://github.com/pg-strom/devel)

------
DigitalJack
I'm an FPGA/ASIC engineer that does some software as a hobby (mostly clojure,
some python. C and Pascal if you go back far enough).

Although I focus more on verification these days, I've done ASIC design for
about 16 years.

I'd be very interested in working with anyone on figuring out if we can make
something better by leveraging hardware.

~~~
pldrnt
I've been working with FPGAs for 4 years and shortly on ASICs before that, did
software in a previous life (and still do), do you have something specific in
mind?

~~~
DigitalJack
No, I personally don't. I was into reconfigurable computing for a while, sort
of where I think intel is going with their combined CPU/FPGA project. Just In
Time logic configuration.

But I figured I would throw my offer out in case there were people thinking
they could solve or enhance something with an FPGA, but didn't have the
experience to get started.

------
koolba
This looks pretty cool. Any idea of which database they're doing this on?

I don't see it listed out in the article anywhere. I would guess it's a FOSS
database (i.e. either Postgres or MySQL) but also wouldn't be surprised if at
their scale they've created something entirely in house.

~~~
turingbook
MySQL is dominant in China Internet companies.

~~~
Diederich
That's pretty interesting. Do you have a source for that?

~~~
stevesun21
NOTE: most of these website are in Chinese

Taobao
[http://mysql.taobao.org/index.php?title=首页](http://mysql.taobao.org/index.php?title=首页)
360buy [http://www.infoq.com/cn/articles/exploration-of-
distributed-...](http://www.infoq.com/cn/articles/exploration-of-distributed-
mysql-cluster-scheme)

Tencent [http://tencentdba.com](http://tencentdba.com)

DiDi
[http://os.51cto.com/art/201604/508514.htm](http://os.51cto.com/art/201604/508514.htm)

~~~
snaky
"Tencent is using PostgreSQL for data mart in their big data system" \-
[https://pgconf.ru/en/2016/89691](https://pgconf.ru/en/2016/89691)

------
trhway
such FPGAs seems to be on the edge of what is export controlled, recently even
regular Intel CPU got blocked for delivery to China :

[http://www.pcworld.com/article/2908692/us-blocks-intel-
from-...](http://www.pcworld.com/article/2908692/us-blocks-intel-from-selling-
xeon-chips-to-chinese-supercomputer-projects.html)

and around an year ago some Russian guys got busted for selling the FPGAs to
Russia (and those FPGAs were less powerful):

[http://kron4.com/2015/03/21/sf-business-owner-arrested-
for-a...](http://kron4.com/2015/03/21/sf-business-owner-arrested-for-
allegedly-smuggling-programmable-electronics-to-russia-laundering-millions/)

~~~
eva1984
Baidu has an office here in US, they could build it here...I guess

------
astrodust
It's no wonder Intel made a big FPGA acquisition lately. This sort of
acceleration has enormous upside potential, _especially_ if this sort of thing
comes part and parcel with your CPU.

~~~
snaky
Intel is not alone - [http://openpowerfoundation.org/blogs/fpga-acceleration-
in-a-...](http://openpowerfoundation.org/blogs/fpga-acceleration-in-a-
power8-cloud/)

> In addition to the CAPI 2.0 coherent links running atop PCI-Express 4.0,
> there is a further enhanced CAPI protocol that runs atop the 25 Gb/sec
> Bluelink ports that is much more streamlined and we think is akin to
> something like NVM-Express for flash running over PCI-Express in that it
> eliminates a lot of protocol overhead from the PCI-Express bus

[http://www.nextplatform.com/2016/08/24/big-blue-aims-sky-
pow...](http://www.nextplatform.com/2016/08/24/big-blue-aims-sky-power9/)

------
nl
MapD does some of this, but with GPUs:
[https://www.mapd.com/product/](https://www.mapd.com/product/)

------
tkyjonathan
Finally! In 2011, when I was working on optimizing 100Gb databases on
dedicated hardware for small businesses, everyone in the database world was
talking SQL chips and PCI-E fast storage. Fast forward to today when cloud and
virtualisation and now I'm working on "big data" 400Mb RDS instances. Its time
we go "retro" and #gophysical for some things.

~~~
discodave
If anybody has the resources to start building dedicated FPGA (or whatever)
instances for SQL workloads it would be one of the public cloud vendors (AWS,
Azure, GCP).

~~~
tkyjonathan
Good! They will have to set aside servers that do just databases with SQL
chips, but either way, I would be happy if they do that.

I would also be interested to see if there is a new trend to go back to owning
commodity hardware, putting CoreOS+docker on it + a database with a hopefully
inexpensive SQL chip. Moving away from VMware, massively expensive servers,
massively expensive SANs, proprietary backups..

Would be very interesting to see.

------
tmostak
Very cool work but since its easy to get memory-bandwidth bound with SQL I
would wonder if it would make more sense to use GPUs with 10X the bandwidth. I
know FPGAs will be getting HBM as well so this might help.

Disclaimer: I work at MapD, a GPU database company.
([http://www.mapd.com](http://www.mapd.com))

------
CodeSheikh
Online stock brokers and finance companies have been using FPGAs for quite
some time now because of latency as a microsecond can literally cost them
millions. It is good to see these beasts making their way into crunching "Big
Data" in non-finance domain.

------
srigan
isn't this already implemented in Datawarehouse appliances like Netezza (now
part of IBM), which targeted solving the problem of analytical queries on huge
sets of data??

~~~
liprais
I guess Baidu hired some former Netezza guys and they decided to copy/steal
from their former company to meet with their KPIs,which is quite common here
in China

~~~
weq
Ahh yes, my old boss was next door neighbours with the Netezza CEO back in the
early days circa '06\. I worked a little with the platform back then, and i
saw this article and was like, thats they they did! Back then, u had to shard
ur database to take advantage of the parrallel processing, not sure how far
its come now.

------
kiril-me
They are crazy! Very interesting case. How they support such DB? As I know
FPGA has limits on updates.

What tools they use to create relational algebra?

~~~
chollida1
> As I know FPGA has limits on updates.

HFT firms have been banging on the drum of making FPGA's infinity programmable
for a decade and lots of commercial FPGA's can be re-flashed now without
worrying about the number of times you do it.

I've seen re-flash counts up to 100,000 times.

Hey put this in the "what has HFT actually done for the world" list of
answers:)

~~~
astrodust
I've never seen an advertised limit on FPGAs. You'd probably grow old and die
before you ran out of reflashes since that process takes a fair amount of
time. 100,000 flashes at even five minutes each is over a year of relentless
flashing.

~~~
rhizome
Is there a Jenkins plugin for that? :)

------
malkia
Naive question, but wouldn't this work only with ECC-enabled graphics cards
like NVidia Quadros? Possibly that's the only reasonable choice for servers,
but the price is much higher.

~~~
astrodust
They're using a Xilinx FPGA card, not a GPU. These are two totally different
things, though the FGPA card is usually a lot more expensive than any GPU on
the market.

------
srigan
isn't this already implemented in datawarehouse appliances like Netezza (now
part of IBM)?

~~~
sixdimensional
Most likely I believe the answer is yes. But since these appliances have been
proprietary, not much is publicly known about what they do internally. I
suspect that some design like the one we are seeing here at Baidu has been
their secret sauce for some time.

~~~
miguelbar
It's no secret. Netezza has been using FPGAs since the beginning. Here is a
link to a paper that describes the Netezza approach:
[http://www.redbooks.ibm.com/redpapers/pdfs/redp4725.pdf](http://www.redbooks.ibm.com/redpapers/pdfs/redp4725.pdf)

The Baidu approach seems very similar.

