
A Trillion Edge Graph on a Single Commodity Node - jcbeard
https://www.nextplatform.com/2017/04/27/trillion-edge-graph-single-commodity-node/
======
thinkloop
Isn't computing on a single machine generally always faster? Clustering is for
handling more data than a single machine can hold, which requires paying
network costs. There are parallelization benefits, but CPUs have so many cores
now.

~~~
makmanalp
This. It can be faster per-node because of removed IO and abstraction
overheads, but perhaps not overall (and they allude to that fact). There are
other side effects too though - you're limited by how large hardware on a
single node can go, and what the cost/performance ratio is with _highly_
specialized hardware like the above or commodity machines.

People also might want distributed systems for other reasons - fault tolerance
is one. So as your SLA requirements go higher, you sometimes end up with
little choice but to pay that cost. Other times you don't.

They do cite challenges with scaling this up, but I think quite a bit of their
innovation is around just leveraging hardware better, which is a big enough
deal on its own.

~~~
yeukhon
But can we have a cluster of computers that look like a single computer
sharing the underlying hardware, because that's just basically linking a
remote device. I supposed special kernel has to be written to overlay.

EDIT: Looks like this is already a thing [1]

[1]:
[https://en.wikipedia.org/wiki/Single_system_image](https://en.wikipedia.org/wiki/Single_system_image)

~~~
makmanalp
That's not quite the issue - SSI (which I didn't know about!) sounds cool, but
it's a programming model that solves similar issues at a different abstraction
layer. The question that remains unsolved is, regardless of what abstraction
you use, whether the overhead of communication (latency, throughput) between
nodes is large enough to make things annoying, and whether it's possible to
work around this effectively or not. Stuff like InfiniBand along with tech
like RDMA is helping a lot with this, and hardcore database installations do
use this.

------
moab
Very interesting paper, but the experimental section confuses me. They compare
their system which preprocesses the graph into the tile-structure (a
compressed representation), but none of the in-memory systems they compare
against use a compressed representation. Is this comparison really apples-to-
apples? If you can reduce the size in-memory of the graph by 60%, then I would
expect an algorithm running on the compressed version to be faster than the
same algorithm running on the uncompressed version (assuming that decoding the
compressed adjacency-list is significantly cheaper than the cost of a cache-
miss).

------
jacobparker
Wasn't expecting to see the reference to the 9p file system!

------
douglasfshearer
The paper doesn't mention "commodity", and I'm not sure that one to four Xeon
Phi can be described as commodity.

Can the title be edited?

~~~
thechao
I live the Xeon Phi---I worked on the original Larrabee project---but this
article reads like an ad fir Phis.

~~~
douglasfshearer
The Larrabee project, and all the things it spawned [1], is fascinating.

[1]
[http://tomforsyth1000.github.io/blog.wiki.html#%5B%5BWhy%20d...](http://tomforsyth1000.github.io/blog.wiki.html#%5B%5BWhy%20didn%27t%20Larrabee%20fail%3F%5D%5D)

------
vertias718
so, any repo code available yet?

