
LegoOS: Disseminated, Distributed OS for Hardware Resource Disaggregation [pdf] - ingve
https://www.usenix.org/system/files/osdi18-shan.pdf
======
lostgame
I smell a lawsuit, or at least a C&D.

I used to work for LEGO. They have more lawyers than I've ever seen anywhere
else.

~~~
m4tthumphrey
I'll call your LEGO lawyers and raise you Disney lawyers.

~~~
nolok
Let's not forget IBM's Nazguls

------
kornish
Adrian Colyer has an excellent write-up of this paper here:
[https://blog.acolyer.org/2018/10/22/legoos-a-disseminated-
di...](https://blog.acolyer.org/2018/10/22/legoos-a-disseminated-distributed-
os-for-hardware-resource-disaggregation/)

------
Ice_cream_suit
A cease and desist letter must be in the mail.

------
markhahn
It's unfortunate that so many are drinking the disaggregation koolaid.

FACT: you cannot disaggregate CPU from memory. Of course, that's the component
you most want to do. Disaggregating IO devices is no big deal (and obviously
doable with a monolithic kernel, since we have pcie hotplug today.)

Why? The fact that DRAM latency has been around 50ns for a decade is probably
the most important performance limiter on any systems today. Exchanging 50ns
local DRAM for 600-1000ns network access is profoundly stupid. Caches? Sure,
they're great, but we already have very large caches that hardly ever take
non-compulsory misses.

In other words: bandwidth is easy; latency is hard.

This is not news, but it's why your datacenter won't have racks of ram and a
racks of memory-free cpus, and a racks of storage.

------
detaro
Interesting, although I find it hard to get a good idea about performance from
the few benchmarks presented. Especially since the tested working sets are
quite small, whereas any serious usage of something like this would likely be
with problems that are a lot larger: could the networking keep up if it has to
handle a few hundred GB, not MB? Given the speeds modern networking hardware
can reach it's possible I guess, especially since the local cache sizes also
could grow appropriately.

------
DyslexicAtheist
_" InfiniBand will soon reach 200Gbps and sub-600 nanosecond speed [66], being
only 2× to 4× slower than main memory bus in bandwidth."_

that's absolutely incredible!

~~~
DyslexicAtheist
LOL InfiniBand was so ahead of its time. Such an old yet amazing technology
that made a comeback thanks to cloud computing.

------
als0
Can anyone summarise how this is different from Barrelfish?

~~~
zekrioca
It is written on the paper:

> "Section 7 > Multi-Kernel and Multi-Instance OSes. > > Multi-kernel OSes
> like Barrelfish, Helios, Hive, and fos run a small kernel on each core or
> programmable device in a monolithic server, and they use message passing to
> communicate across their internal kernels. Similarly, multi-instance OSes
> like Popcorn and Pisces run multiple Linux kernel instances on different
> cores in a machine. Different from these OSes, LegoOS runs on and manages a
> distributed set of hardware devices; it manages distributed hardware
> resources using a two-level approach and handles device failures (currently
> only mComponent). In addition, LegoOS differs from these OSes in how it
> splits OS functionalities, where it executes the split kernels, and how it
> performs message passing across components. Different from multikernels’
> message passing mechanisms which are performed over buses or using shared
> memory in a server, LegoOS’ message passing is performed using a customized
> RDMA-based RPC stack over InfiniBand or RoCE network. Like LegoOS, fos
> separates OS functionalities and run them on different processor cores that
> share main memory. Helios runs satellite kernels on heterogeneous cores and
> programmable NICs that are not cache-coherent. We took a step further by
> disseminating OS functionalities to run on individual, network-attached
> hardware devices. Moreover, LegoOS is the first OS that separates memory and
> process management and runs virtual memory system completely at network-
> attached memory devices."

------
sergius
Reminds me of Plan 9.

