
L4 microkernels: The lessons from 20 years of research and deployment - snvzz
https://ts.data61.csiro.au/publications/nictaabstracts/Heiser_Elphinstone_16.abstract.pml
======
Animats
The paper doesn't mention QNX at all. QNX is similar to early L4, but they've
taken slightly different paths.

L4 got rid of "long message passing", in favor of shared memory and interrupt-
like IPC. This is great for the kernel - no copying delays and no buffering
problems. But it means that communicating processes have to share some memory
pages and cooperate properly. If the shared memory page is something like a
chain of linked buffers, one side may be able to screw up the other side. If
that's how you talk to the file system, it may be possible to attack the file
system process that way. This won't crash the kernel.

Because L4's API is so limited, it's used mostly as a VM, with the usual
bloated Linux kernel on top. QNX has only a few more kernel features than
modern L4, but it can implement the POSIX API with just support in the C
library. When you call "write()" from C/C++ in QNX, the C library makes a
MsgSend request which is directed to a file system service, device service, or
network service. The code for this is tiny. When you call "write()" from C/C++
on L4, you're usually just making a Linux system call to an ordinary big Linux
in a VM. Eventually that Linux talks to other processes which do the physical
disk I/O. The problem is that the giant Linux kernel is still there, just a
bit more isolated from the hardware. The total amount of code that can break
has not been reduced significantly.

~~~
derefr
How do these compare to Mach IPC in OSX?

~~~
rayiner
Note OS X mostly doesn't use Mach IPC for anything. Calling, e.g., write()
results in a system call into the BSD portion of the XNU kernel, not all that
different from what happens on Linux.

~~~
simscitizen
That is completely untrue. While the BSD syscalls are supported, Mach syscalls
and Mach IPC are also heavily used, especially internally at Apple.

Just one simple example: run "sample <process>" (where <process> is any Cocoa
application) on OS X. Pretty much every thread will have an event loop powered
by CFRunLoop, and they'll all be stuck in mach_msg_trap waiting for an event
to occur.

Also, pretty much any sort of semi-complicated IPC on OS X is done over Mach
IPC (e.g. passing a large bitmap copy-on-write from one process to another).
Take a look at the ipc directory in the Chromium source, for example.

~~~
rayiner
Yes, I'd assume it's more heavily used in the higher-level application layer.
I was talking about basic UNIX functions.

~~~
swolchok
Even Unix signals are delivered as Mach exceptions first. If the process
doesn't have a Mach exception handler set up, the kernel's exception handler
converts them to Unix signals and tries delivery that way:
[https://github.com/opensource-
apple/xnu/blob/27ffc00f33925b5...](https://github.com/opensource-
apple/xnu/blob/27ffc00f33925b582391b1ef318b78b8bd3939d1/bsd/uxkern/ux_exception.c#L428)
and [https://github.com/opensource-
apple/xnu/blob/27ffc00f33925b5...](https://github.com/opensource-
apple/xnu/blob/27ffc00f33925b582391b1ef318b78b8bd3939d1/bsd/dev/i386/unix_signal.c#L775)

------
shandor
I'm having really big expectations on seL4. The modularity of a microkernel
with the security of formal methods is something that would give hope of
solving some really fundamental problems with the "IoT scene". I.e. having
good security out of the box (which is horrible in IoT as of now), while still
maintaining good flexibility regarding platforms (which will become more and
more of a problem in the future when IoT devices start actually
proliferating).

Edit: oh, and you can still run seL4 as hypervisor and run Linux on top of it
to get the usual Embedded Linux stack where your CPU has enough juice. And
still keep the critical systems safe behind seL4's capability system. Best of
both worlds? Dunno, hopefully seL4 will be there someday in production
quality.

~~~
tptacek
L4 is great, but it doesn't solve application security problems. L4 by itself
doesn't do much, and whatever you build on top of it --- if you're not
yourself using formal methods, which very few real product teams do --- isn't
going to inherit its security.

~~~
mafribe
True, seL4 verification doesn't solve the application level security problem
but it makes it more approachable in several ways.

\- No longer need the verification of application make assumptions about the
semantics of the operating system. Instead, concrete and verified semantics
are available. This makes application specification and verification easier,
and safer. Fortunately, verification is cumulative

\- The seL4 project has driven the state-of-the-art of verification tool
forwards by a considerable degree.

\- The seL4 project has lead the thinking about proof engineering [1], the
emerging field that that is to verification what software engineering is to
programming, addressing the question how to develop, maintain and evolve
large-scale proofs.

\- The existence of something like seL4 also puts pressure on CPU manufactures
to provide usable formal specifications to their customers, so that we can
verify against rigorous CPU specs. CPU manufacturers have been loath to do
this (for various reasons).

The full verification of seL4 came a lot earlier (by about a decade) than I
thought possible.

[1] G. Klein, Proof Engineering Considered Essential.

~~~
tptacek
What's your sense of the number of IoT vulnerabilities that are due to
misconstrued OS semantics? Mine --- and I've done a bit of work here, but not
that much --- is that there aren't that many.

Really I think it depends on what you're doing. If the work you're doing is
fundamentally kernel work --- if it's _all about_ interacting with the
security boundaries of the hardware (not just "interacting with hardware",
like getting the bits from an RF codec, but manipulating hardware shared by
attackers, like the iPhone SEP) then L4 is a pretty huge security win.

Otherwise: the problem you have isn't your OS, but the programming language
you're using to build with.

~~~
pslam
The problem is it takes more than just implementing the "kernel" (as in low-
level access aka HAL) work under L4. You need to apply that
compartmentalization all the way through the stack, and even subdivide
applications into smaller chunks of responsibility.

What you typically get is a neat subdivision of all the HAL bits, but everyone
stops once they're plugging in applications. This is partly because there's
normally a large amount of shared/legacy code which needs importing, and
partly because people don't recognize the benefits of partitioning an app.

It's not all bad - at least you can be reasonably confident that one
compromised app, or part of the HAL, can't be trivially used to compromise the
rest of the system. But, as you say elsewhere here, if there's just one thing
you're doing, then all that effort didn't really improve things.

~~~
hinkley
I wonder if anyone will manage to create a tool for Rust that can map out code
partitions, or near partitions, based on access patterns of object ownership.
At some tipping point a little bit of data sharing could be refactored into
message passing and finish the job right.

------
nickpsecurity
A number of comments here cover how one would use something like this to
benefit security of real-world systems. After all, it's a building block you
have to combine with other things. This was demonstrated by team behind Nizza
in the paper below with examples including digital signatures, VPN's, and so
on.

[https://os.inf.tu-dresden.de/papers_ps/nizza.pdf](https://os.inf.tu-
dresden.de/papers_ps/nizza.pdf)

The first to get certified to high assurance under recent models and delivered
in products was INTEGRITY-178B. Like Nizza, they implemented desktops that
virtualized the machine with Windows/Linux partitions side-by-side with native
apps directly on separation kernel. Runtimes for Ada and Java subsets let you
write critical components without common errors from C. Special middleware
applies security policies to interactions between components.

[http://www.ghs.com/products/safety_critical/integrity-
do-178...](http://www.ghs.com/products/safety_critical/integrity-do-178b.html)

A recent product that's more accessible is the GenodeOS architecture that
builds hierarchical, desktop system on top of components like seL4, NOVA, and
Nitpicker GUI. They're dual-licensed with open-source available. Work in
progress.

[https://genode.org/](https://genode.org/)

It should be noted that seL4 itself is aiming for embedded. Many of the top
teams are also focusing on language and spec-level models for verification of
holistic properties. The isolation approach isn't enough for the level of
correctness they're aiming for. Anyone interested in such work should check
out Galois's project or CertiKOS.

[http://smaccmpilot.org/](http://smaccmpilot.org/)

[http://flint.cs.yale.edu/certikos/mcertikos.html](http://flint.cs.yale.edu/certikos/mcertikos.html)

------
TheMagicHorsey
If the goal is to provide a verifiably correct kernel, why not build that
kernel in something like OCAML so you can leverage a better type system and
use the existing verification infrastructure in that language?

~~~
jkot
I think you would also have to verify resulting binary, compiler, libraries...

It seems more manageable to verify a few KB of assembly or C

~~~
pjmlp
Not really, to verify C code you need to set the compiler and its
corresponding version in stone for the verification process, as UB can change
even between versions of the same compiler.

~~~
cmrx64
Not really true. See [https://www.nicta.com.au/publications/research-
publications/...](https://www.nicta.com.au/publications/research-
publications/?pid=6449) for how the compiler and its internal semantics are
completely removed from the chain for l4v.

~~~
pjmlp
Thanks for the link, I will have a look into it.

------
gravypod
Where can I get the source for these to look at them? I'd like to know why
they are all so long.

~~~
tom_mellior
Here's the famous verified one:
[https://github.com/seL4/seL4](https://github.com/seL4/seL4)

------
harry8
This is a republish of an old paper which was pure, uncritical review of the
authors' triumphs.

Look for the part where they identify their responsibility for the fact that
after 20 years and thousands of engineer hours at their disposal they still
don't have a microkerenel based operating system worth a pinch of anything.
I'm not trolling, I'd love it if that statement were false.

"For a successful technology, reality must take precedence over public
relations, for Nature cannot be fooled." \--Richard Feynman

~~~
kragen
In 2011, supposedly L4 passed a billion installs, including being the basis
for the iPad 2, the Motorola Evoke, and some Qualcomm phones:
[https://stackoverflow.com/questions/8405505/is-there-any-
app...](https://stackoverflow.com/questions/8405505/is-there-any-application-
of-l4-microkernel)

I don't have personal knowledge of these environments, so I wouldn't know if I
were wrong about this.

------
faragon
In my opinion, the lesson is that microkernels are too small (e.g. lacking
per-user/group security schemes), and monolithic kernels are too big.

~~~
felixk42
Microkernels are designed to let users implement those things on top of them.

