
Unikernels: Rise of the Virtual Library Operating System - brudgers
http://queue.acm.org/detail.cfm?id=2566628
======
amirmc
The projects themselves have progressed since this article and MirageOS is
doing pretty well [1]. There was also a recent HN discussion on an experiment
to create (and run) one VM per URL, which was built via MirageOS [2].

Although there are many applications of the unikernel approach, a longer term
goal that I'm particularly keen to work towards is the idea of distributed
personal clouds. Enabling people to run their own infrastructure, without
becoming sysadmins, would be pretty cool [3]. Of course, there are other ways
to achieve this but unikernels offer certain benefits.

[1] [http://openmirage.org](http://openmirage.org) (I'm involved with the
project).

[2]
[https://news.ycombinator.com/item?id=9263146](https://news.ycombinator.com/item?id=9263146)

[3] [http://nymote.org/blog/2015/brewing-miso-to-serve-
nymote/](http://nymote.org/blog/2015/brewing-miso-to-serve-nymote/)

~~~
jsprogrammer
Regarding #3, I'm working on a project [0][1] that might be able to achieve
that. It's not a direct goal of the project, but could be a subset of the
primary goal. The primary goal is to be able to take a system description and
be able to run it anywhere, be it public clouds or private hardware.

The base image is CoreOS (so, not a uni-kernel, but still pretty sweet). You
can search Docker Hub for containers and create multi-datacenter, (and soon)
multi-provider clouds. I do think uni-kernels may be the way to go, but AFAICT
the proper tooling just doesn't exist yet.

It's still in pre-alpha, so there are many amenities that I plan to add that
don't exist yet, but it will launch clouds (what I'm calling a set of related
CoreOS clusters), create fleet units, and start your applications.

If anyone's interested in helping work on it, let me know or send a PR. All
the code is free (MIT) and on github (link on top-right of site).

[0] [http://www.cloud-launcher.io](http://www.cloud-launcher.io)

[1] [http://www.cloud-launcher.io?simulator](http://www.cloud-
launcher.io?simulator) | Just a simulation; won't create machines; any
credentials will be valid

------
mwcampbell
I'm ambivalent about unikernels.

On the one hand, I applaud the effort to simplify the software stack and
promote the use of a safe language all the way down.

On the other hand, I wonder if running lots of small OS kernels on a
hypervisor is the best way to do this. Might it be better to simply run each
application as a process on a conventional OS kernel, perhaps with containers
for isolation? This post from Brendan Gregg argues pretty convincingly that
virtualization imposes a substantial performance penalty:

[http://dtrace.org/blogs/brendan/2013/01/11/virtualization-
pe...](http://dtrace.org/blogs/brendan/2013/01/11/virtualization-performance-
zones-kvm-xen/)

I understand that a key premise of the unikernel approach is that the millions
of lines of C code in a conventional OS kernel like Linux are a problem. But
in any real Xen deployment I'm aware of, Linux is running in dom0 anyway. So
why not take Xen out of the picture and run each application as a process in a
container on Linux, using raw sockets so you can still implement the TCP/IP
stack in OCaml? For deployments on Amazon, I guess the unikernel approach has
the advantage that you're running on top of Xen anyway, so you're eliminating
a layer. But for running on one's own servers or an ARM-based device like the
Cubieboard2, it seems to me that Xen is just complicating things.

I guess one benefit of Mirage's use of the OCaml module system is that we can
experiment with multiple deployment approaches and use whichever one works
best in practice. So there's nothing stopping me from running a Mirage
application on Linux with raw sockets rather than as a Xen unikernel.

~~~
avsm
> I guess one benefit of Mirage's use of the OCaml module system is that we
> can experiment with multiple deployment approaches and use whichever one
> works best in practice. So there's nothing stopping me from running a Mirage
> application on Linux with raw sockets rather than as a Xen unikernel.

That insight is spot on. The reason we're spending so much time on the clean-
slate MirageOS stack is that it's a codebase that will last the test of time.
It makes it possible to build code that's more future proof against changes in
deployment environments. It's Xen/KVM/vSphere today, JavaScript and Docker
tomorrow, and probably Unix sockets forever, all from the same codebase. Our
use of OCaml + functional programming also had a good trajectory with
practical formal verification, so the future of the stack is exciting.

------
hello_there
In general, I get the impression that the whole virtualization trend is just a
way to re-invent process isolation with newer technology.

~~~
armenb
The process isolation is very useful _but_ they've got a point, why we should
run million lines of code until we get to the point to run a single web
server? Have we ever asked our self why the operating system which its only
responsibility is running a web-server should take 300 seconds to boot?

~~~
jacquesm
Boot time is rarely an issue when working with long running daemons such as
web servers.

We run the millions of lines of code because web servers typically serve up
web applications which in turn depend on a whole host of infrastructure being
available. The millions of lines of code support all the possible hardware
that you could throw at it (and so you can save some if you compile your
kernel just for that purpose), and if you know exactly which services your
web-app will depend on then you can prune even more.

Specialization always pays off in terms of efficiency and it always costs in
terms of ease of expansion and adaptation to future needs.

~~~
nine_k
Boot time of a server is somehow important when you dynamically spin new
instances. If spinning a new instance were a sub-second process, you could
allow the number of instances you run (and pay for) to follow the demand much
more closely.

But here the boot time is a proxy metric of complexity.

It's great to have software that can do a wide spectrum of things. But your
server most often does a _narrow_ spectrum of highly predefined tasks. You
often go to great lengths to obviate and disallow any other activity on it,
for security reasons.

So it would be great if you took the universal software, somehow cut it to
your task set, and could throw away the unused parts (which are 90%+).
Unfortunately, this is not really attainable with traditional OSes. Unikernels
allow you to do just that—with some quite noticeable limitations, as of now,
though.

------
jclulow
It is somewhat disappointing to see the term "OS virtualisation" used to
describe what is really still just virtualised hardware; albeit with a less
traditional OS on top of the HVM hypervisor. It makes more sense to reserve
the term to distinguish container-based technologies, where what is
virtualised _is_ the operating system -- not the hardware it runs on.

~~~
avsm
The term "OS virtualisation" is just fine here -- it refers to the support
required to virtualise an operating system, which conventionally means the
device driver, scheduler and memory interfaces. Providing such a layer lets
any reinterpretation of "OS" still run above it.

Container-based technologies are far more nebulous in their definition, since
it covers individual operating system concepts that don't quite migrate across
distributions. FreeBSD jails, Solaris zones, LXC, MacOS X app sandboxes are
all container-based virtualisation, but differ greatly in their details and
specific interfaces.

------
armenb
This reminds me of Ling/Erlangonxen, unfortunately Ling is not a free-
software, it's opensource but not a free-software.

And the fact that your operating system is the actual language you're coding
with, reminds me of the fantastic project by Niklaus Wirth, The Project
Oberon. The project Oberon is a language and operating system you can learn
more about it at
[http://www.inf.ethz.ch/personal/wirth/ProjectOberon/](http://www.inf.ethz.ch/personal/wirth/ProjectOberon/),
[http://www.inf.ethz.ch/personal/wirth/](http://www.inf.ethz.ch/personal/wirth/)
and a video by professor Niklaus Wirth himself:
[http://bit.ly/1ClvKSX](http://bit.ly/1ClvKSX)

~~~
icebraining
How is Ling not free software? From its LICENSE¹, it's using the Berkeley
DB/Sleepycat license, which is considered Free by both the OSI and FSF.

¹
[https://github.com/cloudozer/ling/blob/master/LICENSE](https://github.com/cloudozer/ling/blob/master/LICENSE)

------
Merkur
Is there a project for a DBMS on MirageOS? I would relay like to try out
MirageOS on a web shop project of mine, but can't imagine to start writing a
database, or even a file system to store images..

~~~
avsm
There are three ongoing efforts:

1) The Irmin Git store, which is designed to run both on Xen unikernels and in
JavaScript. Thomas Leonard is working on integrating the latter, with very
promising results for a React-style interface in the browser. Thomas
Gazagnaire is integrating the Xen backend at the moment. Expect an
announcement of both of these just as soon as they're ready.

2) The Rump Kernel project has very promising results with MySQL running as a
unikernel, so we're working on integrating a MirageOS frontend with a MySQL
backend unikernel.

3) The Ctypes library (0.4) that's just been released supports Xen compilation
for OCaml bindings, so writing bindings to Sqlite would let you link in
directly with a unikernel; [https://github.com/ocamllabs/ocaml-
ctypes](https://github.com/ocamllabs/ocaml-ctypes) . There's an ORM we wrote
ages ago with this in mind:
[https://github.com/mirage/orm](https://github.com/mirage/orm)

So the db efforts are full steam ahead at the moment, but no fully released
one yet for production. Keep an eye on the blog at
[http://openmirage.org](http://openmirage.org), or drop a note to <mirageos-
devel@lists.xenproject.org> for your intended usecase.

One interesting advanced prototype that Dave Scott from Citrix built is the
XenStore database as a unikernel (that powers inter-VM communication) on top
of Irmin. Video:
[https://www.youtube.com/watch?v=DSzvFwIVm5s](https://www.youtube.com/watch?v=DSzvFwIVm5s)
and a presentation at the Xen Developers Summit:
[http://decks.openmirage.org/xendevsummit14#/](http://decks.openmirage.org/xendevsummit14#/)

~~~
mercurial
> 2) The Rump Kernel project has very promising results with MySQL running as
> a unikernel, so we're working on integrating a MirageOS frontend with a
> MySQL backend unikernel.

Interesting. The big issue I see from the proposed model (spin up your
application on-demand) is the need to retain a long-lived connection pool to a
"traditional" container/machine with a standard relational database (the cost
of reestablishing a connection every time would obviate any benefit of the
scheme).

~~~
avsm
That's true with a TCP connection and its three-way handshake. With a shared
memory vchan across two VMs, establishing a connection is less than a
millisecond (if that).

~~~
mercurial
Sure, but if you're using both a read/write master and a hot standby for read-
only queries on another machine, you're going to need TCP.

------
digi_owl
My first thought when reading about MirageOS was that they are effectively
cramming DOS into a VM.

