Hacker News new | past | comments | ask | show | jobs | submit login
MirageOS: A Unikernel For The Xen Hypervisor (openmirage.org)
99 points by godisdad on May 10, 2014 | hide | past | favorite | 52 comments

This is an awesome idea. Every now and then a revolutionary concept comes along that points the way to all future computing. This is the future of any public facing computer. You must understand the concepts here if you are involved with web development at any level. If you haven't yet wrapped your head around this, then you should.

The key concept here is about security.

Imagine you could compile your web application/website into an application that has nothing else but the required network drivers and minimal hardware drivers needed to execute. That is (as far as I can tell from a brief look) what Mirage does. Using such an approach there's nothing for hackers to log in to - no shell to use, no other insecure cruft on the system potentially leaving holes. The attack surface of such a system is extremely low relative to putting fully powered and configurable interactive operating systems onto the Internet.

Hackers are showing that almost everything is hackable without expensive, constant, expert attention to security of every aspect of your systems. Even then zero day exploits expose systems within seconds or minutes of the exploit becoming known.

This is not to say that such systems cannot be hacked, but rather to say that in such an approach the attack surface is dramatically lower than in current approaches to deployment.

Perhaps more likely to change the way that people think about the concept of web application development in the longer term is that the application/server allows for the possibility of "high resolution server instances" in which a virtual machine can be started in milliseconds, perhaps to service only one query, and then vanish again. Instead of renting your instance from Amazon that runs per hour and does nothing most of the time, your infrastructure will be truly scalable with nothing running at all until its needed and then precisely as much computing power will be used as is required to meet the inbound demand. No longer will you need to "start a web server" which then chugs and rumbles into existence as all its various unneeded-at-most-times processes start. Instead, connect your application/server AMI to the Amazon spot instance marketplace.

Holy crap, it's you! Sorry I tried to reset your password the other day. I thought I'd already signed up and forgotten.

On a more on-topic note however, aside from the benefits of running any kernel, such as windows, I wonder if there are any practical benefits of something like this vs something like CoreOS/docker. I'm sure since it's Xen you could run a windows kernel on Linux, but aside from that, I don't know.

> On a more on-topic note however, aside from the benefits of running any kernel, such as windows, I wonder if there are any practical benefits of something like this vs something like CoreOS/docker.

There are millions of lines less of code involved in the deployed Xen unikernel, since there's no Linux userspace/kernel divide involved any more.

You can also use the same application source code just fine with CoreOS/Docker if you prefer, since it can also compile to normal POSIX binaries that use kernel sockets (via `mirage configure --unix` instead of `mirage configure --xen`). This is the main benefit of using modular OCaml code that can swap out entire subsystems at compile time.

Another benefit is that these millions lines less also translate into file sizes.

A bare-bones MirageOS deployment (e.g. a hello world TCP service) is only a few MB if I remember correctly.

It also boots faster than your typical Linux kernel + userspace. That probably only makes a difference if you boot up a lot of short-lived VM instances or trying to recover from a failed instance though.

In terms of 'lines of code running' wouldn't a stripped down Linux kernel + Mirage application in direct mode beat Xen + Linux dom0 HW drivers + Linux dom0 Xen daemons/tools + Mirage unikernel?

I do agree with you that Xen is probably better at providing isolation than Linux, i.e. it is better at what an OS is supposed to provide :)

Also if you are in an environment where your only option is to deploy Xen domUs, i.e. like EC2, then you probably have a performance advantage as well because you just eliminated one layer.

In the end not much changes conceptually compared to a traditional application:

  * instead of being linked with libc it is linked with Mirage's runtime
  * its "OS" is now Xen instead of Linux
  * the drivers of the OS are unchanged (running Linux in dom0)
  * developing a Mirage unikernel is much like developing a traditional application, if you restrict yourself to the Mirage provided interfaces

> In terms of 'lines of code running' wouldn't a stripped down Linux kernel + Mirage application in direct mode beat Xen + Linux dom0 HW drivers + Linux dom0 Xen daemons/tools + Mirage unikernel?

Well, you can drop into a driver domain model and not have a full Linux dom0 (if you don't mind fixing on a particular hardware model).

But don't forget that Mirage is about modularity though -- we have a kernel module version under development too, and a baremetal rPi one. The idea is that as the number of libraries grow, it becomes easier to pick and choose the set you need for the particular deployment environment you want to build an application on (including but not exclusively a Xen unikernel).

That sounds interesting, I'll keep that in mind.

Holy crap, it's me! I did get the password reset in the middle of the night and wonder what that was about then I saw andrewstuart2 and realised what had happened. Seems we are the same person. Do you spend your life saying "It's spelled S-T-U-A-R-T", then people say "errrr, S-T-E-U-W-A-R-T?".

Haha the worst I've gotten is "Okay, S-T-U-W-A-R-T?". I suppose there has to be some downside to having some royal Scottish blood.

This thread had me really confused for quite a while. I thought it was one person replying to their own messages.

Indeed. One must be eternally vigilant with the name.

Are you Paul? My brother's name is Paul.

That is partly true. But if there is nothing to hack into, there will also be nothing to log in to for an admin or an ops person.

In the case of Erlang, for instance, it provides a built-in remote shell which allows one to connect to a running Erlang process, get multitude of information about the VM, loaded code, etc.

As long as there is a convenient way to maintain a running system, that same way can be used for hacking.

You don't always require a shell if all you need is info about the system. Such a logging process can easily be compiled into the application and that data regularly pushed elsewhere. If you need to change what's being logged then adjust the code and recompile/deploy a new Unikernel. I have a version of my website being compiled as a Unikernel and the workflow is very simple [1].

My point is that the whole workflow of creating and deploying applications becomes much simpler as the final VM can be an (almost) disposable item, rebuilt at-will. This is pretty much what the configuration management tools are all striving for but Unikernels work this way by design. Therefore, our notions of managing such systems will also change.

[1] http://amirchaudhry.com/from-jekyll-to-unikernel-in-fifty-li...

Oh, great blog post. I've been looking for an excuse to re-do my website -- right now it's running on a semi-custom Python blog engine.

I've been doing Project Euler in OCaml recently so this is a perfect excuse to expand my horizons.

Thanks! If you're interested in deploying to EC2 then you should also look over Mindy Preston's posts [1]. There will also be more work done on improving cloud deployment over the summer [2]. I expect it won't be too long before we can have a heroku-like experience/system for easily building and deploying Unikernels to third-party clouds.

[1] http://www.somerandomidiot.com/blog/categories/mirage/

[2] http://openmirage.org/blog/welcome-to-our-summer-hackers

Perhaps deployment and management practices will change to take into account this approach to management, rather than forcing the existing insecure approach onto dedicated applications/servers.

I wouldn't mind to have an SSH shell to MirageOS though. Perhaps something like TinySSH could be translated to OCaml?

It's security advantage to not have a shell or anything that someone can log in to. Perhaps export instrumentation data or provide instrumentation via an interface or something.

You mean a pseudo-shell? Because when you're running OCaml on top of a hypervisor, there is no OS and shell in the traditional meaning of those words. So nothing to connect to.

Yeah, I don't mean a classic shell or even the OCaml toplevel. But it would be nice to provide a "standard" way for applications to define their own commands, and to provide some builtin commands for monitoring (memory usage, open connections, etc.).

I'm afraid that going in the direction of using single-language platforms only draws us further for having a standard way of managing them.

Hmm, but there could still be a libraries that take care of the common tasks: how to talk to an SSH client, how to get some basic info from the Mirage kernel, etc. Since these unikernels are so specialized maybe SSH would be overkill, but I wouldn't want each developer to invent their own potentially insecure way of communicating with the unikernels.

We've got an SSHv2 client/server in pure OCaml already, but are waiting for the OCaml TLS effort to be integrated before putting the SSH layer in. It will be use the toplevel libraries and be optional...

The main issue I see is that everything has to be done in ocaml no? What about the maintenance overhead too?

The dedicated application/server is such a powerful idea and security is such a big problem that perhaps some time in the future people will see the idea of "directly putting an operating system on the Internet" as risky madness. All development environments that matter will somehow come to support creation of dedicated application/servers.

There is nothing stopping someone from doing something similar in a different language.

I made a start on modifying Movitz to build a Lisp system that would run on top of Xen.

For people interested in this I recommend the ACM article on Unikernels [1]. It's a good overview of the approach and the benefits/trade-offs. Then take a look at http://nymote.org to see the kind of things we want to start building with this approach.

[1] http://queue.acm.org/detail.cfm?id=2566628

See also: Erlang on Xen (http://erlangonxen.org/).

Also http://corp.galois.com/halvm.

Those three have in common the trait that they support single language runtime: Erlang for Erlang on Xen, OCaml for MirageOS, and Haskell for HaLVM.

Which seems sensible; if you want to write different components in multiple languages, each component can be its own VM and they can communicate over the (virtualized) network. It's the natively-compiled, statically-linked equivalent to putting your runtimes in Docker containers.

I was thinking, earlier, that this approach could be taken even further: rather than relying on Xen, it'd be nice to extract the drivers from the Linux kernel into their own project, an exokernel library with a defined ABI. Platform runtimes like Erlang, OCaml, Haskell, etc. could each build a "complete bare-metal kernel-binary" version of themselves, simply by linking to that exokernel library.

(And Linux could obviously move over to consuming the exokernel library itself. I guess a POSIX environment would technically just be the "complete bare-metal C kernel-binary.")

> if you want to write different components in multiple languages, each component can be its own VM and they can communicate over the (virtualized) network

Those "components" and "virtualized network" really resemble "processes" and "IPC" to me. Except that in MirageOS/Foo-on-Xen/etc each process has its own device drivers. And, indeed, those beg either for putting into a sort of shared component.

But... aren't we almost reinventing microkernels here? With the only distinction of using shared libraries instead of server processes?

Yes, we are. I wouldn't say re-inventing, though. Hypervisors are effectively an optimization of microkernels: modern ISAs (e.g. x86-64) have special instructions to make "hypercalls" faster than plain-old context switches, so any modern implementation of a microkernel would be forced by practicality to act as a hypervisor.

Though, there's another advantage that comes specifically from thinking of each process as a separate "machine": each process gets to participate fully as an Internet peer, with its own unique, public-routable IPv6 address.

You can extract the drivers from the NetBSD kernel into exactly that, its called rump kernel http://www.rumpkernel.org and it can run on Xen among other places see https://github.com/rumpkernel/rumpuser-xen or just in userspace under Linux say - it is getting easier to run any application on it, there is still a bit more work needed (thread support is not there yet). Working on it though, and it should get easier to build and run application code over the next few months.

One advantage of using a BSD rather than Linux is licensing, as linking Linux kernel code into your application will probably force you to release as GPL. NetBSD also has an initial portability advantage, as you could always cross compile it.

Also https://github.com/kerlnel but it doesn't appear to be active.

I don't work in a sphere that would make this relevant to me, but I think this paradigm is probably the future! "General purpose operating systems are dead!", proclaimed bjackman.

Anyway, also see: OSv: http://osv.io/ similar idea but with the JVM. I believe some pretty well known Clever People are working on it - can't remember who, though.. anyone know?

What looks like an alternative in .NET: http://cosmos.codeplex.com/

There was already a "MirageOS", for TI calculators:


One with a frustrating name. It's a shell, not an OS. One of my projects is an OS for TI calculators, and loads of people don't actually understand what that means.

I understand how using type-safety and streamlined modular systems might be good for security, but I fail to see the point in letting developers be closer to the bare metal. All the security abstractions built into traditional OSes are not there for no reason. Why should I trust the OCaml stack more than the alternatives?

This looks like something a really educated/disciplined company could use for creating some crazy cool infrastructure. It's hard to see it taking off for the average Joe.

I think you answered your own question (to some extent). Rewriting the necessary protocols/libraries in a type-safe manner already provides an improvement over the current systems. In addition, (with the standard approach) it's quite hard for a developer to reason about the whole stack they're using and privilege escalation bugs are particularly irksome. Writing the applications with the approach Mirage takes means that there is less code to reason about.

An anecdote I use when describing the benefits is the story of a smart fridge that got hacked and became part of a botnet sending spam emails. Why did that fridge even have code that allowed it to send email? It wasn't necessary for its functioning. We should write software differently if were going to be deploying it to 10x the number of devices compared to today.

More personally, I worry about the software that's going to find it's way into the embedded health devices of the future (cf pacemakers). These devices will inevitably be 'connected' and I want to make sure that the code they use is safe and secure.

> Why did that fridge even have code that allowed it to send email?

If it has a remote code execution vulnerability, it's trivial to make it send spam (or do all kinds of things) whether a MUA was already present or not.

OP here. The Unikernel paper someone linked about has more, but I think the main ideas are: incidental complexity mitigation and performance.

It's much easier to reason about a bunch of OCaml code, so say the authors, than it is to understand the interaction between independently changing pieces of your "stack" that are written in different languages and integrate in disparate ways.

There's only room for one MirageOS in my heart, and it runs on the TI-83+ http://www.detachedsolutions.com/mirageos/

Tried to read the paper mentionned on the website, and wondered : does "libOS" /unikernel concept means that the application is built as a kernel extension, and thus runs in kernel land ( so no context switching occures) ? Is this an explanation for the performance gains ?

When you link the application with the Mirage runtime (what you refer to as "libOS") you create a new kernel. You then run that kernel directly on Xen (like you would run a Linux domU), so yes they run in kernel land.

I'm not convinced that 'Xen + Mirage unikernel' has a performance advantage over 'Linux baremetal + Mirage direct mode', but I definetely see an advantage compared to 'Xen + Linux domU + Mirage direct mode'.

This seems like the rube goldberg version of chroot to me.

Is there anything similar for KVM?

I don't think so. KVM provides something much more like raw hardware (provided by qemu) while Xen provides a slightly higher level interface, and an example "OS", mini-os to run on it. Your best bet would be to use a virtio based driver set and a minimal initialisation but it will still have to do more.

According to the FAQ it should be possible to port it to KVM: http://openmirage.org/wiki/faq

Sure its possible.

You have to also remember there is a historic connection between the Cambridge OCaml people and Xen, with some of Xen being written in OCaml.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact