
IncludeOS: Run your C++ code directly on virtual hardware - ingve
http://www.includeos.org/
======
rmrfrmrf
Is the assumption here that scheduling, drivers, resource management etc. will
all be on par with or better than full-fledged operating systems? Seems like
what you might gain in being closer to the metal could easily be lost by
wheels that have already been invented.

~~~
nostrademons
I suspect it's that you won't need most of them in a typical application.

The idea behind unikernels is that now that we have pervasive virtualization,
most virtual machines are running only a single application anyway. So you
have an instance type for memcached, one for postgres, one for Node, one for
nginx, one for Redis, one for your JVM, etc. In many cases, these apps
_already_ include user-level scheduling and resource management, which often
works at cross purposes with the OS. In other cases, they could run much
faster if they implemented these specialized to their particular workload, and
didn't need to run through an OS scheduler that also needs to support
responsive desktop apps.

~~~
sillysaurus3
The reason this is a bad idea is because it restricts your ability to redesign
a system. If you want to add another program, you can't do it by definition.

Though I suppose the solution in that scenario is "Take it out of the
unikernel, then."

Still, if you rip out the fundamental abstractions of an OS, how would you run
an application that relies on e.g. fork? Or any application that runs a shell
command in any way?

My understanding is probably falling short. I should read up more on
unikernels.

~~~
nostrademons
No, the solution in that scenario is "Spin up another VM, and talk to it over
socket." That VM could be another unikernel server, or it could be vanilla
Linux that you talk to over sshd, Fabric, Puppet, whatever.

It's important to understand the context behind the current unikernel hype: it
assumes that you've embraced cloud providers (or at least run your own cloud
on Xen or kvm), it assumes that you are building a distributed system, and it
assumes your distributed system is built using standardized components. If 99%
of software is various combinations of nginx, Node, Python, Ruby, the JVM,
memcached, redis, PostGres, MySQL, and MongoDB, it can make sense to modify
just those 10 applications to run on bare metal, cut Linux out of it entirely,
realize gains of up to 10x in performance (based on the original MIT exokernel
papers), and still present the same programming interface to application-level
programmers.

Unikernels are not useful to people who just want to hack C on a single box,
nor are they useful to giant companies like Google where _all_ of their
software is written in C++ with POSIX APIs. But they could save a lot of money
for many mid-range businesses who currently host on AWS or GCE and write
largely in high-level languages.

~~~
sillysaurus3
Here I was, ready to rail against this horrific explosion in software
complexity, and then you go and say

 _realize gains of up to 10x in performance (based on the original MIT
exokernel papers)_

Is that true? Do the observed improvements in the field get anywhere close to
that theoretical boost?

If that's true, then that's incredible. Which layer of the OS is responsible
for an Nx slowdown? Where N is whatever the real multiplier actually is most
of the time.

(Aside: it's probably not a good idea to imply "high level language == doesn't
use fork." Fork is a fundamental primitive. There are things you can do by
forking which you can't do by other means. I mean "can't" in the same say as
"yes, every language is turing complete and so therefore can simulate any
other language, but you can't write recursion in BASIC, because you wouldn't
want to try.")

~~~
andrewstuart
We carried out some simple tests of Nginx on the rump kernel a few months ago.
Performance is approximately similar to running on an "ordinary" operating
system +/\- 10% depending on a variety of factors.

~~~
avsm
It's worth bearing in mind that the Rump Xen backend isn't well optimised at
the moment, so there's quite a bit of low hanging fruit that could probably
boost performance quite a lot. I think things like multi-page netfront, more
careful rx/tx batching, and probably scheduler measurements will have an
impact. The focus right now is on getting the individual applications to run
reliably and robustly as unikernels when recompiled under Rump, and
upstreaming patches where applicable.

------
voltagex_
Could I port Python to this and then run a Flask app directly under Xen?

~~~
FraaJad
You should be looking at Rump kernels for this, specifically
[https://github.com/rumpkernel/rumprun-
packages/tree/master/p...](https://github.com/rumpkernel/rumprun-
packages/tree/master/python) directory.

------
peeyek
> A minimal, service oriented, includeable library operating system for cloud
> services

I think the operating system for cloud services is need to be able to run
heterogeneous system. Which Mesos & CoreOS are headed towards this idea via
containerization.

Why build an OS that only run a C++ for cloud services? Is there any use
case/problems that the author trying to solve?

~~~
tinco
It is generally assumed virtual machines have better isolation (security-wise)
than operating system level isolation like the containerization Mesos and
CoreOS are making use of.

~~~
Leon
CoreOS and SELinux together is pretty secure if set up properly; even if you
broke out of a container you wouldn't be able to do anything, even start a
process. But still, yeah, in a unikernel there is no 'breaking out' at all.

------
nv-vn
What is the advantage of this over running a unikernels in Xen?

~~~
bjackman
This is the only unikernel I know of that supports "raw" applications - OSv
applications run in a JVM and MirageOS applications are in OCaml.

Edit: looks like there's a direct alternative called Rumprun:
[https://github.com/rumpkernel/rumprun](https://github.com/rumpkernel/rumprun).
I hadn't heard of it.

Anyway, right now this whole unikernel thing is still pretty new so it's good
to have multiple projects.

~~~
anttiok
Notably, there's ongoing work (with an already functional result) to run
Mirage on top of the low-level drivers provided by Rumprun. The idea in doing
that is enabling the "top-level" Mirage stack to run in environments for which
they don't have native OCaml drivers yet. (disclaimer: I don't speak for
Mirage, though I do speak for Rumprun)

------
AnbeSivam
Hope it gets built for cross platform. I wish this could be run on top of
smartos zones, instead of kvm or xen.
[http://dtrace.org/blogs/brendan/2013/01/11/virtualization-
pe...](http://dtrace.org/blogs/brendan/2013/01/11/virtualization-performance-
zones-kvm-xen/)

smartos zones also has a linux compatibilty layer ( lx branded zones ).

------
nvartolomei
Is this even a thing when we have container boom? Any advantage of running a
virtualization layer just to run a C++ application, why not run it in a
container?

~~~
vezzy-fnord
Where the container virtualizes an instance of an operating system, the libOS
(related: the unikernel and exokernel) goes further into virtualizing the
operating system services and interfaces themselves. It's a logical step
further.

~~~
phire
One advantage is that the application gets full control over the page tables.
At least one program I work on would gain huge advantages in that kind of
environment.

~~~
vezzy-fnord
The same is true for microkernels, FWIW. As far back as Mach, a large
advantage was separating the mechanism of VMM from the policy of handling page
faults, which was done by external pagers in userspace.

------
bluejekyll
I'd love to see some consolidated efforts here. I'm currently looking at
rumprun but need to support Java and at least I haven't found anyone who's
done that successfully. So now I'm also looking at OSv which looks like it's
trying to be compatible with Docker.

~~~
anttiok
Do you know anyone who's even tried to make Java work on Rumprun? I only tried
Python, that worked out of the box. From what I've been following others, e.g.
Erlang, Rust and Javascript (node.js) more or less provided the same
experience.

------
josteink
Are we just inventing new stuff which will make our code harder to debug for
the sake of making something "new" at this point, or am I just being overly
cynical?

~~~
vezzy-fnord
This idea isn't new at all. In fact, it's largely a conceptual throwback to
the days before even batch schedulers where programs had to initialize the
machine they were running on. The modern libOS dates from MIT research in
exokernels during the 90s, itself an incremental step further of the numerous
microkernel projects like SPIN, V-System, Mach, Amoeba and so forth.

It's a 1950s idea updated to 21st century standards using 1970s dynamic
linking, sometimes but not necessarily meant to be run under a hypervisor.

------
billconan
good to see a unikernel for c++

~~~
henriknj
we haven't really seen anything yet.. no code, paper or whatsoever, not sure
what to expect

------
GPGPU
We've been doing a similar thing with Erlang -- running it without an OS using
Ling to run Erlang on Xen

See [http://erlangonxen.org](http://erlangonxen.org)

(And see the demo here:
[http://zerg.erlangonxen.org](http://zerg.erlangonxen.org) )

~~~
abrookewood
I've been following erlangonxen for a while, but things seem to be quiet. What
are you guys working on? Any news?

------
mempko
I just pooped my pants. How excite!

