
IncludeOS: a minimal unikernel operating system for C++ services - lelf
https://www.includeos.org/
======
resoluteteeth
The description on github is much clearer:

> IncludeOS is a minimal unikernel operating system for C++ services running
> in the cloud and on real hardware.

~~~
s_Hogg
This is quite helpful, thanks. What would prompt someone to go this way, I
wonder? There are some pretty minimal OS options out there these days.

~~~
minxomat
See the recent ccc talk on the co2 footprint of services.

~~~
3fe9a03ccd14ca5
I can’t believe this is a thing. Are software engineers in the “ban straws”
phase of global warming now?

~~~
munmaek
A better question is Why Not? Is it really so inconceivable to try to apply
energy conservation to computing as well?

~~~
yjftsjthsd-h
Just as importantly, energy use loosely correlates with resource use => cost.
If we save the planet, great, but it's also good for our bottom line.

------
jrockway
The industry is certainly moving towards less and less stuff included by
default in a production environment. What used to be a full install of Debian
with your app checked out in /var/www is now a Docker container containing
only the packages you need. The even more careful people don't even use
Debian, they use a stripped down distribution like Alpine. The even more
careful people don't have a distribution, they just have their application and
the one or two support files it needs (ca-certs usually). It is intriguing to
apply this approach to OS features itself, because less dependence on the OS
means that more of the kernel can be safely shared. And, of course, there is
less surprise at something that is enabled by default that you don't need. (I
feel like 99% of security is disabling stupid things that are on by default.
With Linux, you never know if you got them all.)

Being able to share the kernel safely means potentially better utilization.
(The progression is dedicated machines -> hypervisor-based VMs -> multiple
customers sharing the Linux kernel. gVisor exists for that last step, but I
don't see anyone willing to sell me the ability to run my random container
next to their mission-critical security-sensitive app, whereas people are
perfectly happy to share CPUs with me if there is a hypervisor in the middle.
But we're certainly moving in that direction.)

It is also intriguing to imagine a world that moves the other direction; just
have a giant computer with thousands of power-efficient CPUs in it and no
virtualization. When a request comes in to your app, your CPU is just powered
on, your app boots in a millisecond, handles the request, and shuts down. I am
not sure even the big cloud providers have solved the time-of-day utilization
problems; being able to turn most of the computers off at night has some
potential to save money. (I would turn my workstation off at night if it
booted in 1 millisecond in the morning, for example. I can also imagine some
power savings for mobile devices if they could go hard off for 59 seconds out
of every minute.)

~~~
mike_hock
"Safely" sharing a Linux kernel?

There's no reason to believe Spectre and Meltdown were a one time thing,
especially since the response has been to apply ad-hoc fixes just good enough
to break the proofs-of-concept. On the contrary, they demonstrate that such
vulnerabilities are possible and we should expect to see them again.

Even barring CPU bugs, the kernel is one giant attack surface, all written in
C, all running with full ring 0 privileges. Find ONE hole to pwn.

~~~
jrockway
It's not mainstream yet but it's the direction people want to go. Like I said,
making the OS smaller means less attack surface, so it's certainly something
worth investigating. But a lot of work has been done to make Linux safe
enough, so that is a likely direction for the mainstream to go.

------
StavrosK
This looks great, but I'm wary of projects that sound _this_ great but don't
include a "downsides" section. I'd love to know what the tradeoffs are, so I
don't have to spend time discovering them myself. Please consider adding a
section like that.

~~~
tomashubelbauer
They link to one:

> A longer list of features and limitations can be found on our documentation
> site.

But unfortunately it looks like the linked page doesn't actually talk about
limitations :) Unless you consider a warning about hlt-ing yourself in the
foot a list of limitations I guess.

~~~
fwsgonzo
Some downsides then:

\- You can't just port anything to IncludeOS. Since you have direct hardware
control and you can schedule work directly to numbered CPUs you can take
advantage of things, but if you don't do that then you are just living in a
limited C++ environment.

\- The network stack hasn't been built with multiprocessing in mind, and some
parts need to be rebuilt because of that

\- There are codepaths that could lead to C++ exceptions, such as being out of
memory, which trounces performance.

\- There is a paging and memory allocation system in the OS which doesn't have
to be there (for example paging can be "burned in" to the image for the most
part, while only leaving room for adding pages for stacks. Memory can be
handled by donating everything to the C standard library) - the goal being to
keep the attack surface lower.

\- Debugging is hard - you will have to connect to GDB while pausing somewhere
in the OS.

This is really just me thinking out loud. Add salt.

------
Paperweight
It would be interesting to run a NodeJS clone or QuickJS
([https://bellard.org/quickjs/](https://bellard.org/quickjs/)) on/with this.

~~~
heyflyguy
This is why I love HN, I never knew about QuickJS. So yeah, that and
includeOS...dude...!

~~~
anewvillager
Duktape is also cool: [https://duktape.org/](https://duktape.org/)

------
missjellyfish
So basically you now run your app in ring0, with none of the security
mitigations the community developed in the past ten years, and force everybody
to code in c/c++. Is it just me or does that sound like a disaster waiting to
happen?

~~~
ncmncm
There is no such language as C/C++.

The alternative for people using this approach is running in a kernel module,
or a kernel-bypass setup with root permissions. Grown-up stuff. Eliminating
most of the kernel, and using C++ not C is a very substantial improvement over
status quo.

~~~
klyrs
> There is no such language as C/C++.

This is a _really_ weird hill to die on; I see you've made the same comment
several times in this thread. C and C++ fit hand in glove, and lots of folks
use them in conjunction. I don't think anybody here has the misconception that
C/C++ is a single language, unless perhaps they don't know either.

~~~
ncmncm
The point is that the practices that almost unavoidably lead to bugs in C code
are not even tempting in modern C++. So, if you are interested in correctness
and memory safety, stepping to modern C++ provides more value for time
invested than the sum of most other choices.

Equating the two is extremely common, even by people who are certainly
equipped to know better. When you feel a need to tar C++ as if it were C, you
signal the weakness of your argument before you have even expressed it.

~~~
klyrs
> you signal the weakness of your argument before you have even expressed it.

Please turn down your flamethrower, that style of conversation is not welcome
here.

~~~
ncmncm
Writing "C/C++", in contexts where it is implied they have the same failure
modes, is provocative. Don't like response, don't provoke.

It is fine when talking about ABI or object-code generation.

------
speransky
Old concept in HPC world, most of ancient supercomputers had ability to
compile compute tasks with unikernel and spawn over compute nodes. Few years
ago concept of Xen bases unikernels circulated in xen mail lists, but look
like there was little interest in community. Includeos can be a “fat lambda”
for those who want to control everything, but how many of such tasks we have

------
dang
Threads from 2015:
[https://news.ycombinator.com/item?id=10409292](https://news.ycombinator.com/item?id=10409292)

[https://news.ycombinator.com/item?id=10661746](https://news.ycombinator.com/item?id=10661746)

[https://news.ycombinator.com/item?id=10741350](https://news.ycombinator.com/item?id=10741350)

2016:
[https://news.ycombinator.com/item?id=11154168](https://news.ycombinator.com/item?id=11154168)

2017:
[https://news.ycombinator.com/item?id=13975994](https://news.ycombinator.com/item?id=13975994)

2018:
[https://news.ycombinator.com/item?id=17848324](https://news.ycombinator.com/item?id=17848324)

[https://news.ycombinator.com/item?id=18390204](https://news.ycombinator.com/item?id=18390204)

------
archivist1
Would it be possible to build a C/C++ app with #include <os> at the top and
then compile it to a fully bootable image, and then boot a real server using
that?

~~~
int_19h
They claim that it works on real x86 hardware.

------
dtrailin
I'm really surprised this isn't in Rust considering security is a priority.
However, I think something like this will become increasingly common for high
performance cloud applications. Many programs do not need a full operating
system with processes isolation and hardware drivers if they're going to run
alone on a VM anyways.

~~~
ncmncm
The most urgent goal for this effort is usefulness. Geek trend box-checking
has to come second.

~~~
dtrailin
Rust is not the right choice because it's trendy, but because it provides
memory and thread safety properties that are hard to enforce in C++. For
systems software the advantages are huge.

~~~
ncmncm
Rust is also years away from maturity (which it probably will achieve, on
schedule). Systems built today need to work with tooling that is mature today.

Memory and thread safety properties can be achieved in C++ by operating at a
higher level than C with mature, well-debugged, well-optimized libraries. No
pointers means no pointer errors.

Integer overflows are equally possible in Rust and C++. Both offer debug build
modes that can watch for those.

~~~
the_why_of_y
> No pointers means no pointer errors.

Use after free can be perfectly well expressed with C++ references, or
invalidated iterators to standard library containers.

> Integer overflows are equally possible in Rust and C++.

In C++, unsigned integer overflows can often be leveraged into out-of-bounds
access; in safe Rust, they cannot.

~~~
ncmncm
Use-after-free can be expressed in C++, but there is no temptation to do it.

------
Quequau
Is this project still active? I don't think it is any more.

~~~
sm4rk0
> Sorry, but no. There are things happening here and there, for example
> pthreads just got merged, but otherwise development is very slow. There are
> plans to run on RPI4 soon (TM), if that's of any interest, otherwise the
> general development is low-frequent.

[https://github.com/includeos/IncludeOS/issues/2183](https://github.com/includeos/IncludeOS/issues/2183)

~~~
zamadatix
The Pi 4 would be cool. I've always wanted to mess with a unikernel but they
either seemed to be dead, barely started, or written in an unusual choice of
language.

Anyone else have a unikernel recommendation besides IncludeOS + rpi 4?

~~~
Quequau
If you are interested in Rust, there's HermitCore and you can run them on the
Raspberry Pi via Firecracker.

[https://hermitcore.org/](https://hermitcore.org/)

------
Izmaki
What caught my eye was the openness for vulnerability disclosures. Right on
the front page. Good stuff

------
x0054
I feel like most commenters in this thread didn’t read this page that explains
the goals of the project with far more detail:

[http://www.includeos.org/technology.html](http://www.includeos.org/technology.html)

------
psim1
It's like sticking the program floppy into the Apple II disk drive and
powering up. Or is that too simple an analogy?

~~~
fwsgonzo
I like that analogy. I guess these images are more single-purpose minded, and
often will do things read-only. Perfect for immutable infrastructure, but it's
not a requirement.

------
smotti
A viable alternative that supports more languages is OSv:
[https://github.com/cloudius-systems/osv](https://github.com/cloudius-
systems/osv)

------
H8crilA
I think the biggest bummer is the lack of threads. Everything seems to be
running in a single loop:

> Node.js-style callback-based programming - everything happens in one
> efficient thread with no I/O blocking or unnecessary guest-side context
> switching.

Unless I'm missing something.

------
cstross
Article blows it around the third para by asserting that publishers
could/should make ebooks much cheaper than paper ones (because obviously paper
costs money, right?).

Then asserts that the retail price for the paper edition is the point ebooks
should undercut by, oh, 25%.

This ignores the fact that publishers are not retailers, they're
manufacturers; the retail sector takes 50-70% of the price the customer pays
for a book, _and this includes Amazon 's kindle store_. In actual fact, the
physical cost of goods for a paper book is less than 10% of the price the
customer pays. Another 10% is editorial/typesetting costs which are exactly
the same for the ebook, maybe 10% goes to the author, and 50-70% to Jeff Bezos
and co.

~~~
hugodutka
I think you may have commented under the wrong post.

~~~
cstross
D'oh, I think you might just possibly be right!

------
justicezyx
Had a few hangouts meeting with the original includeos team.

Great technology, top tier team. Although I believe their distance to Silicone
Valley and relatively cheap capital causes their difficulty.

------
erichocean
This is pointless. Run Linux and isolate it to the first core. Lock your own
app to the other cores, with no context switching. Gdb works. SSH works.
Awesome.

------
dastx
It'll be interesting to write a container system using this and build an
orchestrated with this in mind. Would give you a truly minimal container OS.

~~~
weberc2
“Container” and “OS” don’t mean anything in the context of unikernels.
Unikernels are applications with everything they need to run (on bare metal or
a hypervisor) baked into the application. So there is no OS to speak of, and
the “orchestrator” is probably something like AWS EC2.

~~~
tzm
OP likely meant service orchestration

~~~
weberc2
I think that’s what I meant as well, but I’m not sure what you mean by
“service orchestration”. If Kubernetes is an example of a service
orchestrator, then I think something like EC2 would be the unikernel analog.

------
dehrmann
I keep hoping for a model like this to displace Docker.

------
ngcc_hk
Is there similar thing like this but for Node JS server? I think the direction
is totally right. Just hope we can do this for the JS world.

------
moralsupply
I couldn't find a mention on what drivers for hardware devices it supports

~~~
H8crilA
It's basically only meant to be used on hypervisors:

[https://includeos.readthedocs.io/en/latest/Features.html](https://includeos.readthedocs.io/en/latest/Features.html)

------
moonbug
good luck turning your debugging tools on anything that produces.

~~~
ncmncm
You debug it the same way you debug embedded systems. GDB stubs, and a mapped,
shared page.

Advantage over embedded being you don't need jtag hardware.

~~~
moonbug
that's rather my point.

~~~
ncmncm
I debug regular apps, via gdbserver, routinely, in exactly the way one would
debug embedded apps, and give up no convenience at all, but gain quite a lot.
Gdb runs on my full-featured development environment, and the target process
runs under docker, or on some other machine with the funny network hardware. I
hardly ever start a new Gdb session; I just point it at binaries that I run
wherever is convenient. Debugging a unikernel is trivially different.

------
baybal2
Hardware support is fairly limited

------
known
Sounds like torrents for apps

------
1996
For zero overhead, I have this weird idea of shipping source code, recompiling
it with all the optimizations needed, and executing it without virtualization
but under a different user id.

So I stick to ./configure ; make ; make install

~~~
jbverschoor
Userid means user management and context switching etc, means overhead

~~~
temac
Imaginary overhead doesn't count. Show me the benchmarks.

~~~
chrismorgan
Context switching is expensive when you’re writing high-performance code.
Conventional OSes require that you context switch to the kernel to perform I/O
(disk or network), to say nothing of other processes vying for CPU time as
well.

All this is well-documented and -understood. The overhead is in no way
imaginary.

Unikernels don’t need to do context switches, by virtue of not having
processes.

~~~
namibj
Since io_uring, this isn't the case for most IO on Linux anymore.

~~~
chrismorgan
I’m not terribly familiar with io_uring, and haven’t used it myself yet, but
does it not still require a syscall to drive things, even though the buffers
are shared? Brief investigation suggests the need for that might go away in
the future? But all this io_uring stuff seems fairly immature yet and not
widely used, though still definitely of interest.

~~~
ncmncm
Syscalls to set up, otherwise just watch for atomically updated pointers to
change..

