
Linux compatibility in IncludeOS - ingve
https://www.includeos.org/blog/2018/musl.html
======
murderfs
Claiming that they've "implemented" 42 syscalls seems like an overstatement:
almost everything is a stub that does nothing, and the bits that are
implemented are not even close to being half-baked. Even relatively easy stuff
like dup is missing.

~~~
hugelgupf
The number of system calls says close to nothing of the actual API surface
present here. It's a poor measure. What actually matters is: what can I run?

That said, 42 is not enough to run a lot of applications.

It'll be interesting to get to the point where they can run a language's
runtime or unit tests.

Take it from someone who worked on gVisor: this is gonna take a long time, and
eventually you'll end up with a system that's just Linux. You'll end up either
being bug-for-bug compatible, which means bringing in loads of compatibility
code and architecting your kernel just like Linux, or you'll end up having to
change applications (and then you're not all that compatible anymore).

Neither are fun or sustainable compatibility stories. You'd probably actually
get Linux apps to work on a random OS faster & nicer by porting gVisor.

> Moreover, since implementing a system call is typically pretty simple [...]

You're in for a fun ride, folks.

~~~
perbu
(I work for Includeos)

I don't think we need to run everything. I think we'll be happy if we can take
some common libraries and compile them and they work. That is currently our
goal. Down the road, depending on interest, we can perhaps expand a bit.

>> Moreover, since implementing a system call is typically pretty simple [...]

> You're in for a fun ride, folks.

Thanks for the heads up. It was meant as opposed to implementing the syscall
and layers on top of it.

~~~
hugelgupf
That sounds much more attainable than what the blog post suggested - sorry if
I took that the wrong way!

~~~
perbu
No worries. I never meant for the blogpost to reach an audience outside of our
pretty narrow knitting circle so I didn't pay much attention to detail. I very
much appreciate your insights here.

------
hardwaresofton
If you're into unikernels, rumpkernels[0] are the least-modification-required
approach I've seen.

Basically, if you compile your program with a minimally (as minimal as is
possible) modified compiler, you get "rumprunnable" output. there's a list of
popular packages built to run on rump kernels[1] (Rust is supported[2]). I
haven't tried to use the toolchain yet, but it's pretty appealing to me.

The biggest issue is that rumpkernels seem to not be in developed anymore. I
can't tell if the entire is just off somewhere being productive with their
rumpkernels and are happy or if no one is around at all. The Github
organization[3]'s repos all seem to not have been changed in a while either.

Outside of rumpkernels, Nabla containers[4] also look interesting to me.
They're built on Solo5[5], but the only microkernels on there are currently
MirageOS (for OCaml) and IncludeOS (C++?). Nabla looks amazing (containers
look like a great fit and they seemed to have improved ergonomics massively),
but the limitations[6] are intense.

[0]:
[https://github.com/rumpkernel/wiki/wiki/Info:-FAQ](https://github.com/rumpkernel/wiki/wiki/Info:-FAQ)

[1]: [https://github.com/rumpkernel/rumprun-
packages](https://github.com/rumpkernel/rumprun-packages)

[2]: [https://github.com/rumpkernel/rumprun-
packages/tree/master/r...](https://github.com/rumpkernel/rumprun-
packages/tree/master/rust)

[3]: [https://github.com/rumpkernel](https://github.com/rumpkernel)

[4]: [https://nabla-containers.github.io/](https://nabla-
containers.github.io/)

[5]: [https://github.com/solo5/solo5](https://github.com/solo5/solo5)

[6]: [https://github.com/nabla-
containers/runnc#limitations](https://github.com/nabla-
containers/runnc#limitations)

~~~
perbu
I'm hoping we're be able to port libuv to IncludeOS (which should be doable,
much like libuv IncludeOS is eventdriven). Then we should be able to get Node
up and running. Running Node.js through IncludeOS on Nabla could be
interesting in a "FaaS" context.

Last I heard the author of Rumprun has picked up beer brewing full time and
isn't interested in maintaining the project.

It would be great if someone where to pick it up.

OSv is also very relevant. They provide a much richer and more compatible
runtime environment compared to IncludeOS (I think they even have zfs). And
OSv is pretty active these days, so if you looking for a way to run some high
performance stuff OSv is worth looking into.

~~~
hardwaresofton
Thanks for shedding some light on the possible status of the Rumprun
maintainer! Hope he's happy brewing beers :)

Thanks for mentioning OSv[0], I'd never heard about it before.

I'm excited to see what happens with Nabla, I read the accompanying paper and
was pretty excited about the possibilities. Nabla's greatest strength is it's
ergonomics, IMO, though rumprun comes close (someone took some time to make
dockerized containers for it).

[EDIT] - looking at OSv's (primary?) tool capstan[1], I'm not sure it's quite
the same, it seems more like linuxkit[2] than rumprun. Maybe my intuition is
wrong, but it seems much more like you're bringing along a stripped down OS
than you're writing a program meant to interact with a shared unikernel. The
distinction is so fuzzy in my head I'm not sure there is one, but it feels
different.

[0]: [http://osv.io/](http://osv.io/)

[1]: [https://github.com/cloudius-
systems/capstan](https://github.com/cloudius-systems/capstan)

[2]:
[https://github.com/linuxkit/linuxkit](https://github.com/linuxkit/linuxkit)

~~~
ricarkol1
Nabla containers actually use rumprun under the covers. Here is the rumprun
fork that supports a solo5 backend: [https://github.com/nabla-
containers/rumprun](https://github.com/nabla-containers/rumprun)

------
emilfihlman
The article is "broken", it talks about Musl and then says the found Musl.

------
zapita
There’s a lot to like here.

1\. Focusing on compatibility is smart because it’s the weak point of
IncludeOS and other unikernel projects.

2\. Musl is an incredibly underrated project and I hope one day it becomes the
new standard libc.

This is just a first stab, but it could turn into something great.

I wonder if gvisor and includeos could be combined in interesting ways, or at
least steal from each other’s code and ideas?

------
ncmncm
This is an exciting development that I am watching closely.

It should be noted that whatever the limits of the 42 system calls, network
operations are very well supported. Many useful programs hardly need anything
else.

------
leni536
Would it be possible to run Linux as a unikernel and port your application as
a kernel module? How a leaned down Linux kernel would compare to includeOS?

~~~
perbu
There is a project that tries to turn Linux into a library which could lead to
a unikernel approach with the Linux. If you look at
[https://github.com/lkl](https://github.com/lkl) you can see where they are.

A leaned down Linux-kernel with the application linked into kernel space would
be pretty comparable. But it would be pretty enormous beast.

------
freedomben
I'm pretty ignorant on Unikernels so maybe there's another solution, but not
being able to `fork()` is a significant problem for many applications.

~~~
__bjoernd
My understanding is that unikernels are single-application by design. I agree
that a mechanism to spawn other apps is useful, but in this case it won't be a
simple fork but the creation of a complete new os instance.

~~~
chris_wot
There are plenty of applications that used multiple processes.

~~~
perbu
Absolutely. And most of those are deemed out of scope for Unikernels.

------
jetzzz
Looks like Linux syscalls are becoming some kind of de-facto standard. They
are already implemented by Linux, FreeBSD, Windows and now IncludeOS.

~~~
AnIdiotOnTheNet
The mind reels.

Not because Linux is a bad kernel, mind you, but because UNIX itself reeks of
ideas that were ok when it was originally developed and clunky at best today.
Even Microsoft seems to have given up. The future of computing looks pretty
grim to me.

~~~
solarkraft
Could you elaborate?

~~~
AnIdiotOnTheNet
On what? Reasons UNIX isn't so great? There was an entire handbook written
about it decades ago, much of which is still relevant. Text isn't a universal
interface, files are not a good abstraction for everything (not the way UNIX
does it anyway), the permissions system is extremely limited and quite
backward for modern problems, the way the file hierarchy is organized hasn't
made sense for about 30 years, POSIX has a lot of well known issues, etc.

------
chris_wot
Can someone point me to a more detailed overview of IncludeOS than the brief
summary they have on their home page?

~~~
perbu
For a technical, in-depth overview of IncludeOS and how it was made I find the
main authors presentation at Cppcon 2017 is a pretty good introduction:

[https://www.youtube.com/watch?v=h7D88U-5pKc](https://www.youtube.com/watch?v=h7D88U-5pKc)
(one hour, quite entertaining).

There is also the original paper which was presented at the 2015 IEEE 7th
International Conference on Cloud Computing Technology and Science (CloudCom).

[https://ieeexplore.ieee.org/abstract/document/7396164](https://ieeexplore.ieee.org/abstract/document/7396164)

------
4ad
So, what's wrong with newlib?

~~~
ZiiS
Nothing; it is just much smaller by not providing POSIX and Linux like
interfaces.

Alpine proves full musl can already run a full distro's worth of programs and
is very easy to port anything using glibc.

Newlib tends to be used in projects like Cygwin as a small C standard library,
where the rest of the POSIX API is implemented separately (by proxying to
Win32 system calls for Cygwin).

