That said, 42 is not enough to run a lot of applications.
It'll be interesting to get to the point where they can run a language's runtime or unit tests.
Take it from someone who worked on gVisor: this is gonna take a long time, and eventually you'll end up with a system that's just Linux. You'll end up either being bug-for-bug compatible, which means bringing in loads of compatibility code and architecting your kernel just like Linux, or you'll end up having to change applications (and then you're not all that compatible anymore).
Neither are fun or sustainable compatibility stories. You'd probably actually get Linux apps to work on a random OS faster & nicer by porting gVisor.
> Moreover, since implementing a system call is typically pretty simple [...]
You're in for a fun ride, folks.
I don't think we need to run everything. I think we'll be happy if we can take some common libraries and compile them and they work. That is currently our goal. Down the road, depending on interest, we can perhaps expand a bit.
>> Moreover, since implementing a system call is typically pretty simple [...]
> You're in for a fun ride, folks.
Thanks for the heads up. It was meant as opposed to implementing the syscall and layers on top of it.
That said - why do you have to get musl compatibility by implementing the Linux syscalls? Is contributing to musl an option?
Wrt. to why we have to implement Linux-like syscalls; Musl is written specifically for Linux so in order to "unlock" the relevant POSIX-bits that Musl can provide we have to provide the underpinnings that Musl requires.
I'm hoping we can upstream some of our non-intrusive changes to Musl. However since Musl has a very specific purpose (a Linux-specific C and POSIX library) I'm not certain that our changes are within the scope of the project. To date we've only concerned ourselves with a small subset of Musl and only on amd64.
Basically, if you compile your program with a minimally (as minimal as is possible) modified compiler, you get "rumprunnable" output. there's a list of popular packages built to run on rump kernels (Rust is supported). I haven't tried to use the toolchain yet, but it's pretty appealing to me.
The biggest issue is that rumpkernels seem to not be in developed anymore. I can't tell if the entire is just off somewhere being productive with their rumpkernels and are happy or if no one is around at all. The Github organization's repos all seem to not have been changed in a while either.
Outside of rumpkernels, Nabla containers also look interesting to me. They're built on Solo5, but the only microkernels on there are currently MirageOS (for OCaml) and IncludeOS (C++?). Nabla looks amazing (containers look like a great fit and they seemed to have improved ergonomics massively), but the limitations are intense.
Last I heard the author of Rumprun has picked up beer brewing full time and isn't interested in maintaining the project.
It would be great if someone where to pick it up.
OSv is also very relevant. They provide a much richer and more compatible runtime environment compared to IncludeOS (I think they even have zfs). And OSv is pretty active these days, so if you looking for a way to run some high performance stuff OSv is worth looking into.
Thanks for mentioning OSv, I'd never heard about it before.
I'm excited to see what happens with Nabla, I read the accompanying paper and was pretty excited about the possibilities. Nabla's greatest strength is it's ergonomics, IMO, though rumprun comes close (someone took some time to make dockerized containers for it).
[EDIT] - looking at OSv's (primary?) tool capstan, I'm not sure it's quite the same, it seems more like linuxkit than rumprun. Maybe my intuition is wrong, but it seems much more like you're bringing along a stripped down OS than you're writing a program meant to interact with a shared unikernel. The distinction is so fuzzy in my head I'm not sure there is one, but it feels different.
1. Focusing on compatibility is smart because it’s the weak point of IncludeOS and other unikernel projects.
2. Musl is an incredibly underrated project and I hope one day it becomes the new standard libc.
This is just a first stab, but it could turn into something great.
I wonder if gvisor and includeos could be combined in interesting ways, or at least steal from each other’s code and ideas?
It should be noted that whatever the limits of the 42 system calls, network operations are very well supported. Many useful programs hardly need anything else.
A leaned down Linux-kernel with the application linked into kernel space would be pretty comparable. But it would be pretty enormous beast.
The first one isn't needed in a unikernel. You are already demonized when you boot up.
The second use is not available to unikernel applications. The limitation of a single application per vm applies.
Applications also sometimes have separate processes for sentinal functions, logging, etc. Or default configurations that are multi process, like many load balancers, web servers, API gateways, etc.
Not because Linux is a bad kernel, mind you, but because UNIX itself reeks of ideas that were ok when it was originally developed and clunky at best today. Even Microsoft seems to have given up. The future of computing looks pretty grim to me.
* Filesystem semantics. What POSIX actually says and what most people think it says in this regard is very different. And both groups acknowledge that the semantics are not very useful for things like high-performance distributed filesystems in HPC.
* Signals. Everything about signals is pretty annoying. You can't install per-thread signal handlers, which means you can't do things like "catch" segmentation faults. And signals aren't files, but you can only poll for file things to occur, so you have to reexpose signals as files via signalfd...
A lot of new kernel features have just given up to the point where current advice is to never call it unless you plan to exec immediately after. But the technical debt within the kernel for this misfeature is massive.
https://www.youtube.com/watch?v=h7D88U-5pKc (one hour, quite entertaining).
There is also the original paper which was presented at the 2015 IEEE 7th International Conference on Cloud Computing Technology and Science (CloudCom).
Alpine proves full musl can already run a full distro's worth of programs and is very easy to port anything using glibc.
Newlib tends to be used in projects like Cygwin as a small C standard library, where the rest of the POSIX API is implemented separately (by proxying to Win32 system calls for Cygwin).