Hacker News new | past | comments | ask | show | jobs | submit login
Kerla: Monolithic kernel in Rust, aiming for Linux ABI compatibility (github.com/nuta)
471 points by Klasiaster 3 months ago | hide | past | favorite | 194 comments



Author here. I'm surprised to see my hobby project on Hacker News.

I know this kind of stuff spark the ``it's meaningless to rewrite everything (especially Linux) in Rust'' debate. I agree 100% that rewriting Linux in Rust (or your favorite language) is just a waste of time and such a project won't live long.

That said, what I'd say here is that it's fun. Really fun. Implementing ABI compatibility requires you to understand under the hood of the kernel. You can learn how printf(3) works. You can learn what happens before main().

Especially, in Rust, it's inspiring to explore ways to write a kernel safely and expressively. One of my favorite part is the implementation of read system call [1].

Lastly, You can try this OS on SSH. Hope you enjoy :)

    ssh root@kerla-demo.seiya.me
[1]: https://github.com/nuta/kerla/blob/main/kernel/syscalls/read...


> I know this kind of stuff spark the ``it's meaningless to rewrite everything (especially Linux) in Rust'' debate. I agree 100% that rewriting Linux in Rust (or your favorite language) is just a waste of time and such a project won't live long.

Considering that this is exactly how Linux was born (just a hobby project for fun), I wouldn't assume so fast that it's useless. Moreover, you don't need to justify yourself if you just want to have fun !


Any rust project like this inevitably gets sardonic replies about rewrite-in-rust fanaticism.

I agree they shouldn't have to justify themselves, but it is handy to preempt that.


Also, the rust community is too nice to openly admit it, but it's goal IS to displace C as the main low level language especially for the kernel. This is doomed to attract a lot of attention


I don't see why we need to pretend that every hobby has the potential for greatness. Being a hobby is a good enough end to itself.

In fact in this instance I think it's a little disingenuous to quote Linux and say it could happen again. The industry is totally different now. There's much more competition than there was when Linux was released and that competition is much more mature too. Plus the bar for a production-quality kernel is a lot higher than it was when Linux was released.


> In fact in this instance I think it's a little disingenuous to quote Linux and say it could happen again.

Seems apropos to me, given the fact that a Linux ABI compatible hobby project is under discussion - and that everyone here is familiar with the famous Usenet announcement.

> The industry is totally different now. There's much more competition than there was when...

I wonder how you define "competition"? Because there were way more operating systems in use then, and the industry was far more fractured. Fractured in a way that was meaningful - not like today where you can spin up a VM and be productive in short order, thanks to the significant lack of distinguishing difference. That is the really interesting thing about these hobby projects - they introduce possibilities that are either completely ignored by organizations suffering from inertia constraints, or can't be mimicked because they're diametrically opposed to present designs.


> I wonder how you define "competition"? Because there were way more operating systems in use then, and the industry was far more fractured.

There is way more competition now:

  + Linux (countless distributions)
  + FreeBSD
  + OpenBSD
  + NetBSD
  + DragonflyBSD
  + HardenedBSD
  + Darwin
  + Minix 3 (which wasn't free when Linux was released)
  + Illumos
  + OpenIndiana
  + Nexenta OS
  + SmartOS
  + ...and many others based off OpenSolaris / Illumos
This isn't even an exhaustive list of UNIX-like platforms that are new since Linux and free.

Don't conflate standardisation of the industry with a lack of options. More options do exist today and are in use (eg some games consoles run FreeBSD, Netflix uses BSD, Nexenta is used in some enterprise storage solutions, Darwin may not be used in any free capacity but macOS is clearly used heavily by HN readers, and so on and so forth).

Moreover, I've used FreeBSD, Solaris, OpenSolaris, Nexenta and OpenBSD on production systems over the last 10 years (and the list gets more esoteric if we look past 10 years). So just because you might exist in a Linux-only ecosystem it doesn't mean that's the case for the entire IT industry.


Odd, you purportedly know the difference between an OS and a distro - but that doesn't stop you from generating the above nonsense list. Just look at the last 5 items... seriously - all these Illumos derivatives are distinct operating systems in competition with one another? And to a degree that is no different from Netware vs OS/2?! Be serous, that list only further proves my point about the total lack of distinguishing difference in the present offerings - relative to the pre-linux environment.

> So just because you might exist in a Linux-only...

Obnoxiously presumptuous.


> but that doesn't stop you from generating the above nonsense list.

lol it's not nonsense. Every item I've listed offers something unique.

> Just look at the last 5 items... seriously - all these Illumos derivatives are distinct operating systems in competition with one another?

Actually the differences in the Illumos derivatives are quite fundamental:

- Nexentra is aimed at being an enterprise storage solution with Debian user land. That's massively unlike Illumos

- Smart OS is designed to be a smarter virtualisation and containerisation host OS. It even bundles Linux's KVM virtualization. It's not intended to be run as a desktop platform

- OpenIndiana and Illumos are probably the most similar, however OpenIndiana aims to be more true to OpenSolaris while Illumos aims to be more of a hybrid upstream. So while they're both multi-paradigm (like how Debian can be a desktop or server OS) there are some major differences in their commit tree.

Honest question: how close are you to the Illumos projects? Or are you just an outsider looking in and making assumptions about equivalence based on your experience with Linux? I ask because Illumos forks are much more akin to BSD forks than they are Linux distros.

> Be serous, that list only further proves my point about the total lack of distinguishing difference in the present offerings - relative to the pre-linux environment.

Even if you want to pare down the list to upstreams, you still have half a dozen BSDs (I really hope you at least have enough experience with BSD to realise these aren't just respins like Linux distros), Illumos, Minix, and Darwin. Verses your point in the 90s which was basically non-free BSD, non-free SysV, non-free QNX, non-free Minix etc....you still have those options now PLUS the ones I've listed.

This is the problem with your argument. You're assuming the old choices have gone away, which they haven't...well, apart from SCO and Minix is now free. And in addition we have dozens of interesting new platforms, some of which I've exampled, too.

There's no way on Earth we have less choice now than in the 90s. We might have the industry largely standardising on a subset but that's an entirely different argument and it's only representative of the lowest common denominator doing common problems. However if you look slightly outside of the status quo and you'd see there is a lot of variety still happening in the industry. I know this first hand too -- as I've said in my previous post, I've worked in plenty of places that weren't just Linux shops :)


> Every item I've listed offers something unique.

You then go on to characterize components grafted in from another OS as "unique". That is a laughably low bar, making every potential permutation of existing operating system components distinct offerings in competition for mindshare.

> Honest question...

Honest answer: you are running code I've written. I've been using ZFS in production for as long as one possibly can. My home file server has been SmartOS for many years, before that it was FreeBSD.

> I really hope you at least have enough experience with BSD to realise...

I'm keenly aware of their differences and commonality, having tracked the introduction of a terminal mode bug all they way back to a time when "Melbourne getty" was a thing.

> There's no way on Earth we have less choice now than in the 90s.

Well, I suppose that if you are including abandonware and dead product offerings in the list of competing operating systems...

> We might have the industry largely standardising on...

It seems like you are trying really hard to avoid using the more appropriate word for what has happened: "consolidating".


> You then go on to characterize components grafted in from another OS as "unique". That is a laughably low bar, making every potential permutation of existing operating system components distinct offerings in competition for mindshare.

That's literally no different to how Unix evolved. You talk about diversity in the 90s when most of the platforms were being trolled by SCO for containing the same code base. Don't you see the hypocrisy of your argument there?

> Honest answer: you are running code I've written. I've been using ZFS in production for as long as one possibly can. My home file server has been SmartOS for many years, before that it was FreeBSD.

Then it seems very strange that you don't acknowledge the differences in current platforms while proclaiming that Unix was more diversified in the 90s (if it was that clear cut SCO wouldn't have been trolling everyone).

> Well, I suppose that if you are including abandonware and dead product offerings in the list of competing operating systems...

You're overstating things once again. :)

> It seems like you are trying really hard to avoid using the more appropriate word for what has happened: "consolidating".

No. Consolidating means the removal of options. Those options still exist they're just not as commonly used. Thus term I used of "standardisation" is more apt.

Look, I do understand the rose tinted glasses you're wearing. There are aspects of 90s era systems administration and development that I miss too. But I still feel you're way off the mark with your opinions here. In some places you are exaggerating a nuanced point to such an extent that as much as I'd love to cheer on for the "good old days of computing", your comments simply don't represent my experiences then nor now.


> That's literally no different to how Unix evolved.

Uh, that is exactly my point. You know that you were just arguing that they were unique operating systems, right? You don't see how the comparison totally undermines that position?

> ...while proclaiming that Unix was more diversified in the 90s...

I'll never understand why anyone bothers fabricating strawmen in thread based mediums - I never made that claim, and that is plainly obvious to anybody who has the ability to scroll up. I have a hard time believing that you could, in good faith, read "there were way more operating systems in use then" and honestly think "oh, he is obviously only talking about Unix!"

> No. Consolidating means the removal of options.

Yes... you remember what the option was selecting from? I'll save you the trouble of manipulating your scroll wheel: "The industry is totally different now. There's much more competition... the bar for a production-quality kernel is a lot higher..."

You still wanna keep saying that stuff from before, about options?


> Uh, that is exactly my point. You know that you were just arguing that they were unique operating systems, right? You don't see how the comparison totally undermines that position?

I never said they were unique operating systems. I said we have more choice now than we ever with regards to Unix-like systems and I said they were distinct platforms, code bases, even upstreams. All of which is true. I have no interest in entering a philosophical question of what changes constitute a "unique operating system" however we can at least have a technical discussion about choice and build of the Unix-ecosystem.

> > ...while proclaiming that Unix was more diversified in the 90s...

> I'll never understand why anyone bothers fabricating strawmen in thread based mediums - I never made that claim, and that is plainly obvious to anybody who has the ability to scroll up.

All these comments you make of strawman arguments and now I can see that the original reason for our disagreement was that you replied arguing a different point to me from the outset!

I was only ever talking about Unix-like platforms (given we're talking about Linux ABI) and you misunderstood that post to think we were talking about operating systems in the broader sense. In fact the language I used should have made the scope explicit so I'm giving you the benefit of the doubt here that you weren't intentionally just pulling an asshole move of subtly changing the scope to win a different argument (which is the very definition of a straw man -- since we're already arguing about who's trying to pull a straw man)

> I have a hard time believing that you could, in good faith, read "there were way more operating systems in use then" and honestly think "oh, he is obviously only talking about Unix!"

I was though. You were the one who replied to my point where I was only talking about Unix so good faith would dictate you continue with the same scope and context as the conversation started out with. So yes, I did assume you were still only talking about Unix. Because that's what the conversation was about prior to you joining it.

-----

If we are to discuss your point then I agree there are fewer operating systems in general. And I agree that is a great loss. I'm also happy to talk at lengths about that too..... but I don't really see what relevance that has within a discussion about whether yet another POSIX kernel will "become the next Linux" because even if Kerla was to "become the next Linux" it still wouldn't satisfy your argument about diversity. So why make it in the first place?

Like I said before, I'm using a lot of good faith here assuming your misunderstanding was a genuine one and that you weren't intentionally trying to derail the conversation.


> Being a hobby is a good enough end to itself.

Indeed, there's a word for it which I find particularly lovely, autotelic.


> There's much more competition than there was when Linux was released and that competition is much more mature too.

I think it's the opposite. When Linux was released, basically every major tech company had their own variant of Unix. Now almost everyone has migrated to Linux or Windows.

Competing with modern Linux by creating a drop-in replacement for Linux would be a pretty difficult task given that people have been optimizing Linux for decades now, but on the other hand I think there are ways to improve dramatically on the traditional POSIX-style API that Linux mostly adheres to. For a random example, why can't a given process have more than one "current working directory" at a time? That seems like an arbitrary limitation imposed by an implementation detail in early Unix system, and it causes problems for modularity. There are many other little details like that. I think if Linux is replaced by something eventually, it'll most likely be because the new thing has a cleaner, more powerful, or more generally usable API.

(Kerla is apparently not trying to invent a new API; I'm just saying that's the direction I would recommend to anyone project with a goal of seriously competing with Linux.)


Follow up to add: another way to compete with Linux is on security. The Linux kernel generally has a pretty good security record I think, but there have been plenty of serious bugs over the years. How many exploitable array-out-of-bounds errors or use-after-free errors remain in the Linux kernel? No one knows. If you can rule those out by using a safer language, that might be compelling to a lot of users who care about security above other concerns.

Of course that's hard to pull off in practice. Linux might have classes of errors that wouldn't exist if it were written in Rust, but even if using Rust eliminates three fourths of the code defects, the end result could be less secure if Linux gets ten or a hundred times as much scrutiny from people actively looking through the code for bugs to fix.


If OpenBSD has taught us anything, it's that when you need to start hardening at that level, C stops becoming weakest link and actually the design of the broader UNIX ABIs are the bigger problem. This is why things like selinux and cgroups exist in Linux -- POSIX ABIs are about as secure as Win32 APIs in Windows and thus you need to take additional steps to isolate your running processes if you really care about them behaving.


> I think it's the opposite. When Linux was released, basically every major tech company had their own variant of Unix. Now almost everyone has migrated to Linux or Windows.

There wasn't many UNIXes that targeted x86 aside from 386BSD. And even those that were available were often expensive. Even Minix wasn't free at that time. Which is exactly the reason Linus created his hobby OS.

Now we have a dozen different flavours of BSD, countless Linux distributions and several OpenSolaris spin offs too. Plus many other underdogs used in speciality domains. And that's before even looking at the commercial offerings like QNX, Solaris and macOS.

Just because there are a couple of industry heavyweights that take up the majority of common use cases, don't be fooled into thinking there is a lack of choice nor even that the industry is scared to use anything outside of Linux and Windows.

> Competing with modern Linux by creating a drop-in replacement for Linux would be a pretty difficult task given that people have been optimizing Linux for decades now, but on the other hand I think there are ways to improve dramatically on the traditional POSIX-style API that Linux mostly adheres to. For a random example, why can't a given process have more than one "current working directory" at a time? That seems like an arbitrary limitation imposed by an implementation detail in early Unix system, and it causes problems for modularity. There are many other little details like that. I think if Linux is replaced by something eventually, it'll most likely be because the new thing has a cleaner, more powerful, or more generally usable API.

That's not the goal of this nor any other project that aims for ABI compatibility with Linux and there are plenty of research kernels out there if that's your thing.

> (Kerla is apparently not trying to invent a new API; I'm just saying that's the direction I would recommend to anyone project with a goal of seriously competing with Linux.)

The only way to seriously compete with Linux would be to get large commercial backing. And even then, good luck. Linux won not because it is the best but instead because it's "good enough". Same is true with Windows. Any engineer with aspirations of creating a "better kernel" or "better OS" needs to remember that.


Thanks! I'll keep doing Just for Fun :D


Aside from "rewrite in rust" debate that I was unaware of, there seems to be this pervasive attitude among the HN hivemind that the end-goal for all projects, side- or main-, is "launch", and therefore needs a market analysis to decide the "worth" of such an idea, and it ends up being rather silly.

For example, I've written a Mandelbrot visualizer so many times I've lost count. Not because the world needs another poorly written or optimized rainbow-ladybug-simulator, but because it serves as a slightly-non-trivial hello-world. For example, it's the first end-to-end thing I made in Common Lisp. https://git.sr.ht/~amtunlimited/mandelbrot-plot


I have a A* search algorithm and a toy compiler that I use exactly for the same purpose.

I just rewrite them all the time as means to get a feeling about programming languages.

I dumped a couple of other stuff on GitHub so that HR people are happy to get a link that they never read anyway.

Then I get back to Java and .NET at the office. :)


I've actually been considering adding a toy compiler to my collection of getting-up-to-speed projects, do you mind sharing what you think are good features?


Try to follow along the Tiger Book, in the variant that appeals to you.

https://www.cs.princeton.edu/~appel/modern/

It covers most areas quite alright even if a bit aged.


Not OP, but there's also "crafting interpreters". In the second half of the book you emit bytecode for whatever language you designed in the first half, and also implement a VM for said bytecode.


This is really cool. Ignore the haters:

* Some people bake bread even though there's a good bakery nearby.

* Some people grow gardens even though there's a farmers' market down the street.

* Some people restore old cars even though there's a good restoration professional in town.

* Some people make k8s clusters on rpi even though they could rent that for cheap.

You're making an OS that's linux compatible even though there's already linux. And that is awesome!


Yes, and some people write an OS that is similar to Minix and won't ever be ported to anything beyond 386 and AT harddisks ;)


Wow, I really like the implementation of that syscall. I've got my own toy OS project in C, for the M68k, and my `read` is "a little" less clean [0, 1].

[0]: https://gitlab.com/0xTJ/mosys/-/blob/master/src/vfs.c#L469

[1]: https://gitlab.com/0xTJ/mosys/-/blob/master/src/vfs.c#L907


Thanks! By the way, your MUTEX_WITH macro looks pretty interesting to me. I've never seen the idea.


Even "rewriting Linux in Rust" is far from a waste if you make it run and if your code quality is good enough for others to join.

If only you could write an entirely new (meant to be better, for some cases at least) kernel compatible with Linux hardware drivers - this would be not waste at all but in fact fantastic.


The most challenging point is, as others said, the lack of the compatibility with Linux Driver API. I believe it would be really hard to implement and keep following changes in Linux.

An idea in my mind is to use Kerla for virtualized environments like KVM where only limited device drivers are needed (virtio for example).


>> The most challenging point is, as others said, the lack of the compatibility with Linux Driver API.

IMHO getting a minimal set that works is good enough to get people on board. This is still nontrivial. But for example, getting FUSE working would be useful. Even if the driver itself was not ABI compatible, it would bring functionality and someone might then aim for ABI compatibility afterward which would open even more doors.


> The most challenging point is, as others said, the lack of the compatibility with Linux Driver API. I believe it would be really hard to implement and keep following changes in Linux.

Especially since, as I understand it, even Linux isn't compatible with the Linux driver API across versions; they can and will change internals at will and just update in-tree drivers to match. Hence some of the difficulty doing things like getting a newer kernel on assorted embedded devices (ex. 99% of phones) because you have to port the vendor's drivers to the new version and both sides changed stuff.


Thanks for checking in. Makes me happy to see your excitement for learning the inner workings of things.

What do you think about other Rust OS projects like Redox?

https://www.redox-os.org/


In addition to its huge contribution to OS development in Rust, as a microkernel enthusiast, it sounds exciting to writing a microkernel in Rust, in the "Everything is a URL" principle. Moreover, it can run a good-looking GUI on a real machines [1]! I know it's very hard to be realized.

Aside from Redox, I'd mention Tock [2], an embedded operating system. It introduces a novel components isolation using Rust's power. I believe this field is where Rust shines and am looking forward to seeing it in production.

[1]: https://www.redox-os.org/screens/ [2]: https://www.tockos.org


There are more good efforts, the BeTrusted guys are working on Xous, its a microkernel for a phone like device called the Precurser.

https://github.com/betrusted-io/xous-core

As a embedded service processor OS for a big server rack, Oxide Computer is working on 'HubrisOS'. They seem to have not released it yet, but that will be open sourced.

https://github.com/oxidecomputer

Those are two efforts where I know real resources are going into.


Is Hubris targeted at BMCs? Or is something different?


Hubris is targeted at microcontrollers. Much more information (and, importantly, the source code!) will be available in our talk at the Open Source Firmware Conference[0], the abstract for which elaborates on our motivations and use case:

On Hubris and Humility: when "write your own OS" isn't the worst idea

Hubris is a small open-source operating system for deeply-embedded computer systems, such as our server's replacement for the Baseboard Management Controller. Because our BMC replacement uses a lower-complexity microcontroller with region-based memory protection instead of virtual memory, our options were limited. We were unable to find an off-the-shelf option that met our requirements around safety, security, and correctness, so we wrote one.

Hubris provides preemptive multitasking, memory isolation between separately-compiled components, the ability to isolate crashing drivers and restart them without affecting the rest of the system, and flexible inter-component messaging that eliminates the need for most syscalls -- in about 2000 lines of Rust. The Hubris debugger, Humility, allows us to walk up to a running system and inspect the interaction of all tasks, or capture a dump for offline debugging.

However, Hubris may be more interesting for what it doesn't have. There are no operations for creating or destroying tasks at runtime, no dynamic resource allocation, no driver code running in privileged mode, and no C code in the system. This removes, by construction, a lot of the attack surface normally present in similar systems.

This talk will provide an overview of Hubris's design, the structure of a Hubris application, and some highlights of things we learned along the way.

[0] https://talks.osfc.io/osfc2021/featured/


I think its an FPGA with a costume Open-Source Titan chip on it (RISC-V). It is not really a traditional BMC, its more like a service processor that does secure boot and gets you in the OS. It does a few other things but I think they really want it to have minimum functionality.

This is a great talk about what they do and why: https://www.youtube.com/watch?v=vvZA9n3e5pc


This is a very reasonable inference, as it absolutely was when I gave that talk. ;) Very shortly after that talk, however, we came to the realization that the OpenTitan was not going to be what we needed when we needed it, and moved to a Cortex M7-based microcontroller for our service processor (and a separate M33-based microcontroller for our root of trust); Hubris is the operating system that runs on those two MCUs.


Not that it matters all that much, but why a Cortex when you can get RISC-V chips form SiFive (or whoever) that do about the same stuff?

Does that mean the product will not have an FPGA? I kind of liked that idea of updating the hardware.

P.S: Twitter spaces about the history of computing are really fun. Love to hear more about all the dead computer companies you researched. There is not enough content about computer history out there.


Yeah... a bunch of reasons. We definitely have a couple of FPGAs in the product (Bluespec FTW!), and we anticipate that we will have more over time -- but not for the RoT or SP, or at least not now. The reasons are different for each, but include:

1. ASICs are out of the question for us for a bunch of economic reasons

2. FPGAs by and large do not have a good security story

3. The ones that do (or rather, the one that does) has an entirely proprietary ecosystem and a terrible toolchain -- and seems to rely on security-through-obscurity

4. Once you are away from a softcore, the instruction set is frankly less material than the SoC -- and the RISC-V SoC space is still immature in lots of little ways relative to the Cortex space

5. Frankly, there's a lot to like about ST: good parts, good docs, good eval boards, pretty good availability (!), minimum of proprietary garbage.

We tried really hard (and indeed, I think my colleagues would say that I probably tried a little too hard) to get FPGAs to be viable, and then to get FPGAs + hardcores to be viable, and then multicore hardcores to be viable (one for the SP, one for the RoT). Ultimately, all of these paths proved to be not yet ready. And while we're big believers in FPGAs and RISC-V, we're even bigger believers in our need to ship a product! ;)


Also, where are my manners?! Really glad you're enjoying our Twitter Spaces[0] -- and thank you for the kind words!

[0] https://github.com/oxidecomputer/twitter-spaces


Thanks for sharing!

I have to say, I feel a bit “dirty” carrying so much unused and legacy code around with Linux, so I like people trying to reinvent the wheel just for the pleasure of a fresh start. For the aesthetics. Even if it’s merely a fantasy and not replacing anything soon, realistically. They are also keeping OS development accessible to new generations of geeks. The unfriendliness of C, the gigantic codebase and seemingly distinct culture make the Linux kernel quite off putting, filtering possible engagement by unfortunate parameters IMO. Novel OS development in Rust takes away at least some of those barriers and some of the gained knowledge may be applicable with the Linux kernel later.


Spending your time how you want to is never meaningless. Especially if you’re creating value in the world.


> That said, what I'd say here is that it's fun. Really fun. Implementing ABI compatibility requires you to understand under the hood of the kernel.

> You can learn how printf(3) works. You can learn what happens before main().

Yes! It's such a wonderful experience. This is exactly what I most enjoy doing, just seeing how things work, maybe making my own version. I hope you have lots of fun.

You're reimplementing Linux's kernel-userspace binary interface, right? The system call interface is stable and language agnostic, it's really nice. Some pointers for anyone who'd like to know more:

https://man7.org/linux/man-pages/man2/syscalls.2.html

https://man7.org/linux/man-pages/man2/syscall.2.html

https://github.com/torvalds/linux/blob/master/Documentation/...


I honestly think there is real value here for firecracker style containers. Running memory safe code for your whole stack, with a minimal number of virtio devices? Fantastic!

The permissive license would also be interesting in some applications.


I'm new to Rust and am far less along than you are, as evidenced by this project. I noticed that in the few files I spot checked, there aren't many/any tests. Without any context or opinion, I'm curious whether unit and integration testing are hard for a project like this.


Good point. As you say Kerla lacks unit tests I focused on running a SSH server as soon as possible.

IMO, writing and running tests in the kernel space is pretty easy thanks to Rust’s flexible testing feature [1].

[1]: https://os.phil-opp.com/testing/


Really cool that you mentioned OSv on your github, really not enough eye's on it, i think.


It doesn't matter if it's meaningless to the hackernews crowd, it really only matters if it has meaning to you and if you're learning.


How did you set-up the ssh system? Where a new vm is spawned on ssh ?


nice project! curious how the rust ownership model works in a monolithic kernel context. are the lifetimes of kernel structures associated with a process "owned" by the process itself somehow?


I’m not sure this answers what you asked, objects that are referenced from multiple objects are simply wrapped with Arc<T>.


sortof. i'm just kinda curious if any of rust's cool memory management features work particularly well or have to be bypassed in a monolithic kernel context.


How long did it take you to write this?


About 3-4 months in total: took 1.5 month to run a simple Hello World program, 1 month to implement bunch of system calls, and another 1 month to implement features essential to run Dropbear SSH server (tty/pry, bug fixes, …).


Just to clarify goals of the project:

"I completely agree that rewriting existing large, feature-rich, and robust operating system kernels is not a good idea. However, a question came to my mind: what are the pros and cons of writing a practical UNIX-like OS kernel in Rust from scratch? How does it look like? This is why I started this project."[1]

[1] https://seiya.me/writing-linux-clone-in-rust


Rewriting the Linux kernel in rust would be a rehash of why we ended up with Linux in the first place: rewriting an older OS from scratch.

I would love to see a solid implementation of a real micro kernel based OS based on a message passing core in Rust to become a viable alternative to the now 50 year old Unix architecture. This would give us a possibility of some real progress on the security and process scheduling front, both of which are severely lacking in the Unix model.


Why do people implicitly assume that micro kernels are the best OS design?

In fact, micro kernels have a lot of issues and limit the possibilities of what an OS can do! One of the most important issues is they prevents resource sharing in a rich and efficient way between the components of the kernel. They make building rich relationships between kernel data structures and kernel subsystems very messy, complicated, impracticable and very inefficient.

Micro kernels are delusional and people should stop implicitly assuming their superiority.


Historically, one of the main arguments against microkernels is that having a kernel that's broken up into a lot of threads that each have their own hardware-enforced memory protection and use message passing to communicate with each other is dreadfully inefficient. All those context switches are expensive, and copying all that data around when you send messages is expensive.

Rust's memory model largely makes those isolated address spaces unnecessary, and sending a message at runtime basically amounts to copying a pointer. So, with these performance issues resolved, I think it makes sense to look at microkernels again. The reasons why Linus was opposed to writing Linux as a microkernel in the 90's don't necessarily still apply now in 2021.

There may some things the Linux kernel does that can't be efficiently expressed using a message-passing style, but I don't know if that's actually true or not.


> Rust's memory model largely makes those isolated address spaces unnecessary,

I think you're confusing several concepts. Rust's "memory model" is the C++11 memory model. Maybe you meant the borrow checker and the distinction between shared and exclusive references? Yes, those make it much harder to accidentally screw up other parts of your address space, but it's downright trivial to do it on purpose – you just need an unsafe block and std::ptr::write.

There's nothing about Rust that could replace isolated address spaces. Rust is typically compiled to native machine instructions, there's no sandbox or virtual machine that will protect different modules from each other.


I meant the ownership model.

I'm also imagining a usage model where all the various components of the microkernel come from the same place and have passed various minimum standards like "doesn't have any unsafe blocks". If some program really needs an unsafe block to do something, then that goes into a (hopefully small) library of heavily-scrutinized functions that do dangerous things but expose safe interfaces.

If we compare with the Linux kernel: in Linux, any part of the kernel can clobber memory in any other part; there's no protection at all. Users generally accept that because the Linux kernel is a pretty well-run project, but still bugs slip in from time to time. If instead you imagine a system where the compiler simply doesn't allow one part of the kernel to clobber memory that belongs to some other part, that would be a significant improvement over what Linux offers.

Anyways, security mechanisms don't necessarily have to be enforced at run-time if they can be enforced at compile-time. That assumes you compile the code yourself with a trusted compiler, or you get the binaries from a trusted source. (Though unfortunately Spectre has become a big problem for compiler-enforced data isolation within one big shared address space; I'm not sure whether compilers these days can do code generation in such a way that it doesn't suffer from sidechannel attacks, or if for the foreseeable future it's just something that software developers have to be wary of.)

In theory you might eventually be able to run device drivers from random people or run arbitrary applications in the same address space as the kernel, but that would require a lot of faith in the compiler to reject anything that could be a security hole. It's an interesting model, though; a pure unikernel approach everything shares the same page tables and there's no need to swap them in or out when context switches happen. And invoking a system call can be as cheap as executing an ordinary function. It might even be inlined by the compiler.


> If some program really needs an unsafe block to do something, then that goes into a (hopefully small) library of heavily-scrutinized functions that do dangerous things but expose safe interfaces.

Device drivers make up the largest part of the Linux kernel, and they would by necessity contain lots of unsafe blocks – the compiler can't reason about the interaction with some device. This is typically different from device to device, and there's no way this could be separated into a small library or module.

Look, I agree that a kernel written in Rust could be much safer than one written in C, and even those instances of 'unsafe' blocks would be easier to audit than C's "everything is unsafe". But it still couldn't replace all the advantages you get from micro-kernels with separate address spaces.

For example, it's impossible in Rust to safely unload a dynamically loaded library. The reason for this is, that such a library could contain static data, like string literals, which are of type &'static str. Passing a 'static reference from the library to the program doesn't require 'unsafe', but will leave a dangling reference when you unload the library. Of course, the unloading operation is itself 'unsafe', but that doesn't help you prevent, locate, or track the transfer of a 'static reference.


Exactly. Looking towards the language to keep you safe is essentially client side security, it's the memory model that isn't safe: dumping everything in on executable allows for all kinds of tricks to be performed. Only a micro kernel has with the present architectures the ability to isolate one driver from another in such a way that those drivers can not accidentally or purposefully attack each others address space.

Another advantage is that there is no such thing as a kernel crash, worst case you get a device driver that needs restarting. The whole loadable module model is broken by design and replicating it in Rust isn't going to solve that particular problem (and it isn't one that Rust is trying to solve).


The point was to use compile-time checks instead of a MMU to separate kernel services.

The concept is interesting (if maybe untested) it allows to have explicit opt-in sharing while blocking accidental sharing


> The point was to use compile-time checks instead of a MMU to separate kernel services.

Rust's compile time checks cannot replace the MMU. There were experimental SAS (single address space) operating systems which didn't depend on the MMU for protection, but they used "managed" languages, i.e., all code was executed in a sandboxed JIT.


I think we have to assume that people aren't putting malicious code into the kernel, don't we?


But that's one of the big promises of micro-kernels: You don't have to have 100% trust in the driver for that gamepad you bought from some unknown company.

What about that WiFi driver with 10,000s of lines of code? Are you absolutely sure there isn't some code in it that accidentally or purposefully leaks the rest of your kernel data structures to the outside world? With drivers in their own isolated address space, you wouldn't have to be.


That was the point behind Singularity OS[1] IIRC. Using managed code which could be verified safe by the OS using static analysis.

Can't find it right now, but I recall reading they implemented a network driver with minimal overhead compared to a traditional OS. While they used message passing, a lot of overhead could be eliminated due to the assumptions of safe code.

[1]: https://en.wikipedia.org/wiki/Singularity_%28operating_syste...


Message passing by itself doesn't have to be slow (you can use the paging subsystem for this purpose), but you are looking at two more context switches per system call.


The kernel will have a lot of "unsafe" (the keyword) code.


This is a limitation of the memory model, not the microkernel. If you had a single hardware-protected address space, kernel modules could share resources freely.

But this does argue against message passing, since such systems preferentially try to use messages over sharing whenever it's possible.


Message passing at first seems interesting and neat and useful. It is for some cases. But it comes at a cost of complexity and latency.


Just in case you don't know about it: https://redox-os.org/ could be interesting to you :)


Mach's internals didn't follow the Unix model, and was by almost any metric a failure.

There have been several other "real micro kernel based OS based on a message passing core", and they too, by almost any metric, have been a failure.

Process scheduling in Linux in 2021 is far, far ahead of anything in any other OS, ukernel or MP-based. The idea that there hasn't been progress in this area is really completely absurd. In fact, the general idea that Linux follows "the now 50 year old Unix architecture" is also pushing the boundaries quite a lot.


Meanwhile, the world runs on QnX. The fact that you don't realize it and don't see it is testimony to its success.


I know that QnX is widespread, but far more devices run Android than QnX. It's a bit of a philosophical debate whether the software that runs most smartphones "runs the world" more or less than the stuff covered by QnX, but I'd say that's its a bit of a stretch to just give the win to QnX :)

But you're right, QnX is the one (probably the only?) micro-kernel that has seen widespread adoption. Almost all of it has been in the context of embedded systems, which doesn't invalidate the substantial success, but it does leave QnX as the exception that proves the rule. It also leaves QnX as yet another example of the general failure of microkernels as general purpose computing platforms. QnX design is excellent for the contexts where it is deployed, but there's a reason you don't run it on PCs, tablets, data crunchers, smartphones and many other sorts of computing contexts.


Lots of devices run on Android but that's because there isn't a valid alternative. QnX and it's brethren are used where failure is not an option.

I used it daily on my PC for years and it was hands down the best environment I've used with distance (mostly compared to IRIX, Windows and Linux). Super responsive, never locks up, no weird delays it just works, 24x7, year after year. The whole throughput argument never worked anyway, that was just Linus talking about something that he didn't have direct experience with, it's latency that is far more important than raw throughput in interactive computing because interactive computing is a real time task.


> "real micro kernel based OS based on a message passing core", and they too, by almost any metric, have been a failure.

BeOS wasn't half bad. The failure was probably mostly commercial. It is true that they moved their networking stack into the kernel, but it's not entirely clear to me that they had to do that, it was just the most expedient way to get acceptable performance and stability given the (programmer-time) resourcing constraints of the project.


> BeOS wasn't half bad. The failure was probably mostly commercial.

Fair point. You could probably say this of other attempted microkernels too, to be even fairer. But that doesn't really change the fundamental point that just cooking up "a better OS design" doesn't lead to it being successful. There's a lot more in play that "mere quality".


well if that sort of "successful" is your key metric, this is true for a lot of things and has been known since biblical times: "a person may labor with wisdom, knowledge and skill, and then they must leave all they own to another who has not toiled for it"

you did say "by almost any metric".


the metric i was mostly thinking was inspired by the parent to my post, which seemed to me to imagine that Rust+microkernel => widescale deployment.


BeOS wasn't a microkernel.


Depends on your definition of microkernel. Maybe it wasn't pure, but It was in many ways closer to a microkernel than a monolith; sure the filesystem ran in ring 0, but it was scheduled onto its own threads by the scheduler just like any other process.


I should have (and did) know better. Thanks for the reminder.


Mach is of course doing quite well and moving back towards being microkernel-y as hardware gets fast enough to support it and Apple needs a better security story.


Written properly, a monolithic kernel in Rust would provide many of the benefits of a microkernel, due to the memory safety. It'd be much along the same lines as Microsoft Research's Midori, with software isolation instead of hardware isolation.


It would be wide open to specially crafted drivers, it's the equivalent of client side security (or maybe that should be 'compile time security'). Memory safety is something only process space isolation can bring.



I think I'm aware of QnX ;) (I wrote a clone).


I believe someone is building something like that on top of sel4, which gives even stronger correctness guarantees than Rust, even though it's C.


What are some ways I can increase my knowledge in this domain that the OP is very skilled at, meaning low level OS development?

I've taken an intro to OS class and am currently going through Linux From Scratch [1], which is interesting and is teaching me a lot, but it's more about how to setup a Linux distro using existing packages and not really about reading/writing the code involved.

Any recommendations?

[1] https://www.linuxfromscratch.org/


One of the easiest ways to get into writing your own kernel is to start with the OS Dev wiki [0]. Both the wiki and their forums are a great place for getting your feet wet.

For more general information, there are also a handful of articles and/or blog posts that I've come across when I was getting started that gave a very good introduction to many of the low-level components of how computers work and what the OS needs to do to run them. One of my favorites was Many But Finite [1]. Check out a bunch of his posts about the memory map, the computer boot process [2], cpu rings, etc... Not exhaustive by any means, but good, well-presented info. Many other such sources exist; maybe others will share their favorites.

And if you'd just like to get a more gentle introduction into actual OS code than the monster that is Linux, maybe try looking at some of the BSD's. NetBSD [3] and OpenBSD [4] are remarkably easy to navigate and follow from a code flow perspective, and if you just want to see how things work in a real OS, it's both a good reference and a good place to start.

[0] https://wiki.osdev.org/Main_Page

[1] https://manybutfinite.com/archives

[2] https://manybutfinite.com/post/how-computers-boot-up

[3] http://netbsd.org/

[4] https://www.openbsd.org/


Thank you!


I think you're on the right track. I am similar to you and not so skilled at low level OS development. There is an educational OS called PIOS from Yale's CS department with specific parallelism goals in mind, but it boots from metal so the code is a great resource. Here's the code: https://github.com/bford/PIOS

Also, you could try reading the Plan 9 source code.

(I would say the code is what you're after now, but in case you are interested in more of the theory of why it's designed that way, you can check out the research paper here: https://dedis.cs.yale.edu/2010/det/)


Thanks!


Linux from scratch won't really help you. I'd say start by looking into writing drivers for Linux and looking at tiny OS examples for microcontrollers that can be run in emulators or cheap boards. Bare-metal projects are available for pi and beaglebone. You will absolutely have to do something like that at a minimum, so it's good to apply what you learned in your OS class to that stuff.


This is awesome. Imagine the potential if it was just complete enough to run Kubernetes’ kubelet on a cloud instance. Lots of security minded folks would love it.


That's IMHO how you get something like this off the ground. Make it useful for a simple use-case while offering something of value - in this case the security of Rust. It can then become a preferred solution to some problem and that will bring users and developers looking to incrementally expand it.


I wonder, if you run every container in it's own MicroVM a-la Firecracker like Fargate does (or Kerla's demo host), does the security of the kernel still matter as much?


A minimal interactive demo running busybox is here:

  ssh root@kerla-demo.seiya.me


That’s very cool! Could you share some details on how to set it up for oneself with AWS Firecracker the same way that this demo was set up?

Also, as owner of the instances are you able to see what commands people are running? First thing I did was to type hello, and it said hello world to me :)


Author here. I've made the demo system public [1] but it's written just for me so it should be painful to set up the same environment.

The mechanism is pretty simple: a Node.js server (running on genuine Linux) listens on tcp:22. Once you connect, it boots a dedicated Firecracker microVM instance and forwards network packets between your SSH client and VM.

Regarding the command history, others (including I) can't see what you type. If you could, it must be a vulnerability.

[1] https://github.com/nuta/kerla-demo


Thank you :)


I'm not the project author, maybe you can ask via e-mail, see https://seiya.me/


Ok, thanks :)


is there not an opportunity to implement the posix filesystems interface on top of something other than a block device system. That would be fun.


Very cool!

I think a killer feature would be to emulate the Linux kernel device driver model such that existing C driver modules could be used with Kerla. Further, borrow (no pun intended!) the Linux device driver Rust wrappers from the Rust for Linux kernel project to then enable writing Linux device drivers in Rust.

The question is whether emulating the Linux device driver model (by which I kind of mean mimicking the set of core kernel services needed to support drivers - kernel threads, sync primitives, allocators etc) ends up being a massive task akin to writing the core kernel itself.

I think there might be reasonable trade offs that make the problem more tractable.

Having the ability to have a significantly simpler kernel with Rust’s memory and thread safety guarantees plus the ability to use even a small set of Linux’s massive set of drivers would be very compelling.


Linux drivers don't have a fixed ABI (or even API), making such a feat somewhere between hard and impossible. You'd have to constantly play catch up with the latest kernel refactorings, which would require a tremendous amount of engineering effort.

Heck, it'd probably be easier to emulate the Windows driver architecture - at least those have proper ABI compat.


I asked this question during the recent Linux kernel plumbers conference: "How often in practice do the kernel A{B,P}Is fro device drivers change ?". I got a very mixed response.

My take away was that while 'new' driver classes/subsystems do involve more churn in the core kernel support subsystems, there is a clear trend towards convergence in terms of significant change.

If the change truly was significant and continuous then that would impede productisation with the Rust kernel. I'm not saying that the A{B,P}Is are in any way stable but I personally don't think that keeping up will be an intractable problem.

In the spirit of Kerla's authors prime motive - which appears to be hacking for knowledge - this might be something to try, perhaps with a driver class that is known to be more stable.


FreeBSD’s linuxkpi - bits and pieces of Linux KAPI provided to make porting Linux drivers to FreeBSD easier - works just fine, so this assumption is wrong.


>Heck, it'd probably be easier to emulate the Windows driver architecture - at least those have proper ABI compat.

And better drivers.


Actually many Linux drivers are far better than their Windows counterparts - many Windows drivers are complete shit. It is entirely dependant on your particular hardware. I know the driver for my WiFi in my laptop is far crappier in Windows for example.


> The question is whether emulating the Linux device driver model (by which I kind of mean mimicking the set of core kernel services needed to support drivers - kernel threads, sync primitives, allocators etc) ends up being a massive task akin to writing the core kernel itself.

The kernel has (deliberately) never had a simple or stable driver ABI, so I think this is potentially a huge task. And then you've also left a C-based attack surface in place.

I'm not sure what the minimal set is if you decide to only support 2021-era hardware. PCIe+SATA+framebuffer+USBhost+USBHID?


Towards the C-based attack surface:

I think using the Rust-for-linux wrappers to implement the drivers in Rust would eventually help.

Also, even if initially Linux C drivers were transplanted into Kerla that could be a net benefit if the tradeoff of wide device driver availability vs increase in the trusted compute base was acceptable.

I personally think that tradeoff would be acceptable - if the transplanting ability was tractable. It's a hard one to absolutely agree on, admittedly.

Towards the minimal set of hardware, fair point! I think that one should be able to get quite far with judicious selection of device driver classes and leave the truly in-flux classes alone ?


Maybe starting with the rumpkernel interface is easier and one could already use some of the NetBSD drivers: https://en.wikipedia.org/wiki/Rump_kernel


That's a very good point indeed.

While I completely agree that NetBSD's Rump kernel would be a great way to go (I've played with it in the past and quite like it!) I do wish there was a similar initiative bound to the Linux kernel.

The closest I've seen is Octavian Purdila's Linux-Kernel-Library project: https://lkml.org/lkml/2015/11/3/706

Not sure how far they've come though. Perhaps Kerla could be a prime mover for them ?


Dunno: just a hobby, won't be big and professional like linux.

:)


This may be a naive question, but I cannot help but wonder: As C and Rust aim to be link-compatible, wouldn't it be easier to gradually replace parts of the Linux kernel with Rust code?

The only technical problem I can think of is that Rust may not be available for all CPU architectures Linux supports, but this is just speculation on my part having done no research on the matter.


The Rust for Linux kernel project aims to enable writing Linux kernel device drivers in Rust.

See: https://lwn.net/Articles/862018/

The issue of Rust's LLVM based compiler toolchain not targeting all CPU archs is intended to be solved by the gcc-rs project.

See: https://lwn.net/Articles/871283/

I think the approach the Rust for Linux project is taking is wise: Not about outright re-writes but more about focusing on those subsystems where Rust's intrinsic safety and security properties helps the most.


Last I checked, one of the big problems was the Rust panics when you run out of memory, which is unacceptable in the kernel. Is there any progress on that?


Out of the box Rust doesn't provide any way to allocate heap memory, so, you can't run out if it.

Your ordinary userspace apps use std (the standard library) which relies on the alloc crate, and that provides heap allocation which indeed panics if the allocation fails. Because it correctly guesses that your "clever" strategy to handle allocation failure actually isn't and will just triple fault anyway so it should cut to the chase.

The kernel obviously doesn't have std, and Rust for Linux implements its own alloc crate.

Linus' requirement that you can fail memory allocation just means all the calls in alloc that can actually allocate memory now return Result to indicate whether the allocation was successful. This isn't how you'd do things in userspace, but, this isn't userspace so fine.


Can you elaborate as to why "his isn't how you'd do things in userspace, but, this isn't userspace so fine" holds?

Naive me - not a kernel dev at all - would argue that returning Result<Memory, AllocationError> is always better, even for userspace because it would allow me to additionally log something or gracefully deal with this.

Even if I don't want to deal with it, I could just `.unwrap()` or `.expect('my error message')` it.

Note: I am not trying to be snarky here, I genuinely don't know and would like to.

If answering this is too complex, maybe you can point me in the right direction so I can ask the right questions to find answers myself? Thanks in any case!


> it would allow me to additionally log something

If you don't have any memory your allocations are all failing. When you assemble the log message, the allocation needed to do that fails. Bang, double fault.

Now, often people don't really mean they want allocations to be able to fail generally, they're just thinking about that code they wrote that reads an entire file into RAM. If it was a 100GB file that would be a bad idea. But the best answer is: Guard the allocation you're actually worried about, don't ladle this into the fast path everybody has to deal with on every allocation.


People say that "well if allocations fail all bets are off" but can't you pre-allocate memory for error handling?

Like sit down, figure out all the things you'll want to do on an allocation failure, and once you have determined that you slice a little chunk of memory when you start your app (and maybe _that_ fails and you can't do anything). and when you hit a failure you do your think, then tear stuff down.


It's what we used to do in the days when 4MB was a lot of memory. Batch programs would just abort but interactive programs had to have enough reserve to fail gracefully, possibly unwinding and releasing things until they could operate better.

Now that I see interactive programs taking a gigabyte and the system being ok, I guess we're in a different regime.


What if the failed allocation came from a different allocator/heap than from where the allocations for string logging came from?

In general Don't assume anything about your global process state just because one allocator fails.


Mhm, thanks.

It never occurred to me (being in non-embedded land) that returning an enum as the error or a &'static str instead of a heap structure like String, could also fail.

Seeing that Result isn't part of core, but of std, this makes sense.

Just to tickle my nerve though: theoretically speaking, with your example, it would work, right?

I couldn't allocate 100GB (because OOM or not even enough RAM to begin with) but it could be that the system can allocate the needed memory for error message just fine.

Very interesting.


Result is part of core [0]. Result data and/or errors can be stack-only data. The parent was just saying that many people that say they want to guard against out-of-memory issues aren't cognizant of just how difficult that is.

Add to that that several operating systems will lie about whether you're out of memory, so the 'error' or failure will often not be on the Result() value but come in a SIGKILL instead, it's just adding complexity.

People that are actually worried about it and no how to deal with it, will be coding with a different style and can use the alloc library where/when they need to. (at least when it gets stabilized in Rust)

[0] https://doc.rust-lang.org/core/result/


Thanks for correcting my error.

I've never checked core before, so I did when checking up for this discussion.

I somehow missed Result. Silly me didn't search on that page, but ofc I found it on std

https://doc.rust-lang.org/std/result/index.html

Also thanks for clarifying that values of Result can be stack-only!


Tialaramex answered this in their post already, and you almost answered the question yourself:

> I could just .unwrap() or .expect('my error message') it.

Panicking can allocate. Allocating can fail. Failing can panic. Panicking can allocate. Allocating can fail. You can bite yourself in the ass like a real Ourobouros.

IMO, a prerequisite to using fallible allocation APIs should be attempting to write your own allocator, handling the weird and wacky problem of initialising a data structure (for the heap) in such a way that if it fails, it fails without allocating but leaves some hint as to what went wrong.


Oh, wow, I was under the impression that the error message would be stack only, no heap involved, but as Result is part of the std library and not of core, this totally makes sense.

So for `Rust for Linux` they also need to implement a `Result-like` type that is stack only based to solve this issue, right?

If so, cool, thanks, you just made my day by tickling my learning nerves! :)


It has nothing to do with Result, whatsoever. Result does not allocate. If you used a Result that way, you could certainly try to "gracefully" handle the allocation failure, but if you think it would be easy, you would be wrong. As Tialaramex said, you are probably just going to make the problem worse because it is very difficult to ensure you do not attempt to allocate during allocation-failure-recovery. Rustc doesn't and can't really check this for you.

It actually has to do with `panic!(...)`. When you use `unwrap()`/`expect("...")`, you use the panic macro under the hood; parts of the panicking infrastructure use a boxed trait object which could contain a static string or formatted String or anything else really. The box can allocate if it is not a ZST. I believe the alloc crate's default handler tries to avoid this kind of thing, so that it can't fail to allocate AGAIN in the failure-handling routine. It will likely do a better job than you could.

This is a live issue at the moment, so to go into any more detail I'd have to read a bunch of recent Rust issues/PRs.


An addendum to tie this back to the original discussion: the reason kernel devs want these APIs more than userland is that (a) in a kernel, panicking = crashing the computer, which would be bad, and (b) they have a much bigger toolbox for handling OOM.

They can kill entire misbehaving processes. What are you going to do in your little program, clear a cache whose objects are sprinkled evenly across 150 different pages? You would need more control than you get from blindly using malloc/free/rust_alloc globally. Something like memcached would be able to use these APIs, because it uses its own allocator, and knows enough about its layout to predictably free entire pages at once.


> panicking = crashing the computer

That isn't very accurate. In Rust when programming in no_std, you can (must?) define your own panic handler:

https://doc.rust-lang.org/nomicon/panic-handler.html

Which you would define in the kernel. While I'm not going to speculate on exactly what the implementation would look like, you definitely do not need to "crash" the computer. I haven't done any kernel programming, but I'm guessing the kernel could do some things at that point with shared memory space that is already allocated to deal with this situation and try to recover in some way.

Edit: for example, I just found this in the kerla project: https://github.com/nuta/kerla/blob/88fd40823852a63bd639e602b...

That halts now, but it probably doesn't need to, or could do it conditionally based on the contents of PanicInfo.


Mm no, it's pretty accurate. For a start, notice that the Linux community has been very clear that panicking is unacceptable. The reason is that they cannot realistically do anything to recover.

> panic handler [...] Which you would define in the kernel. While I'm not going to speculate on exactly what the implementation would look like, you definitely do not need to "crash" the computer.

The panic handler loses so much of the context that crashing the computer is the only thing you can practically achieve. You can't retry an operation generically from with a panic handler, it doesn't know anything about the operation you were attempting. The OOM handler gets a Layout struct only. You could try unwinding or something, but within a syscall handler, I don't see how anything good can come from that. Unwinding in the kernel is simply a terrible idea. What else are you going to do?


I disagree that PanicInfo loses so much context. PanicInfo caries an arbitrary payload of &(dyn Any + Send).

Now there is a lot that the allocator could do. If you wanted something to be retriable, it could be interesting if the thing that failed was an async task. If so, that panic info could carry enough information to say, the failure was an OOM, here’s the task that failed, and it is marked as retriable. Yes, this would require a store of tasks somewhere in the kernel. Then based on it being an OOM, see if any memory can be reallocated before retrying, or wait to retry until it is.

This is where theoretically a new async based Rust kernel, especially a micro-kernel, could be interesting. Is stack unwinding in the kernel a bad idea? Maybe. Can it be done in Linux? Maybe not, maybe it’s too much work to track all this information, but I disagree with the conviction with which you right it off.


Thanks for the thorough explanation!


Result already is “stack” based, or sized. It also already exists in core: https://doc.rust-lang.org/core/result/index.html

The Error type currently isn’t in core, but for other reasons, that just got resolved: https://twitter.com/bascule/status/1452029363197784071?s=21


AFAIK one other thing to note is that in Linux userspace, malloc (or fork) might succeed, but accessing the memory later can fail because of memory overcommit.


Yes, this is called "fallible allocations." You add methods with the "try_" prefix that work like the existing methods, except they return a Result which fails if it's out of memory instead of panicking.

We have a light / temporary fork of the Rust stdlib allocator with fallible allocation support: https://github.com/Rust-for-Linux/linux/tree/rust/rust/alloc

See e.g. https://github.com/Rust-for-Linux/linux/commit/487d7578bd036...



>> one of the big problems was the Rust panics when you run out of memory, which is unacceptable in the kernel.

If the kernel is running out of memory, IMHO that's a bug. The kernel is ultimately responsible for memory management right? It needs to prioritize itself over everything else or the system is in trouble.


This sounds like a design choice that was made decades ago and permeated everything else in Linux. I don't think it's changeable at this point, if they wanted to.


That's what may happen with the Linux kernel over time, now that it allows parts written in Rust.

The limitation of that approach is that all internal interfaces (which includes most data structures) have to be C-compatible. You can get much of the safety benefits of Rust, but lose a lot of Rust's ergonomics.


I don't think that the Linux kernel has decided (yet) to allow rust code into the kernel proper, has it?


It hasn't. But the attitude towards using Rust for device driver development has now gotten very positive, especially post the recent kernel summit.


An LWN article about Rust in Linux that's very relevant to this discussion: https://lwn.net/Articles/829858/


Google has decided to go forward anyway.

Just like Android Linux compiles just fine with clang for the last five years or so, it now makes use of Rust.

https://source.android.com/setup/build/rust/building-rust-mo...

If upstream ever cares to support clang or Rust, that is another matter.


Unless I'm missing something, that link refers to building Android user-mode components in Rust, not kernel components in Rust.


That's correct, but it came after them rebuilding the bluetooth stack https://news.ycombinator.com/item?id=26647981


Yes, gradual replacement is easier and more practical for large projects. However, typical C APIs and idioms are different from idiomatic Rust. Rust uses more type-system features, generics, iterators, RAII, and prefers tree-like data structures and far fewer pointers (no linked lists!).

C rewritten in Rust is still very C-like, and requires refactorings that you can't do until it's all Rust.

BTW: There are two GCC-based Rust implementations in the works, so compatibility with exotic platforms is going to be solved.


A practical problem is that in order to gain the benefit of linking Rust and C code, one has to give up Rust's wonderful guarantees at the interface. So until islands of Rust meet up, these interact through a C ABI, and have to expose C-like behavior.


But you still get those guarantees and benefits in the Rust implementation itself, even if not at the interface (to non-Rust code). By a similar argument, a standalone Rust program "gives up" its guarantees whenever the process makes a call to libc or to the operating system (syscalls), but this isn't really a practical problem.


Sure, I agree. The point is that if the total corpus of Rust code is too isolated into too small islands, then the boundary effects of each island may eat up a lot of the benefit.

I still think the project is really cool and worthwhile.


The world really needs _two_ projects, a practical one that is interoperable with the here and now, to get incremental improvements in security from bits reimplemented in Rust.

And then, in the longer term, someone ought to write a book about operating system implementation in Rust that ignores Linux interoperability and focuses on readability, maintainability by showing use of Rust's idiomatic style.


It’s not naive, it’s just been speculated and talked about incessantly since Rust came out. The one naive part is underestimating how long that would take - Linux is one of the largest codebases on Earth. It would require massive coordination over years to rewrite, and in the end all you would get is the exact same product.

It’s just not an interesting or unique perspective, and pretty silly to suggest seriously.


The Linux kernel is massive, so writing any part of it in Rust is an enormous undertaking. Even a few functions at a time, we're talking decades.


Well, writing a kernel from scratch that supports as many CPU architectures and devices as Linux would take about as long, assuming one could attract a critical mass of developers.

Using an incremental approach at least gives us some benefits in the near future.

I'm not even saying a fresh start would be a bad idea. But incrementally replacing parts of Linux seems like a more promising approach, IMHO.


Kudos, it works and thee source code is a beautiful read.

Thanks so much for sharing!


For each piece of software, there just be a version in Rust, and that version must advertise it's rustiness


After looking at the source I find this one to be much easier to read than equivalent C, because Rust is a more expressive language. So looking at what an all-rust kernel would look like seems quite worthwhile. You can't reach anything new if you're not willing to experiment.


The type of software makes the difference - does it process untrusted input, maybe even over the network? This here is a kernel including a memory-safe TCP/IP stack (https://github.com/smoltcp-rs/smoltcp/), and not having it crash or be full of security vulnerabilities due to preventable memory corruption is a quality beyond personal language preferences.


It is possible to overstate a valid case until it becomes meaningless... are modern OS IP stacks written in C really 'crash'ing or 'full of security vulnerabilities'? No.


There has actually been a few crashes in the core TCP/IP stack of both Linux and Windows in the last 5 years, though I'm not sure any of them were due to memory safety (most were logic bugs as far as I can tell). That said, rust's approach to error handling could help avoid crashes here as well.

However, we have seen *many* memory-safety crashes in the drivers to talk to network interfaces. Those really are full of security vulnerabilities. The most prominent recent one was broadcom's wifi driver having heap buffer overflows allowing remote code execution by anyone within wifi range in 2019[0].

[0]: https://www.bleepingcomputer.com/news/security/broadcom-wifi...


The fact is that it is overwhelmingly more difficult to write and guarantee that your C code is "safe" to the same extent that a naively-written Rust program which accomplishes the same thing would be, especially as your software's complexity increases.

Is it possible to write C code that has the safety properties of Rust code which provides the same functionality? Absolutely, in theory. But will you be able to accomplish this and then guarantee this safety and have the confidence that future code changes won't break those guarantees? That becomes exceedingly more difficult to do, nevermind the extra time and effort it would take. In most cases this would be prohibitively expensive, and you'd just get on with life hoping for the best and mitigating what you can.

In my view, one of the killer features of Rust is its ability to be easily ABI-compatible with C as needed, which I see as a pragmatic feature of Rust that acknowledges the messy software reality of our world and the fact that C (or any other language) isn't just going to be "replaced" by rewriting stuff everywhere, neither anytime soon nor probably ever in totality. This lets us more readily combine the great capabilities of Rust with the enormous volumes of useful code that already exist today without resorting to dogmatic approaches of always thinking we need to rewrite the latter. The "rewrite everything in Rust" idea is fun in an academic kind of way, but in the real world economics will prevent that from happening everywhere, in a similar way that economics usually prevents you from validating and providing certain safety guarantees about your C code.


The TCP/IP stacks of the few popular OSes is not full of security issues, but that took a lot of effort that a small team can't do. Also, it's a matter of updating: after reviewing the CVE database, would you be comfortable running a 5 year old kernel without updates?


For each rust submission, there shall be a comment at top lamenting the gradual rusting of hackernews.


Not lamenting in this case, the project is awesome and rust is made for this. I just find amusing that almost all popular rust projects I see have "in Rust" front and center

Must be an awesome language to work on.


I mean honestly who cares? its their time. I use rust over c/c++ these days for practical reasons.

1. the toolchain is better. (cargo ala)

2. maintenance of the software is better. (static linked binaries, easier deploys)

3. memory safety.

4. resource consumption (cpu/ram)

when I don't need 4 I use golang (most cases) because of 1, 2, and 3.

and I choose tools (all other things being equal) written in rust for the same reasons.


Me too, but in my case it's despite of point 2.


Now ask yourself why.

Mozilla decided to create Rust after years of using C/C++. Why?

Because they spent a fortune fixing the same problems over and over again, and they realized that no matter how clever their developers were, those problems were always going to arise as long as they kept using C/C++.

Not because C/C++ is inherently bad, but because is more flexible than strictly needed in some ways.

Rust was designed to prevent many classes of problems.


I'm ok with things rewritten in Rust, really looks awesome as a language. And the fact every rust project advertises rust front and center compounds that image.


A star has been born. A Linux alternative needs to be compatible with Linux binaries and also support modern architectures.

If you want Linux to survive, it needs to be re-written from scratch, optimised in Rust and compatible with the existing Linux binaries or drivers.

This project might have a chance.

From: [0]

> TL;DR: I'm writing a Linux clone in Rust just for fun. It does NOT aim to replace the Linux kernel.

A certain someone thought that their hobby OS Minix clone won't be 'big or professional like GNU'.

It's time for 'change'.

[0] https://seiya.me/writing-linux-clone-in-rust


You think Linux, which the vast majority of web servers run today, is going to die out if it's not rewritten from scratch in Rust?

How on earth have you come to that one?


sigh...

Linux is going to survive. And no it does not need to be re-written from scratch in Rust. Someone lied to you at the church of rust.


But its a fun thing to do, and that's a good enough reason to do it. After all is that not exactly how Linux came into being anyway.


Linux came into being as a fun hobby project. It was successful because it gained traction with the wider community and got huge commercial backing because it was vastly superior to most of the alternatives from a user perspective ("oh hey, I can run my web servers on this 100% free kernel plus the GNU userland and have a mostly good server for zero software cost on commodity hardware"). But today, Linux itself exists, and while there are underlying technical changes that could improve on it, from an end-user perspective (especially potential commercial backers) anything else would be barely different, if they could tell at all. You'd need some huge compelling advantage - that a user cares about, nicer code and 5 bugs a year instead of 10 bugs a year isn't going to cut it - to actually gain any ground.


It gained commercial tracking because it represented a way for IBM, Oracle and Compaq to reduce development costs on their own systems, and thus started sponsoring Linux development efforts around 2000.


Maybe I will write a TODO app in Rust, should it be on top of HN because is some unoriginal,incomplete,bug filled,toy stuff just because the reason is written in Rust? 1

My assumption is that all titles with Rust get blind upvotes, sure a new kernel is cool but would be more cool if is more then someone toy project, this should probably be posted and upvoted in Rust forums so it gets support from the fans and hopefully the Rust guys will decide what kernel should they focus on and stop promoting every student weekend project here.


I mean, if you had a headline like "Show HN: org-mode in Rust" you'd get a ton of clicks


Sometimes, just sometimes, I wished one could post soyjaks on HN.


Absolute codswallop. Linux will be here for many, many years to come yet.


Absolutely! But in Rust.

Why do you think the Linux developers are bolting on Rust in the Linux kernel themselves? [0]

It has already been admitted.

[0] https://lore.kernel.org/lkml/20210704202756.29107-1-ojeda@ke...


They've been writing drivers and such for years now and they've indeed levelled up the tooling integration. Your original post said along the lines of "if linux is to survive". Even if it was a full rewrite in rust (not going to happen) then it's still be Linux, with the same leadership. Most users wouldn't be able to tell the difference.


It'll take decades for Linux to have everything re-written in Rust. Even then it'll take even longer for every machine to run RLinux.


Not with Apache v2 License.


Post it when it's production ready.

A big problem in the Rust ecosystem is the lack of continued work and support on libraries (understandably, since many are new and aren't commercially supported).

For example, pyroute2 is still better for Linux admin that the equivalent Rust crates. Whilst the latter often fully support async, none of them cover all of the features of pyroute2 nor the easy-to-use API.

Hopefully things will improve over time, it's always a shame to see great projects be abandoned.


> Post it when it's production ready.

This is Hacker News, not Enterprise IT News.


I just meant that even big projects posted here in the past like the Headcrab debugger: https://github.com/headcrab-rs/headcrab have been seemingly abandoned now.

Even Rocket https://github.com/SergioBenitez/Rocket seems to have greatly slowed development unfortunately.


Maybe those projects are considered "done".


Nowadays the Rust http libraries to use are warp or axum (but Rocket works as is)

https://github.com/seanmonstar/warp

https://github.com/tokio-rs/axum

re headcrab, https://github.com/headcrab-rs/headcrab/issues/132


> Nowadays the Rust http libraries to use are warp or axum

Warp and axum are certainly not the only choices, and saying they are the best choices is a matter of opinion. Actix-web, rocket, tide, warp, axum are all great web frameworks.


The project author asks for help at the end of the blog post¹ announcing it, if you want to have a production ready kernel, maybe invest some time?

¹ https://seiya.me/writing-linux-clone-in-rust




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: