It's interesting to me that there is not as much push back against Rust from the kernel. Linus (I guess everyone addresses him with his first name), has been very critical, even dismissive of systems programming languages other than C. But with Rust he has a more, I guess we could say passive approach. This might be a good indicator that Rust is doing something very respectable.
On the other hand I'm slightly suspicious of this movement. Rustaceans are very, very eager to promote the language. It would be interesting to know how much this movement comes from within the Linux kernel community or if it is driven more by the Rust community.
In any case the language seems to have potential of becoming a new mainstream systems programming language in a few years.
This is a very weird comment, of course the people driving this have a long history with the kernel... It takes about 2 minutes of using git to find out that Miguel's (the main author) first commit to the kernel was back in 2006, and that he's the maintainer of multiple different drivers.
The rust community is naturally happy to help out, but I can't even imagine us being able to drive this sort of work...
To be fair, I was just a teenager back then and I was out of kernel development for quite a few years afterwards.
But you are right, this is not coming from the Rust community itself. In fact, several independent groups have been thinking of using Rust in one way or another for kernel purposes, as early as... 2013 (!), e.g. from Taesoo Kim:
There were others: John Baublitz had a external kernel module going on, Samantha Miller also worked on enabling a FUSE-like API to write filesystems in Rust, etc. And, of course, Alex Gaynor & Geoffrey Thomas had the most mature Rust abstractions for external kernel modules.
At the moment, in the project we have a mixture of people that come from a Rust or kernel background, which has been great. And most of us have used many languages in our work, not just Rust. Personally, most of my career has been around C++ and Python.
It has an undertone implying some sort of strange conspiracy amongst rust people. Generally to be suspicious of something means there is something nefarious going on.
It has an implication that Linux is something that any enthusiastic group of outsiders could force their changes into. And that there are rust people would create such a group of outsiders.
It has an implication that rust people and linux people are disjoint sets.
It's generally taking this strange adversarial view to a collaborative endeavour. People aren't "rustaceans" or not, it's not a sports team. People are people, they generally program in a wide variety of languages, and use whatever they think is best given a certain context.
I'm not sure if I've fully captured what seems "off" to me about the comment, but hopefully you get the gist.
As for asking questions, that's a clear example of https://rationalwiki.org/wiki/Just_asking_questions - but at the same time the general sentiment seems to be frequently reflected in internet comments. I don't think it's a malicious case of this, just how they chose to express their opinions.
It seems like legitimate questions from an outsider to me. As an outsider from both kernel and rust development myself, I also had the same questions going through my head. Specially given Linus' previous opinions in using other languages for kernel development. That, allied with how eagerly rust is hyped, does make one wonder how much is self-promotion and how much is actually the linux kernel community as a whole embracing the endevour (not just the subset of kernel developers who also happen to really like rust). I feel like they are honest questions. Someone more in touch with the kernel community should be able to answer them.
Linus’ objection to C++, if you boil it down, is that it adds a lot of complexity without fixing what’s fundamentally broken about C: memory safety. Rust’s memory model is much safer than C’s, and it is a significantly simpler language than C++ as well.
When I think about Linus’ lack of objection to Rust I immediately think about his vocal disdain for C++
When I learned rust I thought it felt more C-like than C++ to be honest. Nowadays I’m pretty happy to be in this c to rust to c loop.
I think you’re exactly right about c++ not offering much benefit to someone like Linus. It’s much simpler than c++. Rust has also informed the way I write C (granted, I’ve also gained more experience along the way, but rust forced me to have more discipline for sure).
You can't just make up stuff and go with it, you need to be a little more complete with your analysis. Especially worded the way the other comment was and your comment is, you seem to think there is some sort of confusion because you're lumping in one programming language with another and essentially saying, I don't see the difference therefore something sinister may be at play. Your ignorance is not an excuse. Either know what you're talking about or ask questions without your very, very leading premises.
To clarify, I didn't mean suspicious in the sense of expecting malicious behavior, rather than in the sense of not being overly optimistic about this type of progress, because I think it is very hard to incorporate new tooling/languages despite their merits.
In other words: it would be nice to see this being fruitful, because it would possibly a strong, positive signal for programming and programming languages in general, but I'm cautious or "suspicious" still.
Please don't take my above comment too harshly by the way, I know it could be read as a list of pretty hard criticism, it's really just trying to dive into why the tone seemed off when there isn't one smoking gun. It's a list of the comment could be read like this, not that the comment actually means this.
That's not my definition of long history! My first Linux kernel was just pre 2.0 and I used make config. Actually I used make config for quite a while before it got out of hand and I started keeping copies of the config to work from instead of doing it from scratch each time I did an upgrade.
Quite. There's a term, often used by engineers (Chartered, obviously): "time served". C is time served - it might be a bit weird and start dribbling for no reason but it quite literally runs a huge part of the world.
Rust is not time served and it needs to grow up. Initial signs are positive. We are at the cooing stage over a pretty child that may either be the Messiah (a Naughty Boy) or decide to throw its poo at its parents and stick to licking windows.
> It would be interesting to know how much this movement comes from within the Linux kernel community or if it is driven more by the Rust community.
You make it sound as if those are two non-overlapping sets. I don't think they could pull off this project without intimate knowledge of the Linux kernel.
The biggest pushback was with C++. Why? Because when these discussions still started, C++ codebases tended to be a mess. It didn't help that C++11 was coming up and support was wildly inconsistent among compilers. Ultimately it was still possible to write unsafe code though.
The landscape has now changed and Rust has now positioned itself as a language that actually does have very key advantages (in correctness and safety). While C++ has made large strides, its advantages now lie elsewhere (eg; drawing on a wealth of performant libraries-- something that the kernel has less of a sue of).
If you look at Linus's criticism of C++ (e.g., https://lkml.org/lkml/2004/1/20/20), he criticizes a few specific things about the language - exception handling, implicit allocations, complex OO. Rust, as it happens, doesn't do any of that. Errors are passed back via return values (which is what the kernel already does), memory allocations are explicit by convention and "zero-cost abstractions" are favored, and the OO model doesn't have multiple inheritance or anything, just interfaces (which is also what the kernel already does, in C, using structs of function pointers).
So I'm not surprised that he doesn't have similar harsh words for Rust. It solves a lot of the issues with C that have been causing practical problems for the kernel (apart from confusing aliasing rules and confusing overflow rules, there was also the problem a while back where a bunch of != NULL checks got optimized out), and it doesn't have the problems C++ has - in fact it tends to solve those use cases in a way similar to the kernel's existing approach.
For my part, I've been involved a very tiny bit in Linux kernel development for well over a decade because I find systems programming cool, and I've also been involved a very tiny bit in Rust starting around 1.0 because I find systems programming cool. I started this project because I genuinely thought Rust would be a good choice, not because I was deeply involved with the Rust community and I wanted to evangelize it.
(I looked up the IRC logs where Alex and I decided to start working on this, and the context was that Alex had run into some bug with buffer length handling in the kernel, he was considering sending in some patches to improve the C abstractions about this, and since PyCon was the following week I semi-jokingly asked if we should instead spend our sprints time writing support to ship kernel modules in Rust. Which we did.)
The allocations thing in particular. You can watch serious C++ practitioners staring at a class definition and usage and trying to figure out answers to a quiz question, "How many allocations does this do?". Is that line a Move or a Copy? That's a bad sign for a language you'd supposedly want to write OS kernels in.
Rust got luckier here, by the time Rust was invented programmers knew they really care about this, and so Rust explicitly distinguishes Copy (this object is just bits, so you can duplicate it by copying the bits), from Clone (this object can be duplicated, but you need to call its clone() method to explicitly do that work) and gives Move semantics to all assignments of types which do not have Copy.
So in Rust "let a = b" definitely isn't doing memory allocation, it just moves bits. If you misunderstand the situation and assumed b is still valid because it was Copy, but actually it isn't, the compiler has your back and will politely refuse to let you use variable b after it was Moved. If you did still need b you can write "let a = b.clone()" and now they both still exist but anybody can see this code might be doing something expensive 'cos it calls this clone method. And if the compiler says "Ahem, you can't Clone that" then you've just learned something very important about the thing that a minute ago you assumed was just a few trivially duplicated bits.
Given that this is the kernel, using typical C++ libraries would be out of the question, so it should be easy enough to design things right from the start.
C++ is after all used in embedded contexts where allocations are forbidden and it offers excellent support for this use case.
In such a project where allocations matter one would not stare at class definitions, one would ensure that there are no allocations by forbidding use of libraries like the STL and for example disabling operator new and the copy constructor. If the project is safety-critical there are tools which can enforce such rules.
Edit: I've clarified that I was referring to C++ libraries only.
Ah, I think there is a misconception at work here.
The problem isn't "Kernel code should not allocate" but "Kernel code should not implicitly allocate".
And C++ thinks this is none of your business. You can have no allocation by not providing an allocator - but you can't have no implicit allocation because the language will go out of its way to hide some of the obvious places allocation might happen rather than expose its warts.
For example the C++ language implicitly runs constructors, which may (or may not, it depends) cause allocation, whenever it sees fit to construct any object, even if that object is temporary and will be destroyed immediately afterwards.
Now of course you could say, "Oh I never use objects, those are awful" but then you ought to ask yourself what exactly the point was of having C++ at all. Or you could say, "We'll ban constructors from allocating" but C++ won't help you enforce that rule and the resulting code won't be idiomatic C++. Implied allocation is just normal in C++.
Also note that although Rust's full standard library won't be available in the kernel, you do get Vectors, you get Rust's reference counted "smart pointer" types, Iterators, Option and Result and so on. In Rust such things don't have implied allocation, only explicit allocation which is fine.
Rust used today in the real world does have some places with implied allocation, a notable example is String concatenation. Rust won't let you write
let hw = "Hello, " + "world";
However, the String implementation in the Rust standard library, and currently in kernel Rust, will let you write:
let h = "Hello".to_owned(); // Now it's a String
let hw = h + ", world";
But, that's one of the anticipated objections from Linus, and you'd just remove AddAssign (and possibly Add) from the Strings to cure it by forcing programmers to actually spell out what they want to happen here instead of acting as though concatenating strings is free.
If C++ constructors don't call allocation functions (1), don't initialize members which call allocation functions (2) and don't use features like RTTI which I've mentioned in a previous post, then they do not allocate. It's not trivial to ensure the above and while it's not usual for regular C++ projects, it's certainly manageable because C++ is used in some kernels and has a track record of being used in projects with constraints much tighter that the Linux kernel itself.
(2) seems the most tricky, but in reality, in a context like the Linux kernel where the libraries would be written from scratch or wrap existing C code, this can be controlled well. If one is including a new language in the kernel, it should be comparably low effort for example to write a custom vector container which exactly fulfills their requirements.
In the end I realize that C (and I assume Rust) makes it easier to reason about allocations in simple pieces of code which don't call functions, but this is not a deal breaker for C++ either.
So I don't think that allocations is the answer to the question of "Why Rust over C++?", as you claimed when you said "The allocations thing in particular". It's perhaps part of the answer, together with other technical reasons and personal likes or dislikes. And I believe that the latter are an important part of that answer that few if any are bringing up in this thread.
You'd need some sort of standard library, even if not the normal userspace one. The kernel has a pretty sizable "lib" directory which includes all the string functions etc. you'd expect in normal userspace C. For the Rust-for-Linux work we're using the Rust standard library's libcore and liballoc, just a slightly forked version of liballoc with the goal of getting any changes upstream once we get some experience about what works well. (Along those lines, the kernel has a non-allocating printk() that works like userspace printf(), and we provide non-allocating Rust wrappers that mimic userspace println!(). C++ in the kernel would want a non-allocating iostream, I think.)
I think you could address this point by using a slightly forked version of a C++ standard library that marked all its allocating constructors and allocating conversion functions with the "explicit" keyword. (But you wouldn't be able to upstream this, ever.)
I'm not sure if there's any other case where you'd have implicit allocations that you wouldn't be able to avoid. You may be right that this is doable entirely with library changes provided you avoid RTTI and exceptions.
One specific weird case is the copy done when capturing things into a lambda: https://godbolt.org/z/Y7fG5hjqq I expect the first allocation here because I'm explicitly calling the constructor, and that's fine, but the second one surprises me a bit, because the copy constructor is marked "explicit". Maybe people who work with C++ more regularly than me find this less surprising.
But that goes back to the problem 'tialaramex raised, about how "How many allocations does this do?" is a quiz question. Distinguishing zero from at-least-one is important, and you can do that by avoiding any heap-allocated-container types entirely, but so is distinguishing one from more-than-one.
Using Rust's stdlib isn't out of the question for the kernel, at least not the most fundamental bits of the stdlib. Rust offers a well-defined subset of its stdlib, called libcore, that is suitable for embedded work: no allocations, no OS-provided abstractions, etc. So you don't get HashMap or Vec, but you do get Option, Iterator, str (but not String, the heap equivalent), etc.
Exception handling and implicit allocations are very easy to control in C++, especially in a project where one would not use the STL or any other typical user space libraries. The language has been used in safety-critical projects and low-level embedded work for decades, so it must support such use cases well.
This compiles with no warnings, and at no point in the function "main" do you see that you're causing a memory allocation and a copy by implicitly converting your static message to a std::string, passing it to the function, and deallocating it. If you code-review "main", it doesn't look like you're doing an allocation because you're passing a const char *; if you code-review "print", it doesn't look like you're doing an allocation because you're accepting a std::string, which has already been allocated. So you get an extra, unneeded allocation in the chasm between the two, and it's very hard to track down the performance problem (or, in the worst case, a hang - memory allocation can block on flushing buffers or swapping things out, so you'd better not run into this situation in code that could itself be involved in writing to disk!).
> This compiles with no warnings, and at no point in the function "main" do you see that you're causing a memory allocation and a copy by implicitly converting your static message to a std::string, passing it to the function, and deallocating it
Such a warning would be a bug in this specific case as "Hello World!" is under the SSO size though - there is no call to malloc/new in this specific std::string creation.
I fail to see how that is relevant though: the kernel does not use the C standard library - it's not going to use the standard C++ library either except maybe the freestanding, compile-time-only bits such as type_traits, concepts, ...
In which case it is trivial to have a string type akin to rust's:
Here, the correct answer would be std::string_view anyways (or a custom c_string_view).
> If you code-review "main", it doesn't look like you're doing an allocation because you're passing a const char *; if you code-review "print", it doesn't look like you're doing an allocation because you're accepting a std::string, which has already been allocated.
My experience really disagrees with this part, every time I've been involved in code reviews, it was common knowledge that a function taking a const& or && can cause an allocation at the call site. It's barely a beginner mistake to not take it into account.
Amusingly if you create a Vec or String that operation is guaranteed not to allocate :D
The new() functions for these types are const (ie they can be evaluated at compile time) and produce an "empty" placeholder.
Because Rust's Strings don't have NUL termination, the empty string doesn't need any backing storage, likewise for an empty vector.
When (and more importantly in many programs if) you actually add something to the string or vector that may prompt an allocation, but you called an explicit method to do that, so it oughtn't to come as any surprise.
https://github.com/rust-lang/rust/pull/84266 We have been excising this with a new opt-out mechanism so there are no allocations behind your back but the same abstractions are notheless used as the rest of the Rust ecosystem.
I'm not surprised really. The C language is pretty old, and pretty much none of the features in C++ make writing correct kernel code easier. That's not a mark against C++, it's just landed on a different niche: a high perf userspace application language for expert software engineers. Other high profile languages like Golang and Swift have a similar 'application engineer' focus that offer no meaningful improvements for kernel authors.
So it's not surprising to me that Torvalds is critical of languages that don't cater to his use case. Nor why he's less vocal about Rust. The allure of Rust is inbuilt support for multithreading, protection against data races, and an efficient memory management system.
Fine, but it is completely false that C++ offers nothing useful for kernel coding. You can see, for example, SerenityOS kernel making good use of modern features.
Any feature useful for coding libraries is equally useful for kernel coding, because the overwhelming majority of kernel code is identical to what goes in libraries, and much of it is, explicitly, library code.
I don't think it's merely the Rust Evangelism Strike Force; Linus is notoriously critical of this kind of hype. In fact, his most vocal hatred of a programming language has been targeted at C++. Those criticisms almost surely have some amount of "lived experience" associated with them - he's not the kind of person to just buy into the hype of a programming language and risk the entire kernel over it.
Remember, this is the guy who used tarballs as a version control system up until Bitkeeper gave him free (as in beer) licenses to their DVCS. He didn't hate all version control, as much as he thought that was the case - he hated badly implemented version control, which was the standard at the time with things like CVS. That's why he ultimately wound up writing Git once the Bitkeeper deal became untenable.
In this set of old e-mails from a decade and change ago[0] he specifically calls out STL/Boost being unstable and difficult to work with, C++ providing certain abstractions that have undocumented performance problems, exception handling, and hidden allocations.
Of that list, I can think of several things Rust specifically solves relative to C++.
- Rust traits make reasoning about generic code way easier than C++ templates. In C++, you just write your template, and then the template is resolved at the site of instantiation. If it's not a valid type to put in there, then the compiler tells you that you used the template wrong, but it's still your responsibility to figure out why. In Rust, however, generic types have to be constrained with traits; which list out exactly what a generic type requires. So if you misuse a generic type, the compiler can just say 'well, you need Add on parameter T here to call this', and the error will always be on the type you used, rather than some dependent generic type.
- One of Rust's core goals is "zero-cost abstractions". It does this very well, but it also has some non-zero-cost abstractions. These I'd categorize as "transparent-and-predictable-cost abstractions". For example: Rust is known for it's borrow-checker, which provides compile-time verification of memory safety. However, these might not be the most flexible memory-management solution; so it also provides shared-ownership through reference counting. This does have an associated cost, but it's easy to reason about and clearly stated. They even provided both atomic and non-atomic reference counting options because the latter has less cost associated with it.
- C++ uses an exception handling model built around stack unwinding, which requires that all code maintain a valid stack that the C++ runtime can unwind at all times. This can pose problems for kernels which almost certainly will be doing weird things with the stack that the compiler won't entirely understand. While Rust has exceptions (called panics) and supports C++ style unwinding, it also allows you to change the exception handling model to just terminating the program instead, which matches better with how the Linux kernel handles exceptions.
- Rust also heavily discourages the use of exceptions relative to C++. Panicking is mostly reserved for error conditions that are actually exceptional (i.e. unexpected), while error conditions that are expected can be signaled using return types like Option and Result. There's even special language syntax (called Try, or the ? operator) which will automatically return if a value signals an error.
The only thing I can't say Rust solves is hidden allocations, but it seems like the plan Linus has picked is to tame that beast with a Linux-specific core library, with as much as possible mapped to Linux's existing C abstractions. This probably will work out as this is the intended way to extend existing native code with Rust libraries. (It's how Mozilla integrated things like Servo into Firefox.)
I definitely heard that expression a lot in regards to C++. That being said I would argue that Rust pushes the concept further thanks to its more powerful and expressive (if also more constraining) type system.
I don't believe that the differences which you mentioned between those languages are relevant for kernel development.
* Point one is a convenience/ease of use topic. Absent C++20 concepts or alternatively static asserts and enable_if, error messages will be harder to parse by the programmer. The one place where this may be relevant is for custom containers, but a kernel list for example wouldn't have to implement iterators and certainly doesn't need to follow the STL design which can cause hard to understand errors.
* I don't understand what you mean when you say that Rust "solves" zero-cost abstractions, but C++ has had this goal for a very long time.
* Exceptions would be disabled when doing kernel development. That being said, having exceptions in application development can be very convenient. C++ supports both, but std::optional and the like are not as elegant as in Rust.
> Linus (I guess everyone addresses him with his first name), has been very critical, even dismissive of systems programming languages other than C. But with Rust he has a more, I guess we could say passive approach. This might be a good indicator that Rust is doing something very respectable.
Or that Linus is going through something very respectable.
He still tells people off when they're being stupid. One instance of this happened a few weeks ago when an antivaxer said something on the mailing list.
Trying to figure out how much technical merit vs. guerilla marketing hides behind a project's decision to use Rust is probably an impossible task. With Rust the two are intertwined, the language and the community have a certain militant attitude associated with them.
Looking at the complexity of Rust versus C and the very limited benefits one would get out of adding a few drops of memory safety into a sea of C, one has to conclude that technical merit alone does not explain this decision.
In any case, I don't think Linus will write another language rant after he had an epiphany about his "unprofessional" attitude. And if there's enough corporate support combined with some grassroots support, he's probably figured out that he'd lose that battle even if he were nice about it.
> the language and the community have a certain militant attitude associated with them.
I'm not even sure what this means. What do you consider a militant attitude in this context? From what I've seen, most Rust evangelists (at least here) try to explain its benefits while acknowledging cases where it may not provide something better and is instead equivalent but different or slightly worse (it's not like any language can be better in every respect).
> the very limited benefits one would get out of adding a few drops of memory safety into a sea of C, one has to conclude that technical merit alone does not explain this decision.
I don't think that's a valid conclusion, but even if it is, you haven't explained it well enough. You haven't addressed how much more complex Rust is than C no how limited the benefits would be. You've provided a tiny bit more information than "it's not worth it", which I think is far from sufficient to support that position in a discussion that can have as much depth as is easily achieved here.
> In any case, I don't think Linus will write another language rant after he had an epiphany about his "unprofessional" attitude.
It's not like he's stopped calling being stupid for acting stupid or refrained from criticizing technologies he thinks deserves it. There are recent cases (mentioned in this discussion) that outline recent cases where he's done so. I'm not sure why you would assume he's all of a sudden deciding to keep his mouth shut about what would be valid concerns to bring forth if he had them, nothing to do with interpersonal communication problems he's had in the past.
You've said it yourself - evangelism, the spreading of the Rust gospel through the act of public preaching. :-) Most technologies don't have this, thank goodness. With Rust, I always have the feeling that I'm being "sold" something, even if I may have already bought it... On the positive side, it seems to be working and Rust does have technical merit, but I do hope that the community will mellow out with time and abandon this approach of trying to prove something.
Regarding technical merit, I like to think of Rust as a C++ with switches toggled in different ways and a borrow checker on top. It's complex to think in and to write in compared to C, but it's at the end of the day much easier to write correct code in. I won't go into all the details since that would take a long time, but I hope the kernel team will write about their difficulties with Rust once they encounter them, since that should be a very fun read.
Regarding Linus, I wanted to confirm to OP that their suspicions are not unfounded, because the other answers seem a bit like gaslighting to me, especially the one claiming it's a "weird question". There is definitely more to this than technical merit, but it doesn't have to be anything ill-natured from the Rust community. My theory is that Linus' spirit has been probably broken by the corporations controlling Linux development through their many commits, his getting older and grassroots support for Rust.
I wouldn't trust Big tech as far as I can throw one of their server farms. That said, I doubt that Linus's spirit is broken and he's simply surrendering to them. Assuming you aren't just ~~trolling~~ tin foil hat farming, I'll answer what I think:
I remember Linus said some years ago (probably ~2019 or 2018) that Rust or something like it in the Linux kernel is inevitable. His reasoning was that the Linux kernel is unlikely to remain maintainable in the long run. Due to the C code in the kernel being very non-standard and due to there being a lot of code in the kernel that got written the way it was for a specific reason and most of it being just unwritten implicit knowledge of the old hands. This makes it very hard to onboard new maintainers and PR's often require significant fixes and significant internal expertise to recognize the need for those fixes and be able to make them. As the old hands grow older, they may not be replaceable.
Furthermore, new bugs are now being discovered in the kernel faster than they can be fixed [1].
If I were to guess, Linus is just hoping that Rust or some other modern safe systems language suitable for kernel development (of which Rust is by far the most mature) could help sustain the kernel's maintainability.
As for why Big tech wants Rust in the kernel. I guess that's because they want total world domination. Which will probably be hard to achieve if any idiot can easily zero-day the kernel their infra all depends on and steal all their secret big data which is "the oil of the age of AI" or rootkit a phone making it harder to surveil it's ~~owner~~ addict. ;-)
> Regarding technical merit ...
Rust is very fundamentally different from "C++ with switches toggled in different ways and a borrow checker on top". I could call Java "Cobol with switches toggled in different ways and inheritance on top" but that wouldn't make them quite that similar as I've made them sound. If you think that Rust isn't fundamentally something drastically different from C++, you either don't have a clue about C++ or don't have a clue about Rust. Rust is holistically designed to be safe, which is the only possible way to actually make a safe language. C++, however modern and with whatever programming best practices and tools, is still fundamentally completely unsafe. It helps reduce security vulnerabilities from very very very unacceptably high to merely very very unacceptably high but in this case better is nowhere near good enough. Rust actually eliminates entire classes of security vulnerabilities (with some manageable caveats).
Another difference is that by the time you've addressed all the reasons typical C++ code is unacceptable in a kernel, you are back at simply C. But you can have high level abstractions and safety and, etc. in Rust inside the Linux kernel. For just one difference, Rust's zero-cost abstractions are actually, you know, zero cost. Unlike C++'s supposedly zero-cost abstractions which have unacceptable (in the Linux kernel) overhead. (For example a move in C++ is potentially very expensive, but at worst it is a memcpy in Rust.) But there are plenty of other reasons mentioned in this thread.
I wonder why Linus would think that though, because when one considers how big the kernel already is, and that in e.g. 10 years the vast majority will still be written in C, which will be just as unmaintainable as it is today if not more and that those old hands will be even older... the inescapable conclusion is that the Linux kernel will become unmaintainable period.
I'm skeptical that introducing a new language, whatever that may be, would help. In the short term it will make things harder, since maintainers have to juggle two languages and in the long term, as long as the new language does not displace the old one the situation would not improve.
It seems like this now has enough momentum and strong industry support to actually happen.
With the initial announcement I mentioned that Linux will need quite a customized Rust experience, and was hopeful that this effort could improve the situation for everyone.
It's great to see this actually happen, with changes already flowing back into core and alloc.
> With the initial announcement I mentioned that Linux will need quite a customized Rust experience, and was hopeful that this effort could improve the situation for everyone.
Neat. Is anything needed to disable SIMD in rust libcore? Maybe with this style of feature flag, core can maybe finally start using SIMD properly (conditionally)?
Working on a proposal to shore up opt-out feature so the stdlib can use them without fear, rather than the stop-gap ad-hoc CFG tokens the PR used temporarily.
Are there any other devs who think this might be what enables them to contribute to the kernel? I've always felt like there's this enormous mountain of knowledge I'd have to know to implement the smallest feature in the kernel:
- C pitfalls, probably documented all over the place.
- C idioms. These are hard to locate for most languages; I wouldn't know where to look in case of C.
- Kernel-specific C idioms. This is probably easy to locate and well documented, but I'd still expect it to be several thousand words.
- Code formatting tools. Are there any canonical formatting tools for C, or is it fragmented like for most older languages?
- How to test C code, which IIRC requires jumping through quite a few hoops.
- How to deal with text, which is a nightmare outside of ASCII in most languages.
1. Yea, a road that was used for target practice during a large-scale NATO training exercise is the same as a road that could do with a bit of maintenance with painted potholes to warn people. After all, they both have potholes.
2. You are cherry picking. He also said "These are hard to locate for most languages; I wouldn't know where to look in case of C." Rust has some advantages from for example Clippy and from a very newcomer-welcoming and helpful community.
3. Rust in kernel would indeed have kernel-specific idioms. But it'll probably be very similar to other embedded Rust and relatively easy for a newcomer to pick up. (Ee'll have to wait and see, but there is good reason to expect this.) C in kernel is known to be very hard for other C developers to pick up.
4. Thanks for the info.
5. But some things are special about Rust that makes testing easy.
6. The kernel doesn't deal much with text and even when it does, it generally doesn't need to deal with encodings (let alone modern encodings). That said, text is really a nightmare in C. Even just ASCII is needlessly complex due to null-terination, badly designed functions and the regular C UB pitfalls. At least you did acknowledge it may indeed be a problem for C compared to Rust.
I think the commenter you replied to meant that they know Rust a lot better than they do C, which makes it easier, particularly since C has such a long history.
We have been in contact with a set of companies and academia members
that would like to use Rust as a second language in the kernel. Some
of them have already started to evaluate Rust for their needs using
the infrastructure we have already in place.
In particular, we have got a few statements from major companies.
In no particular order:
Microsoft's Linux Systems Group is interested in contributing to
getting Rust into Linux kernel. Hopefully we will be able to submit
select Hyper-V drivers written in Rust in the coming months.
Arm recognises the Rust value proposition and is actively working
with the Rust community to improve Rust for Arm based systems.
A good example is Arm’s RFC contribution to the Rust language which
made Linux on 64-bit Arm systems a Tier-1 Rust supported platform.
Rustaceans at Arm are excited about the Rust for Linux initiative
and look forward to assisting in this effort.
Google supports and contributes directly to the Rust for Linux
project. Our Android team is evaluating a new Binder implementation
and considering other drivers where Rust could be adopted.
In addition, IBM contributed the Rust kernel support for PowerPC
which was already included in the RFC.
Funnily similar wording to an old slashdot troll message about rewriting the Linux kernel in Visual Basic:
“I hope to see a switch from C to VB very soon. I've already spoken with various luminaries in the C coding world and most are eager to begin to transition. Having just gotten off the phone with Mr. Alan Cox, I can say that he is quite thrilled with the speed increases that will occur when the Linux kernel is completely rewritten in Visual
Basic. Richard Stallman plans to support this, and hopes that the great Swede himself, Linux Torvaldis, won't object to renaming Linux to VB/Linux. Although not a C coder himself, I'm told that Slashdot's very own Admiral Taco will support this on his web site. Finally,
Dennis Ritchie is excited about the switch!”
> Microsoft's Linux Systems Group is interested in contributing to getting Rust into Linux kernel. Hopefully we will be able to submit select Hyper-V drivers written in Rust in the coming months.
At very least the Azure Sphere OS team should finally adopt Rust, instead of doing security marketing with the 7 layers of whatever, and then shipping a C only SDK.
It is like they are absent of the company security efforts, while trying to sell us a story of how much they care about security.
Not even C++ is supported on the Azure Sphere OS SDK.
Ironically they usally tend to evade the question when asked about it, to the point they ended up writing a very long blog post trying to justify C, and it had nothing to do with space requirements.
Rather industry expectations and other marketing speak.
So they are not using RUST because it is bloated but for other will be ok to use it. Linux kernel is already quiet bloated so i don't see what rust can bring.
BSDs start looking more and more interesting.
It is better when others have skin in the game and you are protected from risks. The fool doesn't learn from his own mistakes, the smart one learns from his own mistakes, the wise learns from the mistakes made by others. Very wise Azure people are. :)
IMHO, Ada missed its chance to shine when it didn't have any satisfactory FLOSS build chain right at the time that paying for a compiler became a thing of the past.
I think you may be right, but it's disheartening that this essentially boils down to a failure of hype.
Ada may be a good language, and GNAT may be a fine Free and Open Source Ada compiler, but Ada isn't a new programming language. Software developers only chase shiny new things. Ada is held back by its beginnings as a niche language with expensive compilers, it doesn't matter that this hasn't been the case for decades now.
Don't undersell that programming is tribal, and any language that deviates too far from the C/C++ syntax (from the ALGOL mold, that is) will constantly be fighting against what people consider the "norm" and those preconceptions. The closer the language is targeted at something that C and C++ are still commonly the main choice for, the harder it will be to get some people to consider an alternative if it using different syntax or structure in any way.
I think one big stumbling block people had in adopting ADA (along with there not being a good open source toolchain for a long while as noted here) is that it just looks different to people that primarily use C and similar languages, and that puts some people off.
That's sad, and I would rather it not be true, but it's become increasingly obvious to me from how people treat numerous newer languages I've seen pop up over the years.
There are plenty of very successful languages that look nothing like the C family. I think the syntax of ADA has a different issue, which is that it- to me at least- evokes languages that are considered old fashioned or out of date like BASIC/Fortran/COBOL.
I'd be more than willing to give Ada a chance, but when I looked at code samples I definitely had to fight an automatic response of thinking it was something out of date based on how it looks.
> There are plenty of very successful languages that look nothing like the C family.
It's not that there aren't successful languages and have a different look and feel, it's that there's pushback against them when they are targeted towards the niches C and C++ are already strong in. Obviously many languages exist and many have become popular, but certain niches are more resistant than others and sometimes it's for non-rational reasons.
> I'd be more than willing to give Ada a chance, but when I looked at code samples I definitely had to fight an automatic response of thinking it was something out of date based on how it looks.
That's part, but not all, of what I'm talking about. The style C follows is damn old itself, sometimes predating the languages you're thinking of as out of date.
> it's disheartening that this essentially boils down to a failure of hype.
Maybe also, but not only. In contrast to Rust, Ada docs have been awful, and as long as you didn't get a commercial toolchain you had to play with some gimped version (and the really free version didn't come out until the language was 20+ years old), the build system was as “good” as C, people complain that Rust is verbose, but Ada was still a notch over, the memory model of Ada is a mere reflect of Rust's one, and (maybe I'm being subjective here) Rust's stdlib and philosophy is leagues over Ada's (Option, Iterator, closures, Result, ...).
Ada had the potential to be a Rust 30 years before, but it was never the goal of its owners.
Quiet frankly, they solve different problems. Ada includes a GC as part of the runtime and can still run into memory safety issues if you don't use the GC.
Rust has opt-in memory safety issues but outside of those, does not require a GC.
Rust does not have Ada's constraint system. In that way, rust is easier (and arguably less safe) to program.
Ada didn't get the same love as Rust for the same reason D goes pretty unloved. The problem systems engineers needed solved was memory safety without a GC. Neither D nor Ada solve that problem.
Right, but (not counting extremely recent work that's directly inspired by Rust and therefore necessarily less mature than Rust's implementation) when you use Ada without a GC, you either have to avoid dynamic allocation or use Unchecked_Deallocation, which lives up to its name.
Static memory allocation is totally fine if the thing you're building is, like, a control system for a jumbo jet. You know exactly how many engines you have, how many wings you have, how many pilots you have, etc. Your autopilot has a fixed set of parameters: heading, speed, altitude, etc. You know exactly what tasks you're running: watch engine performance, watch the altimeter, watch cabin pressure, find the instrument landing system and line up with it, etc. If you have some new instrument to plug in, you get a new version of the flight control software, and you're certainly not plugging in new things mid-flight. Even if you aren't running all the tasks at once - e.g., you're usually not landing the plane - you want to guarantee you could turn on any of those tasks. You never want the possibility of running out of memory when you try to land, so you want to allocate memory for landing the plane at design time that's always reserved for landing the plane, and nobody would call that a "memory leak."
A general-purpose OS like Linux is different. You can create as many processes as you want. You can mount as many filesystems as you want, of various formats, whenever you want. You can plug in devices while the system is running. You can add drivers to a running kernel and remove them too. You can configure network connections, turn swap on and off, etc. Inbound network connections can show up at any time. So it needs dynamic allocation.
Ada is fantastic for problems like a flight computer. But it's a different sort of problem. For a general-purpose OS, you are allocating and freeing things in response to user input, and you want to make sure you neither leak memory nor use memory after it's been freed.
Rust is very good at solving that specific problem without the need for a GC. That's what people mean when they say "Rust doesn't use a GC."
* Ada has pointer arithmetic, you can overflow buffers or read out of bounds.
* Dynamic allocation is straight unsafe and unchecked (unless you have a GC, in which case… you have a GC), you can use-after-free and double-free if you copy pointers beforehand.
* Accessing uninitialised variables is not forbidden.
* You can dereference null pointers (though it should cause an exception so I guess that's memory-safe).
> * Ada has pointer arithmetic, you can overflow buffers or read out of bounds.
Not true. Arrays have bounds and those are checked. However, you can do arithmetic with the type ptrdiff_t of the package Interfaces.C.Pointers. You can also do an Ada.Unchecked_Conversion on a pointer that you get from a C function. Obviously that's unsafe.
> * Dynamic allocation is straight unsafe and unchecked
If allocation on the heap fails, it will raise a Storage_Error, but you can catch it. Also, the language has a lot of restrictions on when you may copy and store a pointer.
> * Accessing uninitialised variables is not forbidden.
True, but there are pragmas like Normalize_Scalars (to initialize them to invalid values) or Initialize_Scalars.
> * You can dereference null pointers
True, but dereferencing will raise an exception. Also you can define non-null pointers like this:
type Foo_Ptr is not null access Foo;
or add a "not null" constraint to a nullable pointer type:
Ada deliberately places readability before writeability. It's interested in the maintainability of large mature programs, and isn't intended for rapid application development.
I don't have enough experience with Ada to comment on whether its verbose and keyword-heavy syntax succeeds in this goal.
Ancedotally, I agree with zerr's comment below. Its generally not the syntax/keywords that make code hard to read, or easy to maintain. If you're familiar with the language, you learned to parse those basically sub-consciously.
For me, its always the var/function names and comments that determine how readable a program is.
Besides, if the prose would have been preferable over symbols, we'd be writing e.g. "an unknown number of apples summed up with an unknown number of pears equals to one hundred entities" instead of "x + y = 100" in math classes :)
That's a caricature. We sometimes use words for summation, e.g. Python's sum and fsum functions. I do wonder though if people had to write like that before the development of modern mathematical syntax. Even the '=' symbol is relatively young. Wikipedia tells me it was invented in 1557.
SQL is another 'wordy' language. There are many, many things wrong with the SQL language, but I wouldn't say its wordiness is one of them.
Having Linux make the first move, then learn from their successes and mistakes to implement a better version in your own OS seems like a good move that profits everyone.
It's not like Microsoft isn't pushing Rust on their own platform. Every time a Windows driver does a use-after-free somebody swears at Microsoft for causing a bluescreen.
So why not formal verification or static analysis tools of an existing codebase (Frama-C)? Or dropping in C derived from a formally verified source (Idris / F-Star)? Or a language with good performance and a verified subset (Ada/Spark)?
I've never read anything to make me think that Rust is a safer alternative than any of these options; the guarantees of the language are just assurances of its advocates.
Its not like Microsoft does not work on that, they probably have a dozen different formally verified programming languages around at the moment. You already mentioned F*[0], but there is also Checked C[1], VCC[2], koka[3], dafny[4], Project Verona and probably some more that I am missing. The point being that they have clearly evaluated and have used F-Star for parts of the OS already.
As one of the developers of this patchset, here is my simple answer: try it. I'd love to see it.
I know Rust; I don't know Frama-C or Idris or Ada. What little I know of Ada indicates that it doesn't actually solve the problem because it doesn't have safe dynamic memory allocation. Ada advocates have told me I'm wrong and there's some new thing (inspired by Rust) that does this. Maybe! I'm not sure there's any benefit to the world in arguing it; someone skilled in Ada should just write something and show how it would actually work.
Write a kernel module with any of those techniques, send in a patch, and see what people say. Any effective means of making the kernel less buggy is great. It doesn't have to be Rust.
I claim that Rust is effective because, well, those patches are there, and frankly I haven't been doing any work for this project in almost a year and we're getting so many enthusiastic contributors. So, apparently, a lot of other people also know or are willing to learn Rust. Maybe it's actually better than the alternatives you mention; maybe it just has better documentation. I don't know. I'm not claiming Rust is perfect, just that it's empirically likely to solve the problem.
If you can get together a community of folks who can do any of these other approaches, that would be fantastic. Try it!
> If memory allocation fails, an exception is thrown and a retry with less memory requirements is a possible execution path.
So, since we're talking about the Linux kernel, that's a red flag straight away. Notice a whole section of this new LKML post is about their implementation of Rust's alloc crate? Linus doesn't want useless exceptions here and has previously objected on that basis.
Meanwhile... Over in the C++ world they have this same behaviour. For them these exception handlers are a fiction, in reality popular standard libraries (never mind everything else you're using) will blow up in low memory and it's game over. Maybe Ada does a lot better. Maybe.
You seem to be arguing as if this effort should have gone to Ada instead. So go ahead and do it. There's a group of people who are trying to get kernel support for Rust. This is not a statement about Rust being superior to Ada or Ada's (in)suitability for the same job. It's only a statement that this group of people likes rust for it's safety features. I'm sure they are all aware that there are other, even safer languages, but these people like Rust.
We can lead academic discussions about which language is safer/better/leaner/whatever-er, but the fact is that Rust has a larger and growing mindshare so this effort is unsurprising. For Ada, the solution isn't to try and prove to anonymous forum goers it would've been better choice if only people recognized the fact. The solution is to mobilize the Ada community to put the same amount of effort into the publicity and marketing (yes, the bad M word) of their ecosystem.
Ada clearly lives on in many safety critical systems, but judging by the admittedly biased topic frequency on HN and other technical sites, it has a tiny community who mostly keep to themselves. That's fine, but then don't be surprised when it's Hip Language X that gets the headlines and these sort of efforts.
That effort has been happening for production code in high integrity systems since Ada was released to the world in 1983.
My only point is how many in the Rust community advocate as if Rust would be the first language having features, that actually were already in Ada for the past decades.
As for Ada on Linux, naturally Linus would never go along it.
A C kernel guy liking a language with Pascal syntax, a community that worships a paper that hasn't been true in the Pascal world since early 80's, not even at gunpoint.
If you want to see a use case where Ada was chosen against Rust, check NVidia's project for self driving vehicles firmware.
I appreciate what you are saying, but it is again just repeating why Ada is better than Rust. I posit that's irrelevant to this discussion. Rust is a safety centered language that seems to have champions and a growing community around it. Can we not just be happy that finally a safety conscious language has a following? If you are convinced Ada is better, nobody is stopping you from being the champion and building the Ada ecosystem. Right now, Ada is for all intents and purposes invisible both in terms of libraries, mindshare and hype.
Ada is alive where it matters most, high integrity systems.
Plenty of safety conscious language have had followings since ALGOL, this is the point that many miss when praising Rust, while ignoring our computing history.
I think the concern is that the core argument that "I think Linux should adopt rust for safety" is a bit dishonest, given that there are a large number of well-vetted alternatives. "I think Linux should adopt rust because I like rust for social reasons" would probably be more correct. That Google staff seem to think rust is good for the Linux kernel but not, say, Zircon, would be surprising if those same people actually believed that any legal rust program is safe and inherently trustworthy. "Embrace, extend, extinguish" wasn't that long ago.
I don't use rust so I have no money in this game, but based on the two links you provided, Rust ranks the highest among the "safety minded" languages. It ranks the same level as Ruby, Swift, Matlab or R in the Jetbrains ranking. That's incredible for such a new language.
I mean, neither Ada nor Zircon are even on the lists, so in terms of mindshare, they are clearly blown out of the water by Rust (from the evidence you provide). To discount Rust based on usage when the alternatives being suggested don't even make the rankings is also a bit dishonest.
In fact, the argument "I think Linux should adopt rust for safety" makes perfect sense. Here's finally a safety conscious language that seems to be embraced by a larger community, and even the big tech players. Instead we get people suggesting arcane alternatives that may be technically better, but have a snowball's chance in hell to become commonplace any time soon.
Rust was considered for Zircon. The only reason it wasn't chosen was that at the time Rust wasn't yet sufficiently mature for embedded. And it really wasn't.
In the meantime, Google makes extensive use of Rust in new components of both Fuchsia and Android and even rewrites some legacy components where it makes sense.
Right. It is a lack of knowledge, and I know I've seen you argue that Ada has facilities for this before, and I don't know enough Ada to answer that one way or another.
I think you and 'grumblenum should team up and write a working kernel module in Ada/Spark and then we can have a fruitful discussion about it, instead of just theorizing about whether it could be done. I'm interested in seeing the problems solved, and I'm indifferent to technology. If you think Ada/Spark can solve them, that would be awesome!
(Editing to reply to a small claim in another subthread, "Linus would never go for it" - we believed the same thing when we started. We were originally thinking that we'd pitch this directly at Linux distros shipping out-of-tree modules that they didn't want to keep having to push security updates for. https://github.com/fishinabarrel/linux-kernel-module-rust was out-of-tree and we tried to keep compatibility with kernels 4.4 onwards. But it turned out Linus isn't opposed to it.)
Fascinatingly, the more you post, the less convinced I am that Ada could ever be suitable for this use case. I looked at all these links to see what's been done:
Muen: If I am reading the allocator in https://git.codelabs.ch/?p=muen.git;a=tree;f=tools/liballoc/... correctly, it simply has no deallocator function at all. It does have a "Clear" function, but there's no sign that it has any ability to statically check that calling it is safe. That is exactly what we're talking about.
Nvidia: No public source code, and it's not obvious what actually exists here, beyond a compiler to Nvidia hardware.
Genode: This project is mostly written in C++. There is a very tiny, experimental amount of Ada in it (< 400 lines of code). There's certainly nothing that speaks to the questions discussed in this thread (safe dynamic allocation/deallocation, security, etc.).
ApexAda: This is a marketing brochure. Also this marketing brochure talks about "significant deployments in satellite communications, avionics, weapons control systems, and a myriad of other mil/aero applications," all of which - as discussed elsewhere in these comment - are applications that neither need nor want dynamic allocation (and are generally not exposed to deeply-structured untrusted input, for that matter).
Green Hills: This looks interesting - there's a POSIX filesystem and a UDP socket implementation, both of which (probably) want dynamic memory allocation. That is, at last, an application that needs dynamic allocation. Is it actually written in Ada? I see that the RTOS itself supports development in C, C++, and Ada, and they claim to offer a BSD socket implementation, so they at least have C bindings. I don't see any claim about what language it's written in, and Green Hills seems to do a lot of C work (see e.g. https://www.ghs.com/products/micro_velosity.html). If it's in Ada, then I think this gets the closest to what would be needed for the Linux kernel - do you know how their filesystem and socket implementations handle dynamic allocation? (I'll also note that they lost their certification many years ago, so if the answer is that they have safe code because they spent a whole lot of time perfecting a single version of an OS, then that's a very different problem space from Linux, which is constantly changing.)
Wind River: This is a marketing page without much detail, but it looks like it's a hypervisor to run general-purpose OSes on top of, not a general-purpose OS itself, which means it probably does not need dynamic memory allocation, or that if it does, the safety of deallocation can be verified by hand.
---
No Ada developer has done what's being done in this patchset, and it's pretty insulting to the folks putting in the work to claim it's been done and Gish-gallop a bunch of random links and expect people to put in the time to figure out what exactly you're claiming already exists. Everyone knows it's possible to write code in Ada. Nobody is asking if Ada can be used for completely unrelated purposes. You may as well link to GORILLAS.BAS and ask why the Linux kernel isn't using QBasic, a memory-safe language if you use the subset without PEEK and POKE.
Write - or point to - a kernel driver for a UNIX-like operating system written in Ada, that safely handles dynamic allocation. That's the clear bright-line goal. If you think that's not the right goal, say why.
>I'm interested in seeing the problems solved, and I'm indifferent to technology.
I think the first problem is some kind of language specification. That it's not possible to know what behavior one should expect from rustc means that it isn't a solution. It's another problem. You should check out Ken Thompson's great speech on why this is an issue. https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_Ref...
Or do you think it's wrong to regard unauditable enhancements to free software paid for by Google with suspicion?
Ken Thompson's speech was made two years before the first draft specification of C was published, so I don't understand how it is connected to language specifications.
Here's a way to address the issue he brings up in that speech that does not involve language specifications: https://dwheeler.com/trusting-trust/
As one of the developers of this patchset, can you comment on how these contributions will be licensed? Can you also comment on how a third party can verify or audit the output of a compiler without a specification?
>just that it's empirically likely to solve the problem.
Can you comment on what empirical evidence you based this statement on?
> Can you also comment on how a third party can verify or audit the output of a compiler without a specification?
I imagine it's pretty similar to how you verify or audit the output of a compiler with a specification. If you can show me an example of the sort of verification or audit you envision, I can perhaps give you a more helpful comment.
We do not make any claims about verified or audited compiler outputs or compliance with specifications, and to the best of my knowledge, the Linux kernel does not currently verify or audit any compiler outputs, so I'm not sure I follow why I would be particularly qualified to comment on it, but I could maybe point you at some interesting research.
> Can you comment on what empirical evidence you based this statement on?
Yes, I did comment on this, in the paragraph you quoted. If you'd like me to expand on something can you clarify?
>I imagine it's pretty similar to how you verify or audit the output of a compiler with a specification.
I must be missing the humor here. You can test against a specification to verify implementation. Testing an implementation against itself is meaningless. I would think that this is obvious. Also I don't typically use the word "empirical" to indicate purely qualitative assessments or intuition. You also provided a dead link.
There's no attempt at humor here - I'm just trying to get you to explain what, precisely, you mean by "test against a specification to verify implementation" so that we're in agreement on terms.
When I hear "test against a specification" and "verify implementation," I think about things like CompCert. But the Linux kernel does not use CompCert, it uses GCC or Clang. To the best of my knowledge, neither is tested against a specification and neither has a verified implementation.
Actually, when we first started discussing mainlining this, we had a bit of discussion about whether it was safe to allow rustc-compiled Rust code to link against GCC-compled C code. The specific worry was potential compiler bugs, of which the kernel has historically run into many. We eventually decided the risk was low, but the kernel has gone through periods where it insisted you use the exact same compiler for the kernel and for modules because of bugs. Those bugs have gone away through the normal human process of writing better code, not through formal verification. And when they existed, the kernel's response was not to reject those compilers, it was to work around the bugs.
Also, CompCert does not test against the C standard, which is written in English prose. It tests against the CompCert developers' attempt to re-express the C standard in terms of machine-checkable theorems. The C standard is confusing even to human readers, so it would be misleading to say that CompCert has verified their compiler against the standard as published by the standards committee. (Hence my claim that they're similar; you can verify a compiler of any language provided you write your own machine-checkable statements about what you expect it to do, and you still need other humans to read those theorems and convince themselves that they're the right ones.)
For those reasons, I'm trying to understand what your question is in a precise manner. Are you asking about things like CompCert? Because if so, I don't follow the relevance to our work, though again, I'm happy to point you at interesting research.
Without being an expert in either language, given what Ada is used for, I'm sure it offers at least all of the safety Rust offers and probably more.
But Rust has a lot more going for it than safety. Maybe it shouldn't matter for kernel code, but the focus on developer "ergonomics," high-level functional constructs and pretty flexible pattern matching, supporting both ML-style and Algol-style syntax, very strong language-level tooling for building, testing, packaging, and generating documentation, plus the fact that the language itself is so well documented with great, accessible tutorials.
There's a reason it keeps winning Stack Overflow's most loved language award every year. It's easy to learn for people coming from almost any kind of programming background, supportive of newbs, and equally well-suited for application-level and system-level development, so it ends up with pretty widespread appeal.
So again, I don't know that any of this should matter for selecting what languages you're going to support in the kernel, but as a purely practical matter, it's a lot easier to ask contributors to use a language they already know and use elsewhere than it is to ask them to learn something new that they don't use anywhere else, which is what Ada and Idris and what not would be for most developers.
Rust has far more enthusiasm from potential contributors than any of those options. Onboarding something is meaningless if it's not going to be used and maintained.
FWIW I would trust anything language-level over applying static analysis tools to C (for an evolving codebase like Linux, having the safety be a first-class part of the codebase is a must) or extracting C from another language (the semantics of C are surprisingly weak, such an extraction can only ever be as good as your model of C which will probably be buggy).
> I've never read anything to make me think that Rust is a safer alternative than any of these options
What Rust focuses on is providing a scalable approach to a limited kind of formal verification - focusing purely on type- and memory-safety, not conformance to arbitrary specifications. The latter is of course being worked on, but type safety and memory safety are table stakes.
I can't think of a case of a system like that being revived after 10 or more years. My best guess is that the mismatch between the environment the language was written in and the modern world simply becomes too great for anyone to care after a while.
For instance, I have to deal with JSON. There's no reason older languages can't deal with JSON, but it'll often be inconvenient as the libraries try to deal with mismatches in the type system or whatever, and it just can't ever be quite as slick as a more modern language written with some partial, vague combination of the fact that JSON exists in mind, together with the sorts of techniques used to implement JSON support in modern languages. Or sometimes it'll just be the dominant library for dealing with it just has an old design that baked bad assumptions in, and since it's the dominant library it's the one everything uses, so I have to deal with that everywhere.
And it's just the death of a thousand of those cuts. "But you can do JSON in my favored language!" Yeah, but it's just not quite as slick. "But we have an HTTP client in this language!" Yeah, but it's just not quite as slick. "But we have GPU support in my favorite language!" Yeah, but it's just not quite as slick. And you've ground all that "not quite as slick" deeply into your language over the decades.
So nobody has any one reason they can point at as to why Ada isn't going come back, or why E isn't going to come back, or why Cyclone isn't going to come back... but the reality is that the sum total of the issues is more than sufficient to prevent it from ever happening, and what's going to happen is a more modern language is going to get the support.
Advocates also try to "defeat in detail", in analogy to the military technique, arguments to the contrary, but the problem is that while that may satisfy the advocate, everyone else is still left looking at a metaphorical battlefield that has an exhausting number of "battles" over basic deficiencies, and being "defeated in detail" by an advocate just transfers to exhaustion with the underlying language anyhow.
It probably isn't a "the issue" with Ada/SPARK. It's a horde of small little things, the sum total of which is too much hassle for anyone to want to deal with.
Why past tense? Because it's dead as languages go. Defining "death" as "zero users" is not the definition most people use, because it's not useful; things are clearly dead before that point. Ada doesn't have a bright future. It isn't going to rise from the ashes and have everyone agree that yes, it was right all along, and everybody with 10 years of experience on their resume is suddenly in hot demand as the crowds give them ticker tape parades for having been right all along. That never happens. It may well klunk along for more decades... but it's dead.
What does happen with such languages is that someone comes along and creates some sort of new language taking the best of the old language and mixing it with more modern stuff. I personally think there's an Ada-like niche waiting to be exploited there, probably something that also takes some lessons from Haskell. (Haskell is still alive, but IMHO, headed toward this sort of dead right now, and really could use this sort of revival itself.) An example of this is Elixir... Erlang would probably be also something I'd be calling "dead" if it weren't for Elixir.
(Ironically, often the biggest partisans of the base language are the last to know about the revival language because every deviation from the base language is seen as a flaw from the One True Answer....)
Ada should be considered dead because it doesn't have an npm full of super[1] useful[2] libraries?
> That never happens. It may well klunk along for more decades... but it's dead
Ada is used in 2021 in the exact same places that Ada was used in 2011 (and 2001 before it), if not more. Ada was designed for a specific purpose, one that it still fulfills very well. People have been claiming Ada was dead for decades, yet still it miraculously lives on. Despite Rust, despite Haskell, despite an algal bloom of "formally verifiable subsets of C", and so on.
> What does happen with such languages is that someone comes along and...
Yet... This just... hasn't happened. A successor language more suitable for safety-critical development has just simply not appeared. The manufacturers using Ada —a subset of which can be seen at this[3] link— ostensibly don't seem convinced that any of Ada's many alternatives offer a more appropriate solution for developing safety-critical software.
> I personally think there's an Ada-like niche waiting to be exploited...
The Ada-like niche you describe is, believe it or not, currently occupied by Ada. Ridiculous as this may sound, Ada is actually pretty good at fulfilling its intended function. What needs to change? Ada is not designed to fill the role of Node.js, Java, or C#. It's designed for safety-critical embedded systems, which is where you're most likely to find it today.
> Ironically, often the biggest partisans of the base language are the last to know about the revival language because every deviation from the base language is seen as a flaw from the One True Answer...
Does this apply to your thinking too, or just the proponents of Ada, and other languages you don't like? Are the Rust users just blind to the benefits of Ada? Are you susceptible to the same cognitive blind-spot you accuse us of having?
> Ada should be considered dead because it doesn't have an npm full of super[1] useful[2] libraries?
Wait! There is Alire[1][2]! There is a list of "crates", you can easily include them into your project, and so forth. This is pretty much the same as https://crates.io or https://pkg.go.dev, which means people really should throw their misconceptions away and actually do some research before concluding that the language (Ada) is dead. A couple of quick searches quickly tells me that Ada is pretty much alive, with all of these "modern libraries with modern solutions", and full-on safety without performance loss.
This comment is intended for the person you replied to. :)
Yeah. "Not just quite as slick". What does it even mean? Ada/SPARK has everything you would find in a modern language... and more. People hold lots of misconceptions over the language, that is why I often bring it up and talk about it. That is why I often provide links to stuff people would easily find if they were to look stuff up about the language. As I said in my other comment, it even has a package manager: https://alire.ada.dev/.
Now, I know that there is a trendy view of "the more the better", and it does not have as many libraries there, but that has to do with the fact that it is not yet widespread among Ada users. Alire is really new.
What kind of scalable approach are you describing? Rust is not formally verifiable. I'm aware that there are projects attempting to solve this problem, however as I understand it you can't currently create verifiable production code.
> focusing purely on type- and memory-safety, not conformance to arbitrary specifications
What are types if not arbitrary specifications? Are you suggesting that formally verified software is not memory safe? What on earth does "scalable" even mean? A verified program cannot scale? If not, why not?
Types are not 'arbitrary' in any real sense. Type systems can be described as a sort of formal verification that leverages and is closely aligned with the syntax structure of actual code, thus type systems can preserve compositionality but don't generally support non-local reasoning, at least not in a very elegant way. This is exactly what can make them potentially more scalable than other sorts of FV, hence more useful for programming "in the large".
Types that can be defined by users are, necessarily, part of a language syntax, so that they are aligned with the syntax is just a tautology. Equating user defined types with formal verification is just nonsense. C has types. Python has types. Rust has types. That the program does what you think it should do does not logically follow the mere existence of types. I have no idea what "in the large" is supposed to mean. I feel like GPT-3 is arguing with me.
Maybe I can try to explain the parent comment's point a different way. When saying that rust works on types and not on arbitrary specifications, I think he is saying that rust is more limited than languages that support arbitrary specifications. However, by making this tradeoff it achieves a reasonable degree of safety without incurring a ton of overhead.
I believe the comment that types are aligned with the syntax is meant as a contrast to other languages which include specifications written in a format substantially different from or fully removed from the implementation. This can reduce the friction of using type-based verification when compared to formal verification capable of describing an arbitrarily complicated spec.
When discussing the scalability of types, I think he is saying that because types are coupled to the implementation, and don't support non-local reasoning, it is less likely that you will run into issues with type checking as you try to compose many small components into a larger program. In contrast, my impression is that with full formal verification, it can become extremely difficult to properly verify a large system.
Regarding your comparison to C and python, I think it's clear that the parent was comparing the specific type system and borrow checker that Rust provides vs. formal verification, not making a statement about the general concept of a type system. In particular, I don't think it's reasonable to assume he was saying the existence of types provides any sort of safety. Rather, it's clear he was saying the use of a powerful type system (such as that found in Rust or Haskell) to implement a limited specification of the program functionality can provide a degree of safety.
"In other words, the Curry–Howard correspondence is the observation that two families of seemingly unrelated formalisms—namely, the proof systems on one hand, and the models of computation on the other—are in fact the same kind of mathematical objects.
If one abstracts on the peculiarities of either formalism, the following generalization arises: a proof is a program, and the formula it proves is the type for the program."
I believe what zozbot234 is saying is that Rust focuses on a specific subset of formal verification, and on trying to make a language with that enforces that subset in a way that is still somewhat ergonomic.
I'm assuming "scalable" in this case is referring to the fact that fully verifying a piece of software is currently takes a large amount of effort. Rust's approach is more "scalable" in the sense that they decided to focus primarily on memory safety, and punted on other forms of verification in the hope that it would be more ergonomic/require less effort.
> I've never read anything to make me think that Rust is a safer alternative than any of these options; the guarantees of the language are just assurances of its advocates.
Note that some formal verification work has been done on the guarantees that the borrow checked provides (which led to finding and fixing an edge case bug), so there is a little more concrete evidence that just the "assurances of its advocates"
I don't think that the mere existence of a "working group" or grant funded research positions for verifying rustc or a subset thereof is the same as having a verified compiler. That's certainly not a strong argument to against using rustc for safety over one of the many verified compilers that actually exist. At least, I didn't see in the "rustbelt" list of papers anything which appeared to indicate that a verified compiler or language subset do in fact exist. Furthermore, that the proposed patches rely on the nightly compiler and a special variant of rust's core, which can only be enforced by special feature tags seems to suggest that Rust-Linux is not being written in this hypothetical, future compiler.
> At least, I didn't see in the "rustbelt" list of papers anything which appeared to indicate that a verified compiler or language subset do in fact exist.
You might want to look harder, because there is certainly a verified language subset (anything that can be written as LambdaRust, which covers a large and substantial swathe of the language).
If you're looking for a formally verified, optimizing compiler for any realistic language, particularly a low level language, you have pretty much one option: CompCert (I'm not sure where this large group of verified compilers you claim exists lives, but I'd love to see them). As someone who would love a verified Rust toolchain, I can't help but notice that very few safety-critical systems actually use CompCert in practice, which makes me a bit suspicious of claims that this is something Rust needs to be competitive in this arena.
Finally, I'll point out that verified compilers like CompCert only promise to translate correct C to correct machine code; they do not necessarily aid you in coming up with correct C in the first place. Miscompilations certainly happen, but not nearly as often as undefined behavior in the source program. So asking why people would use Rust when verified compilers exist for other languages is sort of missing the point.
I think Compcert isn’t widely used because of its license. GNAT is also GPL. Everyone lost their mind over Oracle charging a nominal sum for their compiler. I don’t think rust would have its corporate backing if it had similar licensing.
Also, there is only undefined behavior in rust. The compilers behavior can change at any time for any reason any time somebody with commit rights thinks that it should. You cannot begin to assert provenance about some thing that can change at any time.
> I think Compcert isn’t widely used because of its license.
I somewhat doubt this, but even if it is true, the fact remains that people are not actually using it for the vast majority of mission-critical software. This suggests that the elimination of miscompilation bugs, which is what you get from formal verification of a compiler, is not high on their list of priorities.
> Also, there is only undefined behavior in rust. The compilers behavior can change at any time for any reason any time somebody with commit rights thinks that it should.
This is certainly not true of Rust in any practical sense.
I am not saying Rust doesn't need a certified version of its compiler that people can legally rely on for stuff like this (as opposed to practically). I am saying that this objection has nothing to do with the existence of a formally verified compiler, which is also not in practice used to compile mission-critical C programs. I think you are confusing certification of a compiler for mission-critical software, with formal verification of the language's compiler, semantics (including type safety of the safe subset--a proof that would not go through for C as it explicitly unsafe), and standard library.
In a number of ways I'd argue that FV of an average CRUD app is a very hard problem, for two reasons:
- the requirements generally change quite frequently
- the requirements are generally a long way from being "formally verifiable"
Where I, personally, have found the most value so far using FV techniques is from modelling "tricky" things whose requirements are quite solid. A few examples from my work:
- A state machine for managing LoRa communication. The requirements in the LoRa spec are pretty good. I took those requirements and designed a state machine I could implement on a low-power device and modelled that SM in TLA+ to verify that there were no deadlocks/attempted transitions to invalid states.
- A state machine for managing the Bluetooth communication between an Android app, a Bluetooth-Iridium (satellite) bridge, and the server on the other side of the Iridium link. Similar to the LoRa case, I used TLA+ to ensure that there were no deadlocks in my proposed SM (and uhh made a number of corrections as I went). Ultimately, due to the design of the BT/Iridium gateway, the exercise resulted in the conclusion that it wasn't actually possible for this process to be 100% reliable but it was possible to detect and retry the edge cases.
- Modelling a somewhat awkward email/SMS invite flow for a mobile app. You could get an invitation over email or SMS, and your account transitioned through a few different states as both your email address and your phone number were verified. Modelling this helped ensure that your account couldn't get into a state where no forward progress was possible or into a state where you could use a half-completed account.
Just a note, I (personally) don’t consider TLA+ a FV technique — it is a really good technique that actually scales much better than formal verification programs like Agda or Coq.
It's formal verification of an abstract machine as opposed to running code. The problem is that formal methods as a field doesn't yet have a rich enough vocabulary to distinguish between its flavors. I use "code verification" and "design verification", which is a little more specific, but has its own issues.
Let us see what they have, using formal verification (Ada/SPARK).
- JWX is a library for handling JSON data and more. It is implemented in the SPARK programming language and has been proven to contain no runtime errors. As a result, JWX is particularly suited for processing untrusted information.
- The RecordFlux specification language is a domain-specific language to formally specify message formats of existing real-world binary protocols. Its syntax is inspired by Ada. A detailed description of the language elements can be found in the Language Reference.
- Formally verified, bounded-stack XML library
This is just Componolit.
Let us check the F* language:
- F* (pronounced F star) is a general-purpose functional programming language with effects aimed at program verification. It puts together the automation of an SMT-backed deductive verification tool with the expressive power of a proof assistant based on dependent types. After verification, F* programs can be extracted to efficient OCaml, F#, C, WASM, or ASM code. This enables verifying the functional correctness and security of realistic applications. The main ongoing use case of F* is building a verified, drop-in replacement for the whole HTTPS stack in Project Everest. This includes verified implementations of TLS 1.2 and 1.3 and of the underlying cryptographic primitives.
---
Given the above, what exactly do you mean that it does not scale?
Whoops, I completely forgot about libsparkcrypto (https://github.com/Componolit/libsparkcrypto) which is a "formally verified implementation of several widely used cryptographic algorithms using the SPARK 2014 programming language and toolset. For the complete library proofs of the absence of run-time errors like type range violations, division by zero and numerical overflows are available. Some of its subprograms include proofs of partial correctness.".
Looking at the examples you've provided, I'm pretty sure the Linux kernel's codebase dwarfs their combined codebases into insignificance. Since it is very hard to not have NP-hard asymptotic complexity when formally verifying a codebase as large and as demanding (in terms of performance, etc) as the Linux kernel, I would expect that if you tried to rewrite Linux in the languages you've mentioned and formally verify the rewrite, you will be able to keep the entire AWS cloud (globally) very busy for a very, very long time. On the other hand, if you were to do the rewrite in Rust and use some hypothetical formally verified Rust compiler to type check it (which is mathematically equivalent to such a formal verification in, say, F* limited to only verifying certain properties), it'll probably type check in minutes or even seconds on the laptop I'm currently writing this reply on.
For a less hypothetical example. RedoxOS has a codebase that dwarfs Sel4 (but it is still much smaller than Linux in turn), yet it type checks with much less compute cost than it takes to formally verify Sel4. Or heck, Redleaf actually formally verifies in a tiny fraction of the time it takes to formally verify Sel4, because the Rust type checker not only significantly reduces the amount of requirements that still needs to be formally verified but also reduces the complexity to verify those requirements to linear to the codebase size due to the restrictions that the Rust type system and module system puts on valid Rust programs (with a little bit of care when writing the code). Of course, until you have a certified Rust compiler there might still be miscompiles of Redleaf and until all the features of Rust used in the Linux kernel is formally specified, we cannot do the same with the Linux kernel as for Redleaf even if somehow all of Linux got rewritten in Rust. But in Rust at least it is conceivably possible given sufficient resources and sufficient time. I very much doubt it will ever be possible without help from a sufficiently expressive and practical type system.
Oh yea. Then there's another problem. Most (all?) of the examples you've mentioned have requirements that are well specified and pretty much set in stone. The requirements fro the Linux kernel... aren't quite as obliging.
It's security parameters are well-defined enough that you can use it to launch isolated VMs with... less secure OSes that are sandboxed from each other.
That's true, but as a microkernel, seL4 has a very limited scope. It's not in the same ballpark as, say, the Linux kernel. It's more like a hypervisor. [0]
From a quick google, seL4 has around 10,000 lines of C code, whereas the Linux kernel has around 30,000,000.
Also to note, ARM toyed with the idea of using a Linux distribution as alternative to MBed, but it was short love, it is already dead, it only lasted one release.
The kinks get straightened out by people working with it and trying to make things. Waiting won't accomplish that.
> Like it was mentioned in the RFC, the Rust support is still to be considered experimental. However, as noted back in April, support is good enough that kernel developers can start working on the Rust abstractions for subsystems and write drivers and other modules.
I also believe that this is a little bit hype driven but not quite. I've been talking with a Linux dev at a conference in Cluj and he said that people nowadays are not interested in Linux development as he was in high-school because there's so much more out there and way easier to enter (web, mobile, game dev etc.). A new language like Rust can reinvigorate the Linux ecosystem + the benefits of safety and other language features. Zig could have been an alternative but unfortunately it's still under development and quite far from a 1.0. But, in my opinion, once you fragment the ecosystem with Rust, adding a third language like Zig should not be a problem. So, yeah, the train still hasn't left the station.
Honestly, I think this particular argument is bogus.
From my point of view what counts is actual kernel development. If you are going to develop kernel because it uses cool new language but you would not if it was still C, you are probably not the right person.
Now, I think Rust has potential to be good kernel language, I am mostly concerned with toolchain, ability for people to read the code with comprehension, ensure correct code, etc.
> If you are going to develop kernel because it uses cool new language but you would not if it was still C, you are probably not the right person.
You are taking this the wrong way around. It's not about people contributing to $PROJECT because it uses $COOL_TECH, it's about people being turned off from contributing to $PROJECT because of $MEH_TECH.
If Linux was written entirely in assembly, we'd be having the same argument about the introduction of C.
As a potential alternative thought process I would add that kernel development sort of uses their own flavor of C that you also have to learn. While in case of a language with better abstraction powers, you can actually read a function and knowing only the language but not the project, could potentially contribute — as opposed to C, where I have to know this cryptic macro, and that by convention this and that struct has to be embedded x bytes into another struct so that it can be used with the linked list utilities made.
Also, simply by having god damn namespaces more readable names could be used at no downside.
> From my point of view what counts is actual kernel development. If you are going to develop kernel because it uses cool new language but you would not if it was still C, you are probably not the right person.
Why's that? C is a tedious language that makes you do so much manual bookkeeping, I don't think I'd ever write it again (if I needed C for some reason I'd write a program to generate it for me rather than doing it myself). I think any decent developer would be put off by C to some extent. Sure, the people who really want to make a kernel may put up with it, but an arbitrary enthusiasm barrier is not the way to attract good contributors.
> I think any decent developer would be put off by C to some extent.
Any decent developer would be put off by any language to some extent.
Personally, I'm more put off by Rust than C (they put me off for different reasons). Currently, the one puts me off the least is Zig, but it still has many details that I don't like.
On the other hand C is simple language, there are no discussions about how to do something in C between people who know C.
C is tedious but kernel programming isn't about spewing vast amount of code anyway. It is more important to be able to read it, understand it and ascertain it is correct. This is one of main reasons for why C++ is not there.
Except for, you know, all that Undefined Behavior stuff, which makes it practically impossible to write correct programs even for very skilled developers, or the lack of abstractions, which makes it hard to safely re-use advanced data structures.
"C is a simplistic language" would be more accurate.
Code is either clear and simple, or extremely obscure and safe, or kinda in between is complicated but not complicated enough such that its not safe.
"Simplicate and add lightness" aerospace mentality people like C.
At some point any project expected to do ever more complicated tasks, like a kernel, will be unable to use simple C and will have to take the hit of a more complicated language.
Simple is good when matched to simple tasks; my microwave oven or TV should ideally never data network with the external world and never malloc memory and never do string processing. C makes an amazing toaster or car automatic transmission controller.
C is simple only as a pseudocode language, i.e. to reason about the happy flow. As soon as you start adding corner cases and error handling, all your happy flow code now gets obscured by numerous ifs, switches, and ($deity forbid) goto statements. Then, add pointers masquerading as output parameters, pointers masquerading as arrays, and pointers masquerading as pointers and your language isn't so simple anymore.
>"I think any decent developer would be put off by C to some extent."
Call me indecent. I use C for firmware development running on small microcontrollers and consider it perfect for this particular task. Not put off by C for this task at all.
It is a little bit bogus, I did not say that I stand by it or that I think that this is the WAY. I'm not even a Linux developer or a kernel dev, though I've done embedded programming at the beginning of my career. Just that if we won't take risks, we will never know whether the approach of Rust or Zig or just plain-old C would prove to be THE approach to systems programming. Besides fragmentation, how else can you harm what we already have? I believe the ecosystems fix themselves, if you offer to the developer three choices (in our case C, Zig, Rust), we will see how things pan out and we will have a clearer picture. Java did not become the language of Fortune 500 companies out of sheer luck, people simply chose it more often than its counterparts because the feedback was positive. I expect the same in systems programming.
Now it is clearer that foo is responsible for getting rid of the allocation corresponding to text, which is on the heap. out can be modified and potentially reallocated. s is read only and it won't be deallocated by foo. x might be mutated in-place by foo, but it won't be deallocated.
Having a more "complex" way to represent in the typesystem something that C only bothers to call "a pointer" makes things better: safer, easier to skim, explore and understand an API, and allows the compiler to catch API misuses.
Being a smaller simpler language doesn't make it easier to read or write. Go is an even "simpler" language than C, but you quickly suffer from boilerplate-blindness when reviewing it. Being able to write good abstractions can make a huge difference. Having lifetimes, or the use of the semantically-correct integer type checked by the compiler reduces the cognitive load on the writer and reviewer.
Yep that is my experience. C++ sucks. Thinking in rust and getting your ideas out is much much much easier. C is portable assembly, I prefer a language that protects me from footguns. I have better things to think about.
It's even worse, C is “portable” PDP-11 assembly. It has been a few decades that a C program is only distantly related to the assembly the compiler will generate.
The git repo has over 150,000 commits, 3,000 contributors, and 9 years of release history. The last 6 years of which have been post 1.0.
Also, Mozilla has been writing large parts of firefox in Rust since ~2017. There's some interesting writeups on hacks.mozilla, including this one on reducing CVEs with Rust
Sizes of repos have nothing to do with it. By that measure we should all leave C and convert kernel to C++.
I do embedded development on ARM Cortex-M3 and while it is possible to do this in Rust, I have experienced a bunch of issues with the toolchain and I have decided to stay on C until toolchain improves.
Don't get me wrong, I like Rust. Just don't allow liking something get in the way of practicality.
Anyone feel like we get a "Linux Rust support" post every month lately?
Feels a bit like the "new more efficient battery technology" recurring post.
edit: to precise it's not that I'm not supportive of the effort, just that it would be nice to just have a post when it's actually merged. The title is a bit deceptive
I quite appreciate the reports. I normally don't follow the linux kernel development, but the possible support of Rust is one major step for the kernel which is hugely interesting.
The Linux kernel has become one of the most, if not the most important piece of IT infrastructure. Progress of any further developments of it as well as the security itself are prime concerns for all users. So far the development was solely limited to C, with all the tradeoffs. It is highly interesting to see whether a more modern language can be incorporated and the Rust effort looks very promising. Both from the language capabilities as well as the progress being made.
That's because the "merge window" is open once every ~2 months; this is the second time this series is being sent, with improvements based on feedback from v1.
Also, rust is only being considered for kernel modules, not kernel core. And even then, only kernel modules that don't need to build on architectures rust doesn't support. So if you don't need any of those particular modules you can just disable them.
In the long term, LLVM/Rust support for more architectures improves, and support for older architectures is dropped, this will become a larger fraction of kernel modules, but it will be a very long time until it is considered for something like ext5.
Linux can be compiled with LLVM (clang), and the current rust code strongly prefers that you do so. It's possible to do gcc + rustc, but it's less supported, and if you pass the wrong gcc flags I believe it just won't work.
> until just a couple of years ago it could not as it used a lot of GCC specific syntax
Note that the kernel still uses a lot of GCC specific syntax; it's just that LLVM/clang was laboriously made to understand all of that (with a few exceptions where the kernel was changed instead). And when they were almost done, the kernel started requiring another new GCC specific syntax, so they had to chase after that too.
It seems you can already write kernel modules in Ada: https://github.com/alkhimey/Ada_Kernel_Module_Framework. Obviously it comes with a boatload of restrictions [1], so no tasking, RAII, exception propagation, or dynamic allocations.
This might be a stupid question because I don't know much about Linux development.
Have these patches been accepted into the kernel? Or are they still just "propsals"?
They've been accepted into linux-next, which is a staging ground and used for testing, but that by no means guarantees that they will ever be accepted into the mainline kernel.
Not going to be merged into a release branch yet, this is more of a "status update, give it a try" posting. We'll certainly see a few more of those before we get a proper Rust-enabled Linux release.
Wow, I'm impressed that alloc crate modifications are already there ! Seeing this rate of progress (and the funding behind this work) makes me quite optimistic !
Beginning of the end of Linux I guess... Mixing multiple languages deeply in the core rarely works due to different requirements and limited interoperability. 5 years later we are going to read about how to migrate existing Rust code to C, like what's happening to Scala projects these days.
It seems to me like a Trojan horse from G and MS that pursue their own interests with their own OSes.
> like what's happening to Scala projects these days.
I don't know much about Rust so can't comment on that part. But agreed on the Scala part. Having seen Apache Kafka use Scala in the early days, when Scala was considered "hip", alongside Java code in the same codebase, I can relate to the mess it created and continues to be even today. At some point there was a plan to move away from Scala in Kafka, not sure where that reached, I stopped following that project since a few years now.
But another jvm language (kotlin) seems mixed with java just fine.(did it designed to do so?) Two way calls just work as is. Interfaces just maps to interface in each other. The only problem is probably the gradle config, but it is a one time thing.
> 5 years later we are going to read about how to migrate existing Rust code to C, like what's happening to Scala projects these days.
Just the opposite. We're going to see C deprecated for new code and efforts to move the existing C codebase to Rust. C is pretty much unsafe at any speed; Rust allows even junior front end developers to fearlessly contribute bare-metal code once they've learned the language.
Ah, but Rust will help reduce security problems! Even if the number of potential Linux contributors is cut to a fraction due to the difficulty in learning two languages with steep learning curves, better security makes it all worth it.
The Linux developer pool will increase. C programmers are mostly old hands these days, and they're a dying breed. Rust opens the possibility of kernel development to younger or less experienced programmers, many of whom only have had experience with modern languages and frameworks.
If you can't see how having inexperienced programmers writing code that is guaranteed memory-safe beats having world-class expert programmers writing dangerous code because even world-class expert programmers cannot wield C safely then why are you working in the field?
Is this a joke? Is the existing infrastructure built by countless geniuses in C not working at all? Do you think inexperienced programmers could build it given a "safer instrument"? How many junior devs understand intricacies of basic OS-level locking, not mentioning Paxos and other scary algorithms whose formal correctness can't be even proven in theory?
> Is this a joke? Is the existing infrastructure built by countless geniuses in C not working at all?
Given all the CVEs that arise due to buffer overflows and UAF, it's not working very well. Safe Rust makes those classes of problems Just Go Away.
> Do you think inexperienced programmers could build it given a "safer instrument"?
Inexperienced programmers can get started without the near vertical learning curve C presents. Just wrapping your head around pointers scares many programmers off, and even once you grok pointers, attempting to use them correctly is a minefield. This makes kernel hacking daunting for the average dev. Rust gives them an onramp.
> How many junior devs understand intricacies of basic OS-level locking, not mentioning Paxos and other scary algorithms whose formal correctness can't be even proven in theory?
Not very many... but junior devs don't need to work on that stuff. They can make more peripheral contributions -- fixes to device or file system drivers, etc. -- while they're building their skills, completely confident that they will not make a class of stupid but very common and dangerous memory errors. Not so with C! As I said, not even great programmers can be relied upon to write safe C code -- the model presented by C is just that abstruse.
Too bad that Ziglang is still too immature at this point. It seems to be more in spirit with C and UNIX - simple yet powerful, while avoiding most of C's footguns. Would have been interesting to see what Linus thinks of it.
Adding a 3rd language would be madness, Rust is already complex enough - so that window of opportunity will be closed.
Zig is a really nice language, but it doesn't go as far as Rust in the correctness department, which is what's motivating Rust adoption in Linux (other Rust aspects being merely nice-to-haves). It's not clear that Zig would bring sufficient gains to overcome the cost of adding a language, even if it was as mature as Rust.
I think the reverse is true, with Rust explore the new multi-language codebase and tooling (LLVM) for Linux, it would be easier for Zig to be another supported language for the kernel.
Every language added means less people are able to review patches, and it means a higher barrier to participation. Do you think there should be a limit to the number of supported languages or not? If yes, what would that number be?
With the codebase as big and complex as Linux, you don't have to know everything to be able to contribute. It will be different module being implemented in different language, and more language supported is more people can contribute to the codebase (people who has been put off by C).
Furthermore, for a language to be supported, it must have enough infrastructure (tooling, people, convention...) to support that language first, so people to review patches shouldn't be a problem.
Obviously, language should not be added willy-nilly, it must prove that it can solve some problem that the current set of languages can not solve efficiently, and base on that merit, it will choose to write/rewrite the modules that have the most of those problems first.
...
Also, why is this in big-endian order?
Let's just try to kill big-endian data, it's disgusting
and should just die already.
BE is practically dead anyway, we shouldn't add new cases. Networking
has legacy reasons from the bad old days when byte order wars were
still a thing, but those days are gone.
For values that fit in a machine word, there are adder and multiplier designs that make the difference irrelevant. For larger values, or with some other adder/multiplier designs with different trade-offs, LE is dramatically faster.
Specifically, the problem is with "carries". When you're adding (or subtracting, or multiplying, or dividing, I'll just discuss adding) two binary values you might have to carry a 1 to the next place.
If you've got a BE value and an adder stage smaller than that value (say, a 32-bit number and 1-bit adder stages) you have to carry a 1 many times to output the result. If you're receiving the value in BE order 1 bit at a time you can't start the computation until you have the LSBs of both values, since if they're both 1 they'll affect the second bit by their carry. So you're stuck waiting for the entire value to start the computation. Further, during the computation you have to wait for every 1-bit adder in sequence.
There are "fast" adder designs that don't have to wait for every bit, but can instead work on groups of multiple bits with a carry-out at the end of the group. So if you've got an 8-bit group size, you'd have at most 3 carry delays during the computation of the output. For BE, you'd have to wait for all 4 bytes to be received, then wait for all the carry delays. For LE, you can start the computation as soon as the first byte is received, saving some time.
The larger the adder group size the more die area is needed, the stronger the drive strength of the transistors needs to be (bigger fan-out), and the slower the maximum clock of the overall system. On the other hand the bigger the group size the fewer carry delays, so addition can take fewer cycles. Most CPUs and MCUs implement single-cycle addition of their word size. Some CPUs even implement single-cycle multiplication at their word size.
On pretty much anything Linux is running on, full words are loaded into registers before a multiply begins. Even for say, something like x86 where a multiply can have a memory argument and say that it straddles a cache line boundary so you could get a portion of the word, the system still splits it into load to (temporary) register, execute mul, and store to memory micro-ops.
Correct. There are a few cases where the operands don't fit into a single machine word, the most notable being many cryptographic operations. Particularly RSA and ECC, which involve multiple-precision arithmetic.
There are also non-Linux cases, mostly microcontrollers. EG the Arm Cortex M0 doesn't have a hardware multiplier, the M0+ does.
And then there's that one guy who got Linux running on an 8-bit AVR by emulating a 32-bit ARM and running it on that[1]. I'd consider this a silly edge case. Too fun not to mention though.
When you add numbers together you start at the little end. Imagine a bignum implementation with multi-word numbers. Now imagine you have a bunch of them in a file you want to add up. If the numbers are in little endian order you can do a streaming implementation that reads and adds at the same time. If they are in big endian order you need to read a whole number before it can be added to the accumulator.
Obviously this example is very contrived. This sort of thing was much more of a concern on 8-bit computers. But little endian still seems more natural to me.
With little endian, if you take a pointer to an integer of a large type (eg. uint64_t) where the value fits in a smaller type (eg. uint32_t), you will get same correct value accessing it as a uint64_t or uint32_t on a little endian system. This can make integer type conversion/casting slightly more efficient, and simplify code a bit.
One small argument for little endian is that if you have a pointer void* then in little endian format you can interpret it as a int8_t*, int16_t*, etc. (or char*, short*, etc. in old money) and get the same numerical value if the number is small enough that it fits into all the types you try. I don't think that has much practical use but it does have a nice feel about it.
I get it, but it also means values will appear to be correct, instead of obviously wrong, if the data is cast without concern for the value range. Then one day someone enters a value and exceeds that range and BOOM
>> Is there a technical reason why little endian is better besides the fact that it's more popular due to x86/ARM?
To me, big endian makes a lot more sense
Some of the other replies have minor technical reasons, but I've always preferred big endian for the readability. Having said that, I'm happy to part with the idea of big endian if it means an end to having 2 options to worry about. One thing that bothers me a lot about RISC-V is that the standard claims to allow either big or little endian implementations. Little has won and nothing new should support big endian IMHO. The benefits of either are largely irrelevant, but the existence of both is a problem. Or maybe it's that the existence of code that cares is the real problem ;-)
> Networking has legacy reasons from the bad old days when byte order wars were still a thing, but those days are gone.
Yeah, this isn't true. Low level hardware receiving data still likes to use shift registers:
* Zero shift reg
* clock in one byte, shift into shift register
* Clock in next byte, shift into shift register shifting the previous byte left one
* Repeat for as many bytes as you have
If you want this to work for a variable number of bytes, then you need most significant byte first, so that everything more significant is pre-zeroed. This is not theoretical - we did this for an FPGA network offload thing last year.
Processors can't agree on endinaness, but network protocols have. "Network byte order" is a standard thing that is almost completely universal across communication protocols.
This kind of misses the point of his comment though. There's now tons of ASICs and FPGAs in networking gear that relies on network byte order for optimal performance. No one's advocating for changing that.
What he is saying is that for pretty much everything else (read: typical CPUs) it's all little endian now.
I am out of my depth here, but I do not really understand why this matters. Could you elaborate?
Basically, I understand that shifts for incoming serial bits is very convenient. But as long as you specify the "word size" used in your local memory, I do not see the problem: just use 8 (for instance) shifts for your 8-bit word and then go to the next memory location. Why is this fantasy wrong?
The parent comment is generic and not about the patch submitter. "in good faith" is precisely the kind of phrase that you'll hear more often on lkml if the people in question (again, not the patch submitter) become more active.
It'd be very hard to make a statement like "Rust can do X that C can't" without there being a statement "You can do Y, then Z in C and that is like X" because clearly they are both Turing complete languages at the end of the day.
So I'll instead just give reasons why people might prefer to program in Rust:
- Memory management / safety (no GC, variable lifetimes are tracked by the compiler, no dangling, no null pointer ...)
- Ergonomics (built-in unit tests, feature-full std lib, unicode strings, modern dependency manager, more formalized macros ...)
- Functional-lite (you must explicitly discard unused enum variants, the "null" value is an enum variant ...)
Long story short: Rust makes it really easy to write sound software. I think you see that on crates.io where Rust packages are hosted. Lots of packages quickly reach maintenance mode and receive very few updates, but remain extremely useful. Once you've written (compiled) something in Rust, you feel much more confident that is is free of latent runtime bugs.
Without commenting on whether or not this is necessary or good for the Linux kernel, I'll say that there are probably many devs out there that feel like working other non GC languages feels like pulling teeth after having worked with Rust. And that might be part of the pressure.
Note that the kernel won't use cargo to build most (or all) of the rust code.
Though dealing with rust dependencies by hand is usually nicer than dealing with C dependencies by hand because the build process is more standardized.
Compile time guarantees that you didn’t access memory you didn’t really “own”.
Dangling pointers are not a thing.
No double free.
Forces you to think up front about what are the possible state something could be in, and so you won’t get runtime or weird UB.
lastly, to be more pedantic for everyone who is at Linus-levels, it’s like all the best static analysis and warnings/errors all rolled up natively into the language. Please don’t bombard me with “but I can just run clang’s X and get the same thing”. No you can’t, and if you did this forces you to run it regardless.
Well, no. You’re talking about using all the flags and tooling like i just mentioned, which 99.99% of people don’t use every time they run a build. Even then the best and strictest of flags + Werror still falls short.
Then you still have to run extensive fuzzing and testing to trigger different paths and whatnot.
Apart from all the safety benefits mentioned, a modern language with modern conveniences.
Type inference
Language support for tagged unions (which they call enums for some reason), which among other things results in error handling which is "out of channel" instead of magic return values.
Speaking out error handling - syntax for easily forwarding errors to the calling function.
Generics, and therefore
- Libraries of collections with a rich collection of methods implemented for you.
- Library support for lazy iteration
- ...
Method (postfix call) syntax allowing for readable chaining
Hygenic macros
Compiler supported test harnesses
Compiler supported linting (that is really good)
Compiler supported automatic formatting (that is really good)
> Language support for tagged unions (which they call enums for some reason)
Probably because the simplest case of Rust enums (without any data other than the tag itself) can act similar to a C enum, to the point that you can "#[repr(C)]" it and use as FFI for the equivalent C enum. One understated advantage of Rust for C programmers (which is on topic given that the Linux kernel is written mostly in C) is that its syntax and keywords have many similarities with C, starting with the use of braces for blocks and semicolons to end statements.
Ya, that was a throwaway comment that I probably should have left out.
> and use as FFI for the equivalent C enum.
Only if you're willing to accept undefined behavior if the foreign code passes you an unknown variant. For actual ffi work I've ended up using a macro that makes a "#[repr(C)] struct EnumName(u32)" with a bunch of constants for the values.
As far as I know that flag doesn't give you permission to create unknown variants or effect backend code generation that will assume there are only the defined variants. I could be wrong, but I don't see anything about this documented in the rfc: https://rust-lang.github.io/rfcs/2008-non-exhaustive.html
It would be correct, up to the size of the integer. As far as the abi is concerned a C enum is just an integer. As far as the abi is concerned a struct containing a single integer field, is also just an integer.
If there are any values that don't fit in a u32, I would need to use a different int type. C enums automatically use bigger int types if they don't fit in the smaller ones.
If I was on an architecture where int was smaller than 32 bits it would also be incorrect (except by coincidence that 32 bits was the smallest type that could hold all the values). I could "solve" that by using libc::c_int instead, but that only sort of solves the issue because if libc::c_int is 8 bits, and the enum has a value 257, it's just not correct anymore.
I suppose it's possible that some abi treats a struct containing an int differently from an int, I doubt it though. If that's the case, #[repr(transparent)] instead of #[repr(C)] would be slightly more correct.
> As far as the abi is concerned a C enum is just an integer
In some ABIs this is not true, a C enum is a char if the constants fit. Your code that assumes enums are stored and passed around as 32-bit integers is not entirely portable.
Ultimately, at least at the time, there was no other option that had better compatibility. Now I suppose you could use MaybeUninit<#[repr(C)] enum> or something. Realistically I don't have much confidence in rustc properly matching enum sizes with C implementations that don't follow the typical behaviour here, so I doubt that would be worth it.
I guess the linux kernel is generic enough that it might end up on some such platforms, I guess in that context I'd just bite the bullet and accept undefined behavior if passed an unexpected enum variant, it's not like that should happen in a coherent system like the linux kernel (vs my use case which was wrapping some library that definitely might chose to add enum variants).
You can do all sorts of C things in Rust, and they're merely not idiomatic.
For example your Rust function that always returns 42 should probably be:
const fn fortytwo() -> i32 {
42
}
But C programmers might try:
const fn fortytwo() -> i32 {
return 42;
}
And that works fine, the compiler sees that you wanted to return 42 here, and, although the value at the tail of the function is an empty tuple and thus incompatible with your function signature you never reach that tail, so no problem.
2. Ergonomic use of functional idioms like sum types (Rust's enums)
3. Memory safety without GC overhead - Windows, Linux, curl, over and over again 40% of vulnerabilities in larger C projects are found to be preventable if there was a memory safe language
The main benefit is that in Rust, use-after-free, data races, and bounds overflows are statically checked with zero runtime cost. It eliminates entire classes of common severe memory safety bugs at compile time.
I came for the memory footprint but I stayed with Rust for this. Rust became my daily-driver for hobby projects because refactoring and tinkering with old projects is a breeze. For me, Rust projects don't seem to require much cognitive burden to load into my human memory. I do not claim that this quality is unique to Rust however it is very much present in Rust.
Safe, GC-less memory allocation. Nim requires a GC, at least runtime refcounting, in order to not leak memory: https://nim-lang.org/docs/gc.html
The ownership/lifetime system is basically novel to Rust (well, it's based on Cyclone, which would probably have been an interesting choice for the kernel too because it's based on C, but Cyclone is no longer actively developed), and it allows you to get the performance of manual memory management (no refcounting required) without either leaking memory or hitting use-after-free bugs.
Rust's ownership system also tracks shared/immutable references and unique/mutable references at compile time, avoiding a whole bunch of setups for creating race conditions or otherwise corrupting data.
But shared/immutable vs. unique/mutable references aren't the same thing, they're something more interesting. Rust has three types of ways you might be able to get at a variable: you could own the variable, you could have a shared/immutable reference to it, or you could have a unique/mutable reference to it. The Nim proposal only distinguishes the ownership case (responsible from freeing the data when it's done - basically RAII or __attribute__((cleanup)) or autorelease) from the unowned case.
Having shared/immutable references gives you a neat feature for achieving correctness, not just memory safety (and I strongly disagree that the idea of "memory safety" in that post really counts, incidentally, because by that definition Heartbleed wasn't a memory-safety bug, but for the sake of argument let's say it does). The compiler prevents you from having more than one mutable reference at a time. It also prevents you from writing data you own while someone holds a reference to it, or even reading data while someone holds a mutable reference. So if you're writing code where you pass a reference to some other routine, and that function ends up holding onto it for future use, you won't be able to modify it. That prevents logical bugs where code doesn't realize that data has changed or conversely doesn't realize that a change hasn't taken effect.
Again, this is not solely about memory safety. This is about easily writing correct code, of which memory safety is one aspect.
You can think of this as basically type-level reader/writer locks. At any given time you can have multiple readers and no writer, or exactly one writer. While runtime memory safety is one thing that reader/writer locks can do (e.g., preventing you from calling realloc() while other people still have a pointer that they intend to read), they're used for a lot more than that.
Furthermore, Rust's approach requires no runtime overhead and preserves correct behavior. Nim's approach requires refcounting, and the post argues that refcounting could be disabled in a release build if you're confident your code is correct.
The current direction in Nim is towards borrowing and borrow checking, move semantics with type annotations, and using refcounting where needed. Similar to Rust really, which also uses refcounting where needed.
Nim had the concept of borrowing from very early on. It's not a bolt-on, it's part of the design philosophy. But it's had consistency issues. The move semantics in newer Nim improve on it.
I would say one of the major things Rust offers over Nim at the moment is the implementation quality of the Rust compiler, standard library and type systems is generally of higher quality and more consistent. Things are thought-through more completely in Rust's design, and tested more against real world problems.
The multi-threading in Rust is also of much higher quality. It's just buggy in Nim.
Nim however has a much cleaner syntax, and the quality issues could change if the community really wanted that (I don't think it does).
They both have slow compile times for large programs.
Not learning Rust or C (what it's replacing in this context) will probably not seriously dent your career prospects unless your career is already in systems programming.
Learning a language people do systems programming in can be a good experience but it doesn't need to be Rust for you to get that experience.
If you want to learn Rust because you want to, that's fair too.
Most of your job will involve taking user data, validating it, putting it into some database, then retrieving and serving it, probably as JSON.
You may edit an HTML template here and there.
Maybe even some configuration work for CI/CD.
Rinse, repeat, with whatever new language/framework is hot right now.
Basically, you will be solving the same set of problems on every job, but in more convoluted and unnecessary ways, because the guy who worked at Google says that's how they do it over at the FAANG.
I was being half-sarcastic. But, if you take most jobs out there, and you reduce them to the barebones of what it is you do every day, like really distill it - it's that.
If you have passion for that kind of stuff and that is your "thing", yes, of course. But most of the time, we just got rent to pay and there is not enough room for experiments with your life and career. Experiment in college, kids, while you can make mistakes - including sexy mistakes.
There are a lot of people on HN that are just stuck in this industry because it pays correctly but are doing things that are far from what they expected when they started. Please don't exclude them for no reason.
That's fair. I just didn't like the negative-sounding comment that it will always end up with CRUD-for-rent. If people enjoy business line apps thats awesome, it can be fun, but there are alternative roles out there too.
To be fair, even if it’s off topic, finding a job in this industry is stupidly easy. Finding a job that is fulfilling, well paid, and in a great company is difficult.
And I don’t even mean it’s a tragedy : I largely prefer doing CRUD-for-rent in a great company than technically fulfilling job under asshole management in some unstable startup.
From a professional perspective, what do you want to work on?
Web technologies? CI/CD, IT automation, etc.?
Or something more like operating systems, robotics, hardware, drivers?
Or do you want to self-educate for the sake of it, to learn different paradigms and be a better programmer in general? (Everyone wants to do this, but not everyone has the time).
Golang and Python are incredibly popular in the automation/"glue code" space.
Java is popular in enterprise development (think banks).
Systems languages are popular if you're developing... systems.
Depends what you want to work on. Relative to other software projects, not many jobs involve kernel development, and those that do will mostly continue to use C, as it's production-ready for the Linux kernel, and development often involves extending or modifying existing code.
No. Rust jobs are a niche - not even 0.1 % of the jobs out there. Most jobs are in frontend, mobile or standard business backends. So Java, C#, Python or Go are much better choices regarding job prospects and going to be for a long time to come.
"The Rust team has prepared a new release, 1.52.1, working around a bug in incremental compilation which was made into a compiler error in 1.52.0. We recommend all Rust users, including those currently using stable versions prior to 1.52.0, upgrade to 1.52.1 or disable incremental compilation. Guidance on how to do so is available below."
Is this mature enough to be bring in the kernel ????
As far as I know this is all still so new that such a thing doesn't really exist yet. I think the closest that there is to that is probably my post on implementing /dev/printerfact: https://christine.website/blog/dev-printerfact-2021-04-17
Reading the comments here makes me less optimistic about Rust support in Linux. It starts with adding support, and will perpetuate into flame wars about why so and so needs to be replaced with Rust cause C sucks. Before long the kernel devs will no longer enjoy working in a toxic environment.
A new systems programming language that gains wide industrial adoption is a historically momentuous thing. Lots of discussion is totally warranted - it's not just another hype thing anymore.
IMO, especially for short nouns, using the full name goes a long way towards making something more readable.
Using pronouns can very quickly become confusing. When someone says "it" are they talking about rust, linux, the kernel, linus's temper? Using "rust" instead of "it" completely removes that ambiguity.
Pronouns are pointers. Pointers cause problems - in programming by pointing to something that disappears, in spoken language by having antecedent confusion.
Rust attempts to solve the worst of pointer problems - I think overusage of pronouns would undermine the whole Rust effort.
--[[ Some architectures are probably going to be excluded since GCC supports more architectures than LLVM, which is what Rust uses as its compiler backend.
I'd rather see a new kernel optimised and written in Rust from scratch rather than dragging the legacy skeletons around and also bringing the vulnerabilities with us.
]]--
Some architectures are probably going to be excluded since GCC supports more architectures than LLVM, which is what Rust uses as its compiler backend.
I'd rather see a new kernel optimised and written in Rust from scratch rather than dragging the legacy skeletons around and also bringing the vulnerabilities with us.
> On compilers, we would like to mention all the work that has been
going in GCC Rust (a GCC frontend for the Rust language) and
`rustc_codegen_gcc` (a `rustc` backend for GCC). The latter now passes
all `core` tests, and the former is now working on traits. We continue
to track their progress as they may become the best way to have
GCC-built kernels with Rust support enabled.
Writing a new kernel with all the capabilities of Linux is an absolutely enormous undertaking, billions of dollars worth: "it would cost approximately $14,725,449,000 dollars [...] to rewrite the existing code." -- https://en.wikipedia.org/wiki/Linux_kernel#Estimated_cost_to...
There's a kid in university somewhere about to send an email that says "I've written a version of Linux that runs on my rasberry pi, it's not going to be big and fancy like Linux, but I've written it in Rust and hope people like it."
30 years later, there'll be a kid in university somewhere about to send a thoughtmail that says "I've written a version of..."
>I'd rather see a new kernel optimised and written in Rust from scratch rather than dragging the legacy skeletons around and also bringing the vulnerabilities with us.
There is Redox, which was intended to be practical, but I'm not aware of much public interest in that. Phillip Oppenheimer blogged about one too, I think, but that's explicitly academic.
I think it’s pretty funny that you can farm up votes by just mentioning a project in rust, with knows substantive information provided about its value or quality. I wouldn’t consider either of these things more than a toy.
> I'd rather see a new kernel optimised and written in Rust from scratch
That'll take a momentous amount of effort and it'll take 20 years to get close to the adoption Linux has today. I'd rather see if we can improve Linux in that timespan.
For this reason rust is currently being proposed to be used for drivers and other code which is already platform or architecture specific, not the core, so as not to affect the support for these architectures. When rust has a mature enough GCC backend then this won't be an issue and it may be possible for it to move into core (though this would still be a big step and likely quite a while away, for example because it would introduce a hard dependency on a rust compiler).
Given that they devote a section of the notes to discussing efforts in supporting multiple architectures and, explicitly, GCC, you'd figure that some do find those efforts are worthwhile and relevant.
In my experience with mid size projects the key to success has always been controlling complexity. I can only imagine that this is even more true for large projects.
Generally the complexity an additional language brings to the table is immense in several aspects.
Specifically Rust has it's own problems and drawbacks.
Looking back at the history of Rust and Firefox/Servo, introducing Rust was overall quite a disservice.
I feel Rust offers several strong points whilst actually being performant and widespread enough to be in the kernel. I doubt you absolutely _need_ to know both to work on specific parts written in either
I think Rust support should be in a more experimental branch for the kernel first before even thinking of adding it to the main code base. Mostly because I think it needs more supporting code and documentation to get people on board with such an inclusion: complexity or not.
"linux-next" is the experimental branch for the kernel.
"The linux-next tree is the holding area for patches aimed at the next kernel merge window. If you're doing bleeding edge kernel development, you may want to work from that tree rather than Linus Torvalds' mainline tree. "
This is a bigger point than it might sound. While "learning C" doesn't sound like a huge task, learning C well enough not to create disasters when working on a kernel, is a huge undertaking and requires a lot of experience.
Being able to do kernel development in Rust should lower the entry barier into kernel a lot. Because it has more high level constructs and especially for all the safety Rust brings. Any competent programmer should be much more comfortable to start kernel dev with Rust than with C.
C is a terrible language. It's portable assembly. I've done some small OSS projects in C and as soon as I realized Rust was stable enough I tossed them and rewrote in Rust.
Rust feels like a modern language, like Go or C#. C is barely better than FORTRAN.
I'm massively more productive in Rust and I don't have to worry about a million ways of causing UB or memory bugs.
Everything from dependency management to testing to cross compiling are dreamy in Rust compared to C.
You now are going to have two languages + bolting on a kernel riddled with chronic security holes and still supporting dead architectures and 32 bit systems.
Well see if brown-fielding Rust into the Linux Kernel is all worth it for them.
Rust support in the kernel is good news, but it's just inexcusable that we've had to wait this long for a post-C language. Human beings cannot consistently write correct C --- not you, not me, not Linus Torvalds. Spending decades writing C when a safer language was available caused the industry needless pain.
C++ support could have been viable a decade ago had it not been for strong egos on LKML refusing to admit they were wrong. Rust support in Linux is at least partially a face-saving way of backtracking on C++ in the kernel.
Rust is great, but with the sole exception of Rust's reified lifetime management, C++ is the better language.
Agreed about the inability to write C, but the part about C++ is simply not true.
I don't know about "better" in the general sense, but for the specific use case of the Linux kernel, Linus has identified some entirely reasonable objections (though aggressively worded) to C++: https://lkml.org/lkml/2004/1/20/20
- the whole C++ exception handling thing is fundamentally broken. It's
_especially_ broken for kernels.
- any compiler or language that likes to hide things like memory
allocations behind your back just isn't a good choice for a kernel.
- you can write object-oriented code (useful for filesystems etc) in C,
_without_ the crap that is C++.
Those objections do not apply to Rust, which avoids exception handling, avoids implicit allocations, and has a much smaller OO system that resembles what the kernel does for filesystems. Those objections continue to apply to C++.
There are good, principled reasons to accept Rust and continue to reject C++.
(There are also good reasons to believe that C++ doesn't solve any of the problems with human inability to write correct C, meaning that it wouldn't be worth the cost of adding a new language: https://alexgaynor.net/2019/apr/21/modern-c++-wont-save-us/)
The passage you've quoted is exactly the sort of angry technical schlock that's held back Linux specifically and the industry generally.
> the whole C++ exception handling thing is fundamentally broken
Not an argument. It's the closest thing in Linus's excretion of unfounded technical opinion to an argument, but it's nevertheless not an actual case for anything. Don't like exceptions in C++? Don't use them. (That said, exceptions aren't "fundamentally broken" no matter how many times Linus repeats that baseless slander.)
> any compiler or language that likes to hide things like memory allocations behind your back just isn't a good choice for a kernel
This is a general-purpose tirade against any abstraction whatsoever. Good software engineering is all about judiciously "hid[ing] things", including memory allocation. And as it turns out, Linux has plenty of abstractions, some of which involve memory handling. Also, you don't have to use any particular abstraction with C++: you get to choose the abstractions. The language doesn't force any on you.
Besides: Rust can do the same sort of abstractions that Linus claims disqualifies C++ as a "good choice for a kernel". It's almost as if C++ was fine even back in 2004 and that the only difference between then and now is that Linus has matured.
> you can write object-oriented code (useful for filesystems etc) in C, _without_ the crap that is C++.
This is a restatement of Linus's opinion, not an argument. You can write everything in assembler too: why take on the baggage of the heap of crap called C?
I agree that the email does not set out a solid argument and the "angry technical shlock" is unclear - that's what I was getting at by "aggressively worded".
I think there exists a solid and reasonable argument along these lines, even if Linus did not phrase it, though, which is what I'm getting at. (And I suspect Linus and others do believe that argument, even if they're not good at communicating it, so it would be worth engaging it.)
It's true that you could write C++ without exception handling, and that you get to choose the abstractions. But it matters what the language makes easy for you - what is idiomatic and conventional in the language - what programmers who use that language have come to expect their libraries to do.
Otherwise you could just say that we can solve our string problems in C without switching languages by simply using length-prefixed strings and buffers and not relying on NUL termination. After all, the kernel doesn't use a userspace standard library, and even in userspace there are string-handling libraries in C that avoid those problems (e.g., http://bstring.sourceforge.net/ or https://github.com/antirez/sds).
But the fact remains that the kernel has its own implementations of strcat and sprintf nonetheless (in the lib/ directory), simply because C programmers expect them. And people call NUL-terminated strings "C strings" and length-prefixed strings "Pascal strings," even though you can do length-prefixed strings just in C and NUL-terminated strings just fine in Pascal.
It is very hard to write abstractions in C++ that work the way the abstractions in Rust work and are still natural and easy to use. No one is disputing they're possible to write. But if you look at the blog post I linked, it demonstrates that even with the benefit of decades of experience in C++ and knowledge of what Rust is doing, and even with additional language features, and even with the input of people who did in fact choose their abstractions (std::string_view was based on libraries local to Google, LLVM, and Bloomberg), you still do not get the level of safety that the equivalent Rust feature has.
Concretely,
1. C++ has language-level features (most importantly, try/catch/finally) that make error handling via exceptions easy and natural to use. Rust has language-level features (most importantly, pattern matching on sum types) that make error handling via a result type easy and natural to use. You can of course choose not to use exceptions, but you're giving up a good language feature and you don't have language support for the thing you want to do instead. This is much like saying "Don't like NUL-terminated strings? Don't use them" - technically true, but have fun avoiding string literals.
2. This is not a general-purpose tirade against abstraction. It is a very targeted tirade against implicit conversions to heap-allocated types, which are natural and idiomatic in C++. See my comment https://news.ycombinator.com/item?id=27749301 elsewhere in this thread for details on exactly what Linus is objecting to and why it is entirely reasonable for a kernel to object to it. Again, you could certainly decide to avoid implicit allocations in your nonstandard library, but it is not idiomatic C++.
3. C++'s object model is legitimately quite complex, involving things like multiple inheritance, partially-constructed objects during the constructor, and phrases like "morally virtual base". It works - there's definitely a lot of working C++ in the world - but that level of complexity isn't needed by the kernel, and (certainly in my personal experience) it's easy to get into deeply confusing situations. The kernel also cares about things like fitting objects into cachelines. The kernel doesn't need multiple inheritance with the ability to have different offsets into an object depending on what pointer type you're using, but it does need the ability to specify that objects implement multiple interfaces that carry no data (what the kernel calls "operations structures," and what Rust calls "traits"). In C++, the only mechanism the language gives you for this is multiple inheritance. There are use cases where full multiple inheritance is useful, and C++ is preferable to Rust for such cases. The Linux kernel is not one of them.
It's true that you can do all the things Rust can do in C++ if you try hard enough. But I'd reiterate the point which you made clearly, that human beings simply do not seem able to write flawless C in practice, even though it is theoretically possible, because it is simply not idiomatic. From the blog post I linked, it seems like there's no reason to expect that asking humans to write flawless and unidiomatic C++ will work out any better.
Will Rust be better than C? The kernel uses a lot of non-local reasoning and global resources, when Rust handles them, a lot of its benefits might evaporate. I guess the main authors behind this change have a more optimistic vision of this, than me, though.
> Can't disagree with the idea of having C++ in an os kernel strongly enough
Why? How, precisely, does C++ hurt?
> It has certainly been tried and in the cases I know of, regretted.
Did it? Fuchsia didn't regret C++ in the kernel. GDB didn't regret moving to C++. (I know GDB isn't the kernel, but it's systems-y.) Android's libc, Bionic, is also written in C++.
Every single argument I've seen against C++ in systems code has been specious: people act like typing "g++" instantly implants a copy of the GoF book in your head and turns you into a design patterns robot. "I don't like it" is not a technical argument. Neither is a profanity-laced rant, as satisfying as it might be to pound out such a thing.
You can just use C++ as a safer C. Linux would have gotten huge rewards in both safety and developer efficiency if LKML hadn't spent a decade doubling down on their nonsense arguments against C++.
> Every single argument I've seen against C++ in systems code has been specious: people act like typing "g++" instantly implants a copy of the GoF book in your head and turns you into a design patterns robot. "I don't like it" is not a technical argument. Neither is a profanity-laced rant, as satisfying as it might be to pound out such a thing.
Well put. Given the style of C used in the linux kernel, it should have been possible to restrict the use of C++ in the same way. Yes there is lots of bad C++ in there, but it doesn't have to make its way into the kernel. The maintainer s can reject code that is uses inheritance, or templates in the wrong way, etc.
It's like believing you shouldn't use C in the kernel because it's possible to write lines that are over 80 characters long.
At least one of the L4 implementations. IBM's K42. I think one of the windows kernels tried it once then reverted? Or did every single windows kernel use C too? Symbian really hit a wall taking Nokia down with it. The N95 was well established in the market before the iphone hit but they couldn't get the slow out. Blame it on C++, something else, whatever.
C++ is fine for systems code done tastefully. GDB, GCC, libc, java, go nuts! Kernel, not so much.
There's a few things going on. You have to agree on a subset of C++ which is a royal bloody pain in the backside at best. It's like a bracing style meeting where the issues actually matter. Some of this is decided for you, you can't have exceptions. Now you can't have operator overloading either. Now constructors are a bit of an issue to consider fairly deeply, "YOU'RE KIDDING YOU'RE SO WRONG I HATE YOU!" And I don't even know what you think about it yet! ;-) Now do that with thousands of contributors. Nasty. Google and Fuchsia? I think they've already got rid of most contributors with the CLA. https://cla.developers.google.com/about/google-individual
I couldn't find an actual license and didn't feel like curling to bash to find out what's in the code. It's not surprise that most who try it have a microkernel based system. It is doable. It's been done! Criticising Linus for refusing to go that way seems a little naieve to me tbh. He can point at considerable success as a result of his fundamental approach. That may not be the end of it. But it clearly wasn't a disaster. Critics haven't managed to achieve the success the early days of linux yet as far as I'm aware.
C++ compilers have universally sucked incredibly hard until fairly recently when gcc and clang really lifted the standard. No the crooks at intel and the way they sabotaged their customers to make their competitors look bad do not count. They lost the right to be taken seriously at that point.
C++ for cross platform has been horrible until very recently. A = B + C + D; You know exactly what that does in C without thinking. C++ not so much. Having those extra layers of thought required between the reader and the machine is unpleasant for OS stuff.
C++ has its place. Many of its features are extremely useful and seductive. Many can be used tastefully in userspace to make better, stronger, more reliable code. The Kernel? I don't see it . You want to do dynamic dispatch in C. You can see exactly how that will work on the machine. Virtual functions. No, there's magic there. The implementation isn't guaranteed and can change.
There are simply more corner cases, gotchas, foot guns and nastiness where you really don't want it. Kernels are hard enough.
BeOS kernel was written in C, "It may come as a surprise to some BeOS developers that in our wonderful, buzzword-enabled, OO-powered OS, there is not a single line of C++ code in the kernel." (https://www.haiku-os.org/legacy-docs/benewsletter/Issue4-39....)
Windows NT kernel was all C apart from bits of the graphics layer - friction from that team's use of C++ is a theme in /Showstopper/.
Whoa, Symbian's C++ was quite different to the C++ most people understand. 2-phase construction, no exceptions (Leaves and Traps), no string type (Descriptors).
It was quite difficult to write, and new hires had to essentially be retrained. Expecting an app ecosystem to be built on Symbian C++ was unrealistic. It's the lack of an ecosystem that killed it really.
Not really, any C++ compiler is able to compile Symbian C++, it was just a bunch of conventions on how to write C++ code.
C++ haters seem to always try to use the move that unless one isn't using 100% of language features and standard library, one isn't using the language.
Yet the same measure doesn't apply to C, when it is crippled to use in kernel code.
On the other hand I'm slightly suspicious of this movement. Rustaceans are very, very eager to promote the language. It would be interesting to know how much this movement comes from within the Linux kernel community or if it is driven more by the Rust community.
In any case the language seems to have potential of becoming a new mainstream systems programming language in a few years.