A very helpful summary from r/rust on the same topic [1]
>A few notes for people who haven't been tracking the Rust-in-kernel situation. Drivers especially have been targeted for a few reasons. First, drivers are isolated, with abstractions that don't require interaction with the rest of the system. Second, Linux is available on more systems than the Rust compiler has been ported to. Because drivers are inherently system-dependent, it is not an issue to introduce a dependency on rustc on compatible systems. Third, bugs tend to arise more in drivers than other parts of the codebase, so Rust can do the most good starting there.
I'm not sure I agree with this. As someone who does development for prehistoric PPC embedded systems, I get to take advantage of a lot of drivers that were never really intended for my platform, yet work just fine because they're based on standard busses like PCI and USB.
Consider some architecture supported by Linux, but not yet Rust. Now, consider that a vendor might create a driver for their fancy new PCIe network card in Rust with the logic being "well, this card was never intended to be used in anything other than an ARM or x86 system". There's a good chance that if that driver was written in C, it would have worked just fine (albeit not officially supported by the vendor) on that old non-Rust-supported architecture, but now that's not an option.
Rust should not be used in the Linux kernel until it supports 100% of the architectures supported by Linux itself.
> Rust should not be used in the Linux kernel until it supports 100% of the architectures supported by Linux itself.
I don't think that's realistic.
If we buy the idea that Rust will allow kernel developers to write more robust, safer, more secure drivers, I think that should override a desire to allow things like driver portability to arches that hardware manufacturers never expected or intended. I would judge that support to be a "nice to have", but it shouldn't block what I consider to be more important concerns (security and robustness).
Regardless, the Rust support here is still considered experimental. I would be surprised if any company adopted it for a Linux driver for at least a year or more. That's plenty of time for people to get esoteric platforms supported in LLVM/Rust if they want to. And it also gives more time for the Rust GCC backend to become more mature and usable, and I believe that will give Rust support for every arch that GCC supports (more or less), which should cover your objection.
(At any rate, LLVM+Rust does currently support PPC, so your specific argument here isn't relevant.)
Yes, why not? Must it absolutely be so that a 20 year old Macbook can run the latest kernel? One that isn't even released yet, it's just in the preliminary planning phase? It's not even like new software wouldn't run on it. Most software runs fine on 10 year old linux kernels. Also, by calling them 'users' you're anthropomorphising, who are these actual real human beings being "left behind"?
edit: ok so it sounds a bit mean if you interpret it as if I'm not considering UnkleJoe to be a real human being. What I mean is, I don't think UnkleJoe is trying to get new devices to talk to his prehistoric PPC embedded system just for fun. An organisation is maintaining these systems, and that organisation is mooching off of Linux's excellent backwards compatibility. What's the economical cost of supporting that organisation, versus the economical costs of not implementing ksmbd in Rust?
Security updates for one. People are always complaining about insecure IoT devices/routers that get conscripted into botnets. Dropping kernel support entirely compounds to manufacturer firmware negligence - meaning end users can't update to the latest Lineage/DD-WRT/OpenWRT firmware, even if they want to.
> Most software runs fine on 10 year old linux kernels.
If you had to wrangle with world of custom Android ROMs or open source router firmware - even as a user and not a developer - you know that software does not work fine on old kernels. Trying to support both old and new kernels is a royal pain, especially if you have add shims to support old kernels that lack new features.
I thank $DIETY for Linus' pragmatic approach, I trust him to balance user support against developer zealotry. I wonder how long it'll take for Linux to go to shit after he retires. Kernel notes for v42.3: dropped support for Risc-V. Remaining supported archs are x64 and ARMv21-24
I thought we were talking about new drivers being developed in Rust. Your IoT security upgrade probably doesn't include a hardware upgrade, so it will not require these new drivers.
Regardless, it is the responsibility of the device manufacturer to support their old devices. It is their problem, hold them accountable.
> I thought we were talking about new drivers being developed in Rust
The discussion moved to what happens to drivers that are not ported to Rust, and GP was arguing they should be abandoned/dropped - as opposed to requiring that all driver's be switched over to Rust first.
> It is their problem, hold them accountable
How can they be held accountable, pragmatically? Also, owning a CVE-riddled, insecure device is actually the user's problem. Open firmware can at least ameliorate the situation right now: where manufacturers cant be bothered to publish/upstream/update their drivers to the latest kernel API in the same language. Asking them to rewrite drivers in Rust and upstream is obviously a tall order - barring the passing of new legislation.
The same as any other project that decides it may drop support: either stick on an old version, or work to add support for the platforms you care about. Rust's lack of platform support isn't because it is impossible for Rust to gain support for those platforms, but because users of those platforms haven't added support yet. Obviously that is easier said than done, and I doubt that Linux would make a decision like this, but the options are pretty straightforward if they did.
If you need to support plugging in new hardware to a very niche embedded architecture, you will still have many options. These range the gamut from writing a driver based on the reference driver for your environment, to requesting/funding the development of such C driver from the vendor, to funding support for your hardware architecture in Rust/LLVM. Yes, all of these cost time/money. Maintenance of software ports to various architectures is very much not zero-cost, and assuming it is will eventually bite you or your business hard.
As another thought experiment: let's say the development of drivers in Rust is less expensive (either in $$$ spent on engineer time, in $$$ lost to bugs and security issues, or any other valuation you choose. If you/your community of people who desire new hardware to be supported by your embedded architecture can not cover this difference in cost (to make it worthwhile for drivers to be written for you), nor can afford to develop and maintain support for your architecture in Rust (so that you can benefit from the development work done by the vendor), nor can afford to upgrade to a newer hardware platform (which solves the problem completely), then your business is already screwed and you just don't realize it yet.
That said, this is the responsible way to introduce Rust code to the kernel - if this had be done by limiting future releases of Linux to only those supported by Rust, I'd be out with my pitchfork too, because that's something that really needs to be planned very carefully. As long as stable kernel releases continue to exist that support the full set of architectures, everyone's existing code will keep working, receive security fixes, etc. This is fine.
I completely agree. We should not get ahead of ourselves just yet. It seriously should support most if not all of the architectures that Linux itself supports.
LLVM is the real bottleneck here. I've worked with a platform that would have been easy to build a custom rust for (even if it wasn't officially supported), but was held up by the lack of LLVM support.
Yep. But there are two different active projects for compiling Rust with a GCC backend at the moment (one as a completely new frontend written as part of the GCC codebase, one as a libgccjit backend for the existing Rustc), there's also a Cranelift backend for rustc that you could add new architecture support to, and LLVM does add new targets, it can just take a bit longer to get support into LLVM and they don't keep architectures that are lightly or unmaintained around as long as GCC does.
For instance, Tier 3 support for m68k-unknown-linux-gnu was recently stabilized in rustc as a result of m68k support landing in LLVM (https://github.com/rust-lang/rust/blob/master/RELEASES.md#co...), which has been one of the previous sticking points for Rust packaging on Linux distros, some of which still support m68k architectures.
Chances are that Rust would also work just fine even if the arch is not officially 'supported', as long as LLVM exists on that platform.
Also, this is not going to retroactively affect any drivers, only the ones (potentially) developed from now on. As time passes, Rust support is expected to increase.
Yes, x-compiling is certainly an option, particularly since developers typically have access to hardware easily capable of such (it's just a minor pain to set-up).
Linux has always favored support for currently (and in the foreseeable future) relevant architectures over widest possible spread.
I don't quite understand the desire to run recent software on museums piece hardware, but for those who must, there's NetBSD.
> Rust should not be used in the Linux kernel until it supports 100% of the architectures supported by Linux itself.
Why is it that so many people feel like they need to make sweeping pronouncements on the exact compatibility guarantees that a project as large as the Linux kernel must maintain across its entire code base?
Different maintainers of different subsystems of the Linux kernel make their own decisions on compatibility and support effort. There are a few basic principles shared across the project as a whole, such as user-space ABI compat guarantees, lack of kernel-space ABI guarantees, module licensing requirements, etc.
Not every piece of hardware even exists in the form of a PCIe expansion card. Some drivers are for peripherals that are embedded into a specific line of systems-on-a-chip. If I'm writing a driver for some peripheral that only exists on an AWS Graviton chip, I don't see any reason why some HN commenter should dictate that that driver can't be written in Rust just because there isn't yet upstream LLVM support for DEC Alpha chips.
I feel like most of these takes are due to someone getting upset a couple of years ago because some core library in Gnome like librsvg added a dependency on a Rust compiler back when rustc didn't yet support m68k systems, and this got someone working on packaging upset due to conflicting distro packaging policies.
The world moves on, however. The Linux kernel drops support for old hardware all the time. It introduces features that might only work on certain hardware. Also, rustc has added support for more platforms, including moving some platforms like aarch64 to Tier 1 support, and adding support for m68k.
While maintaining backwards compatibility or wide portability of drivers is a good thing, so is the much easier ability to write maintainable and secure code that Rust brings, so the desire to simply block one good thing that works on the vast majority of machines running Linux in favor of some small niche use cases doesn't really seem like the soundest decision.
There are at least two efforts that support Rust with GCC (a rustc codegen backend that uses libgccjit for AOT compilation, and a GCC frontend that parses Rust).
That is actually a good point, but on the other hand you have to start somewhere.
I hope vendors would be more supportive of third party drivers because that obviously makes their hardware more attractive, but that has not been the case the last years.
I also don't have an opinion on the rust foundation. Member setup doesn't convince me, hope it doesn't influence the language negatively.
Strongly agree here as a long time driver developer. I honestly don't see the point to rush rust into drivers in the kernel now, if anything, it makes the driver space even more complicated.
> The GCC backend for the Rust compiler (through `libgccjit`, although
used for ahead-of-time compilation) was merged into upstream Rust:
https://github.com/rust-lang/rustc_codegen_gcc
If the backend keeps progressing, we hope to soon be able to start
experimenting with compiling the Rust side of the kernel with GCC!
This means that using it for the core parts may be feasible sooner rather than later, no? Developing those abstractions may not be a priority of course, but it might be allowable at least from a platform support perspective.
> This means that using it for the core parts may be feasible sooner rather than later, no?
More like much later. There are several performance-critical patterns used in the Linux kernel that are difficult to express in unsafe Rust and impossible to express in safe Rust.
I think promoting the use of Rust for driver code is an excellent compromise.
As an aside, I'd like to see Linux go further and move many of these drivers to user space. Now that we have io_uring, old performance-based arguments for keeping certain drivers in ring 0 hold less weight.
Even once Rust has first-class support for GCC as a backend, I suspect that supporting the long tail of more obscure platforms will be an ongoing process. And for exceptionally weird platforms (e.g. where a byte isn't 8 bits) Rust may never bother to contort itself enough to support those, although perhaps Linux also doesn't support such platforms?
> Even once Rust has first-class support for GCC as a backend, I suspect that supporting the long tail of more obscure platforms will be an ongoing process.
Linux's own support for the long tail of more obscure platforms is also an ongoing process. People aren't asking for Rust to do better than the Linux kernel in that regard, they're just asking for it to not immediately disqualify a range of platforms.
One could reasonably argue about the value of supporting those platforms, but the Linux kernel, reasonably, doesn't want to make that decision on the sole basis of the availability of Rust support.
> And for exceptionally weird platforms (e.g. where a byte isn't 8 bits) Rust may never bother to contort itself enough to support those, although perhaps Linux also doesn't support such platforms?
Linux doesn't support such platforms, and I don't think Rust ever will either.
FWIW Linux does support s390 (n.b., not to be confused with s390x), which is a 31 bit arch. Rust does not support it (and I'd guess LLVM doesn't either). So Linux does support some "weird" arches still.
This is partially what caused the shitstorm when the python Cryptography library took a hard dependency on rust to compile:
You can emulate it using Hercules. When searching I found plenty of guides showing how to set it up. I'm not linking to any in particular, because frankly I have never tried them so I don't know which one you should follow.
AFAIK, s390 also has 8-bit bytes. The "31 bits" is the pointer size; IIRC, it's actually 32 bits, but with one bit reserved for compatibility with its even older predecessors.
Just a guess, use of the gcc backend for rust will face hard opposition by the same people pushing for rust in the kernel specifically if it is chosen (as it should be) by the kernel as the default choice.
i don’t want rust in the kernel, because it will make my job as a small device manufacturer even more complicated. it is already super hard / time consuming to understand how to build custom drivers and tune the kernel for specific applications, so i don’t need or want another language involved in there. in the end, you still need to understand what you are doing, and rust is not going to change that.
the days that linux was only used as a secure server OS are long gone. it is used everywhere for all kinds of applications and by lots of small manufacturers who are using the pi or other platforms.
also, you cannot just develop a driver in isolation, you have to take the typical use cases in mind while you make choices how to handle things.
I understand your concern and I would definitely feel the same if some language I wasn't familiar with was suddenly introduced in the kernel. But at the same time as a big Rust enthusiast I think it's really a good idea in the long run: C is going nowhere but it's really dated, design-wise. When you've tasted the sweet nectar of type inference, strong typing, union types and of course the safety of the borrow checker, returning to C feels like banging rocks together.
The main drawback of Rust here I think is the longer compile times (I dread to imagine how long a fully Rustified kernel would take to build) and a generally more complex, but also more expressive language.
More generally I trust the kernel maintainers not to succumb to the hype and overdo it. People who still exchange patchsets over mailing lists are probably unlikely to be blinded by shiny things.
From the perspective of people who don't know Rust, it's not a sweet nectar, but a suspicious murky liquid.
From the perspective of people who don't like aspects of Rust, it's a yucky goo.
This is _not_ like with C. Few people are enamored with C, but - the language itself is relatively simple (Duff's device notwithstanding...), so in my analogy it's like water. You may not adore it, but you can see through it, and you'll be ok with gulping it down to quench your thirst.
> From the perspective of people who don't like aspects of Rust, it's a yucky goo.
> This is _not_ like with C
I beg to differ. This is exactly how I feel about C. C is the language that's confusing as hell to read for anything other than really simple use cases AND makes it super easy for me to shoot myself in the foot.
No, you are wrong. C++ is one example. I've been using that language now for about 20 years and I still don't use all of its features. It is possible to write code in that language that you assume will work a certain way but then in practice it does a lot of things you didn't think you signed up for. Recently I started using STL more, after having stayed away from it for most of my career, and the performance of my application went out the window, and I had to override all memory allocation related functions to fully understand where the problem was coming from. It's easy to use the STL and make assumptions about performance, then find out later you get a lot of issues that come with it that you never signed up for. And that is why I don't want rust in the kernel. If a language does a lot of stuff for you, there will be a price to pay.
> It is possible to write code in that language that you assume will work a certain way but then in practice it does a lot of things you didn't think you signed up for
This is also possible for C. Most commonly around Undefined Behaviour. C only does what you expect it to do if you know the C spec inside out. Rust can have confusing behaviour too, but it is generally much better in this regard than C or C++.
You may find it easier to reason about C than Rust, but that's only because you know C better. I find it much easier to reason about Rust.
As long as you stay away from compiler-dependent, platform-dependent, and/or undefined behaviour, you should be fine. Main problem is that a surprising ab´mount of "looks normal" code is actually UB.
You would be "fine", but the compiler will optimize away alot of your code, and will rearrange a lot of the rest. I wasn't even talking about platform-dependent or undefined behavior.
I'm not sure that it's the concept of pointers that make C code harder to maintain than other languages. Lots of languages have pointers, including Rust.
It's a question of whether unguarded pointers (and a lack of language features that keep pointer math from devolving into an unreadable mess) are good to have in areas where pointer errors can result in critical bugs. We started discouraging goto statements not because developers didn't understand them, but because they did understand them, and they understood that the tools you use influence the type of code you write. And Rust doesn't even really get rid of pointers, it just adds rules around them.
We all accept some degree of abstraction/safety tradeoff, otherwise we'd program in lower-level languages than C that are even simpler and even easier to understand. C does introduce a lot of complexity over more primitive assembly languages. And Rust introduces a lot of complexity over C.
Different people pick different points where they think the tradeoffs are optimized. But we have a pretty large amount of data suggesting that even <quote>good</quote> programmers in industry make pointer errors in C. So it's pretty obviously not lack of understanding that's causing these issues, it's that unrestricted memory access and pointer math is inherently error-prone. Whether it's SO error-prone that's it's worth introducing a much more complicated, rigid language into the kernel is left as an exercise for the reader. There is a downside to introducing more complexity here, all abstractions have downsides.
But it's real dismissive to say that if everyone understood C pointers then their code wouldn't ever devolve into an unreadable mess. I just don't think that's an opinion that can be backed up by any real evidence, pretty much every org struggles to some extent with writing secure C code. And we have a lot of experience in the field of software engineering that should have taught us by now that sometimes unreadable code is the result of language affordances and patterns.
Also there are many different ways to undestand and use pointers; there are many many programmers that are really good at using pointers and are able to engineer essentially safe architechtures, but many of those very good programmers would come up with incompatible architectures around pointer usage.
My wild guess is that most "simple" pointer errors (use-after-free, double-free, unintentionally using unitialized memory, ...) happens at interfaces where there isn't perfect comunication about what component is responsible for keeping track of each task.
Yeah, I think about a very offhand comment that Carmack made once that ended up resonating with me a lot, even though I don't think it's universally across the entire board true:
> "Most bugs are a result of the execution state not being exactly what you think it is."
There is an aspect of language design that is about making it hard to be surprised by things. And I vaguely suspect that a lot of pointer errors fall into this category; people assume something is being freed, they assume that a subroutine is cleaning up after itself or that it's not manipulating something, or that it doesn't need to call some setup method before it's run. These are boundary errors between interfaces where allowing users to (inadvertently) mask what is happening can lead to logic that feels very convoluted and hard to follow, and can also lead to simple errors around just forgetting to do stuff.
I wouldn't call that Carmack quote a mantra, I'm not saying it explains all programming (or that all architectural restrictions are good), it's just a lens that I keep coming back to when I think about language design and architecture; there's a real tradeoff in removing flexibility, but there is also a huge amount of value in systems/patterns that make it very hard to bury relevant information. And if a language can auto-surface some of those problems, that can lead to cleaner code.
It depends what level you mean. Understanding and using pointers on basic way is simple. Understanding pointer provenance, aliasing rules, etc. are a completely different level and people not very familiar with the spec (so almost everyone) do not understand a lot about pointers. http://open-std.org/JTC1/SC22/WG21/docs/papers/2019/p1434r0....
I think type inference is much more important for people that used dynamically typed languages instead of C devs. You could argue that it is also much more important for Rust.
the point is that you still need to know what you are doing. rust won’t change that. C is everywhere, and that is not going to change anytime soon. your time is better spent understanding more of the kernel instead of learning another tool set.
one of the biggest hurdles for me was the DTB declarative language. that was the most stupid idea ever to invent that and i lost 100s of hours trying to get my DTB files right for my hardware. so you can understand i don’t want more hipster shit to deal with.
You need to know less about what you're doing with Rust at any given time versus C, for the programmer.
Also at this point, Rust isn't "hipster shit". You're just not familiar with it, and that's fine. But it's in wide use now and proven to solve major classes of recurring issues.
i’ve looked at it briefly. i do not need or want it, and that is fine. what is important, is to keep the kernel tree simple and understandable. that means minimizing the number of tools and languages involved. which means not introducing rust.
i don’t care about vulnerabilities. it is very annoying that everyone assumes that linux is only used for applications where security matters. plenty of devices using linux do not need to be secure and forcing rust onto the entire linux kernel community because of a vulnerability argument is stupid and will only make it harder for people to use linux in a project.
i think that has changed over the past 20 years. linux is now everywhere and not just on servers in data centers. and this requirement is not a valid reason to force rust onto all device creators who choose linux.
Assuming anything new is bad because of a bad experience seems like a poor lesson to take.
Of course you need to know what you are doing still, but pretending humans don't make mistakes even when they know what they are doing is ignoring reality.
As a counterpoint security in e.g. Bluetooth has been an ongoing problem and even "mature" code like ext4 has recurring problems. Somehow I doubt the problem is that the people involved are not smart enough or trying hard enough. Maybe it's worth revisiting what tools we use.
there are many situations where security does not matter. so that is not a reason to force rust into the code base. not everyone is working on a device that needs to be secure.
this is not about me. it is about keeping the kernel tree reasonably accessible for small startups and device creators who do not have the funding of FAANGs and cannot afford to take on more complexity on top of something that is already very complex.
You always need to know what you're doing, but Rust lets you encode more of what you mean in the code and the compiler checks that you're not violating your own invariants. That's the killer feature for me. I feel like I'm working with the compiler, not against it.
Sometimes I feel like C's compiler is almost adversarial, going out of its way to find small mistakes in my code and have them blow in my face at runtime instead of notifying me. I understand why it works that way of course, but usually when my Rust code compiles, it works. I can't say as much for my C code, despite nearly 20 years of practice.
I do share some of your frustrations with the DTB thing, I'm not a fan of it either, although something like that was needed and I'm sure any alternative would have its haters.
> your time is better spent understanding more of the kernel instead of learning another tool set
No matter your skill level, that won't stop you from committing memory-safety mistakes that may lurk for years until they suddenly result in a catastrophic failure. Evidence for that is abundant.
I have, and it was the most frustrating language I've ever tried. You sit there staring at the code wondering what's wrong with the variable you created until you Google it and find out it's a certain type that you never heard of. It's ridiculous.
I'm sure it gets better as you memorize the millions of different types, but what benefit do you get from all this pain?
I also think it's just a boy's club at this point. Big pipe dreams of rewriting all the C++ and C code is all we here about. It's not going to happen. The language is too f*k*g hard to learn so no one will. It's like trying to make everything in Haskell, c'mon, get real.
How do you mean, "memorize the millions of different types" and "You sit there staring at the code wondering what's wrong with the variable you created until you Google it and find out it's a certain type that you never heard of"?
I don't think I've experienced this in any language, even ones with the most expressive type systems. More typing is better, as far as I have experienced, and as far as I have heard from people who take it up in their projects.
The compiler will tell you exactly what goes wrong if you pass the wrong type. If you mean that type inference was confusing, it's the same as `auto` in C++. Except the Rust compiler does an enormous amount of handholding with errors, often showing you exactly how to correct your code.
>> I also think it's just a boy's club at this point.
Probably mostly because of that.
Not that the other points are necessarily easy to follow since they're not well explained. If knowledge of why some type was hard to know or had to be googled (or for example noting what one of those types is) was presented, then people might have been able to respond to those points.
Instead it reads as "it's hard, I used it and it's hard, and I had to do some stuff I didn't like, and so my impression is it's just a bunch of people faffing off and it's too hard to take seriously", which isn't really useful for a discussion, and additionally is dismissive. Every one of the criticisms presented could be said about C without any change because no actual concrete details are provided.
It's not like Rust is perfect, there are many valid criticisms of it that have real examples and make useful points. Those comments probably deserve your defense more than this one.
A file format to describe device trees, which describe the HW attached to the system. Mainly used on !x86 arches, as an alternative to ACPI-style autodiscovery:
Adding Rust doesn't make it harder to do any of this. When you build a custom driver, you could choose to develop it in C or in Rust. It gives you the choice, and many of us find it easier to build in Rust than in C; but even if you prefer C, you will still have that choice.
Most of "tuning the kernel for specific applications" consists of making changes to the kernel config or compiler settings. Rust won't change the kernel config system, and compiler settings for Rust are quite similar to those for C, as Rust uses LLVM as its backend, and LLVM is used by Clang which goes to a lot of effort to provide a mostly GCC compatible interface.
> in the end, you still need to understand what you are doing, and rust is not going to change that.
If think you understand what you are doing in C, you probably don't. C is a fantastically difficult language to write correct code in at scale; even the best developers make simple mistakes all the time.
I understand the frustration at seeing something else you might have to learn and deal with that might make things a bit trickier during a transition, but that's pretty much the nature of keeping up with any kind of ongoing development process. It's not like the Linux kernel sprang fully formed out of Linus's head on day one and it's just been a matter of adding drivers and making small incremental fixes. There are all kinds of changes that require you to change and adapt, like the removal of the Big Kernel Lock, or pretty much any internal API work, and yeah there can be work to keep up, but that doesn't mean you should hold back progress.
That just sounds like "I don't want progress because it will mean I have to learn new things and I don't want more work for myself". Well... tough luck?
Regardless, if you want to keep using C to write your Linux drivers, the new Rust support does not change that.
If you're a device manufacturer trying to manage common code for a variety of hardware platforms with subtle differences in toolchains / kernels, dealing with all that complexity can be pretty tough. If Rust drivers start popping up in the kernel, you now need to start dealing with twice as many toolchains and twice as much complexity.
One of those toolchains is used to build the bootloader, kernel and majority of userspace software for the devices. The other toolchain is used to build an I2C temperature sensor driver that was rewritten in Rust so someone could put Rust driver experience on their resume.
Dismissing this and saying "suck it up, this is the price of progress" is fine. But it doesn't change the fact that this could be a giant PITA for people making devices and developing the BSPs for them while providing almost no value. I'm not trying to argue that appeasing this group is important enough to totally reject Rust from the kernel, but as a maintainer, making life difficult for a huge number of kernel users is probably something you'd like to avoid.
I think a lot of this will come down to how Rust patches are reviewed and accepted. If Amazon or Facebook want to upstream some new Rust driver they wrote for their own hardware, there's probably no negative impact to anyone by merging it. But I'd expect maintainers will be very hesitant to accept patches that touch existing drivers. I guess time will tell.
amazon or facebook can afford to pay lots of developers to deal with this. i cannot, and if they force their rust code onto the community, it will close the door for lots of smaller device manufacturers including myself. and that is the main issue. linux does not exist to entertain the interests of those with more money.
Reading up on the Rust spec and concepts is fairly trivial compared to letting a lot of devices and their drivers interface each other. I don't think Rust inclusion will change that however. Time will tell, but your response is rather childish.
no offense but aren't security concerns just as pressing in the world small manufacturers, embedded etc.. given that those kinds of devices are nowadays part of pretty vital infrastructure?
the lack of memory safe languages is probably the nr. 1 contributor to critical bugs in software and regardless of what we're working on we ought to learn new languages if they help to improve software quality.
I don't believe that the change to rust (and I like rust a a lot) will lead to a measurably more secure kernel in the long term, although it will mean that maintaining the kernel will require knowledge of Rust in addition to C (and some flavors of assembly, occasionally). On top of that, the Rust that can be compiled into the kernel will be a subset, or its compilation will be tuned [idiomatic and safe allocation of memory within the stack, efficient Rust/C interoperability with dynamically allocated memory, in a Rust-safe way, etc.].
Let me explain why I think that:
In the short term, having more eyes looking at and re-implementing the scaffolding for drivers and the drivers themselves will certainly lead to a very positive outcome - many potential bugs (of which, most of them will not really be exploitable, but some will) will be uncovered which couldn't have been introduced with safe Rust. However, at some point, the coolness factor will drop, drivers will become fossilized, a larger developer base that has to maintain older Rust drivers might cut corners, use more unsafe features, misunderstand/misremember the nuances of this or that, and what was once exciting is now tedious. Maybe not, maybe Rust will slowly replace the whole kernel, however, at the current stage at which Linux is, I find it unlikely (but possible).
> a larger developer base that has to maintain older Rust drivers might cut corners, use more unsafe features, misunderstand/misremember the nuances of this or that, and what was once exciting is now tedious.
The great thing about Rust is that it makes it much harder to cut corners, and much more obvious when someone does (making it easier to pick up at review stage). Invariants can be built into APIs in a way that simply can't be done in C.
But the Rust safety guarantees aren't about eyeballs, they're about boring mechanical processes, the same difference as say, having your CI run unit tests and refuse to provide artefacts if the tests fail versus telling developers "Make sure you run the tests before shipping a release" which they will promptly forget to do.
The machine doesn't get bored. The fact Rust may not be "cool" any more doesn't alter the fact that your subsystem maintainer isn't going to accept your patch that doesn't compile because you tried to take the lock for foo, but then used locked variable bar - something C has no problem with but which in Rust of course does not compile.
I just really hope this thing will not be a bitch to cross-compile. I'm not interested in building llvm + rust in addition to my 5 or so gcc cross-compilers, especially looking at the cadence of rust releases.
So as long as it will be possible to use the system's llvm+rust to build for all 4 CPU archs (at once) I'm currently building my kernels for, I'll probably not be too cranky about rust in the kernel.
Cross-compilation with Rust is fairly trivial; it's just a matter of "rustup target add some-host-triple".
GCC's cross-compilation story is indeed a nightmare. I expect if you have to use the GCC Rust backend (after it's ready) for arches rustc/LLVM don't support, you'll have that pain, but I expect it will just be a matter of adding an "--enable-language rust" to the GCC configure script run, similar to adding C++, ObjC, etc. to a GCC build. So the cross-compiler build will take longer, but won't involve extra steps or infrastructure.
Binutils/gcc cross-compiler can be built rather quickly, once you know the right configure flags for each. Then it's just make && make install. You can speed the buil considerably by building with -g0 and not doing a bootrstrap or building needless targets.
Then you have a cross-compiler in some --prefix that you can point the kernel build system to via some environment variables and you're done. It's a well known process.
It's basically built just like any other dependency and doesn't take much time.
Hardly a nightmare. If it would be possible to just add rust to the list of gcc languages, that would indeed be very easy.
Cross compilation is actually much easier with Rust than it is with gcc. Every Rust compiler is a cross-compiler for all architectures that Rust supports; all you need is the rest of the toolchain like linker and libraries. rustup handles installing the libraries, and as lld (the LLVM linker) adds support for more platforms it can also be used as a cross linker, rather than a linker per target as in the older toolchains.
what if your driver or user space app is affected by a poorly architected driver which was written in rust? now you need to deal with that rust source code. a true nightmare.
Why would that be worse than if it was written in C? A rust codebase would be much easier to find the bugs in and refactor accordingly, than in C.
Plus it's not like most people are going to dive into driver code without experience in the driver itself. So it's a niche set of folks this would affect, the majority of whom would likely be writing the rust code anyway since they're likely the ones most familiar with the driver aspects.
People have said the same thing about every language. I'm positive people complained about using C over assembly (people still do for certain things).
I ask you to imagine a situation where a kernel is written in assembly, and they decide to start allowing C to be used for drivers. How would you view a change such as that?
I disagree. You can learn the semantics of C in a day, assembly requires a ton more memorization and a ton more code to write. C is more abstract that assembly but it is less complicated, on its face, than assembly or rust.
If you're familiar enough with programming to learn C in a day, you can learn assembly in a couple hours. It's really not that complicated. You can copy to/from memory addresses to local temp variable containers that are various sizes of bits (registers), you can do various math operations on those bits in those registers, you can compare those registers and test that comparison, and you can jump from the current point in the code to any other point by label or address (go-to). Sometimes operations will result in other registers you can check to see if something happened (overflow). The OS will provide some interrupts to do some base things as interrupts such as IO.
If you didn't know how to code in assembly already, you now know enough to do everything C can do with a simple reference sheet. All the extra boilerplate (such as program segments and where to store costants such as what the 'strings' program sees in a binary) you can read two to three pages to explain what you need for whatever OS you're on.
Only if you know C but not Rust. I personally find Rust fairly easy to follow and C a complete nightmare (the details in C are ok, but it's really hard to see the forest for the trees).
I know C and can get by in Rust and I disagree that rust is easier to follow. C is a simple language, the complexity comes from large systems, while Rust is just not.
Well I know Rust and I can get by in C. Perhaps that makes a difference.
> C is a simple language, the complexity comes from large systems, while Rust is just not.
I mean I agree that C is simple if you're making a 50 line program that writes to stdout or something. But more often than not with C or Rust you are working with a large complex system. And I'd argue that when working with such a large system, Rust programs are less complex than C programs.
There is a huge difference between reading and writing C. C is very easy to read, but it's quite difficult to write (secure, portable, performant) C. Code is typically read a lot more than it is written. I haven't written a single line of code in torvalds/linux.git but read probably tens of thousands of lines of Linux kernel code over the years. I'd wager that for every kernel developer there are at least 10 people reading the code to figure out how something works, or how it broke.
C makes many questions when reading trivial to answer, questions like "how do I get here?", "where does it go?", "what is this type?" and "what does this code do?" are generally very easy to answer without expert-level C knowledge. From what I've seen, that's not true for rust.
> with abstractions that don't require interaction with the rest of the system
I have no idea what this is supposed to mean. Drivers obviously interact with each other and with the rest of the system via all kinds of in-kernel interfaces.
Drivers are also not inherently system dependent. Even the sample rust driver is for android binder, which is not a real HW device, but some kind of IPC mechanism. So that one is quite system independent.
Most drivers don't interact with other drivers, only with kernel interfaces. A driver for a PCIe device has to interact with the PCI subsystem, but that subsystem isn't a driver in the traditional sense.
Consider this: Does your GPU need to talk to your mouse or your real-time clock? Or does your sound chipset need to talk to your HDD? The answer to both should be no.
Even a file system driver does not need to know about the underlying storage driver, since it operates on a block device abstraction.
Of course, some kernel modules have dependencies on other kernel modules, but those dependencies generally don't cross subsystem boundaries.
(There are of course exceptions to the examples I listed, but the argument should apply to the majority of device drivers.)
I guess I have to deal with wrong drivers then, but many of the drivers I had pleasure working with cross many boundaries. For example look at rockchip type-c port stuff I've been looking into last month:
There's a phy driver for USB, phy driver for Type-C phy/mux, display port CDN driver, type-c TCPM controller driver, generic alternate mode DP driver, charger power supply driver, and more, and all these have to communicate together (and do so over extcon and type-c mux/role switch/orientation swtich interfaces)
USB 2.0 phy driver detects what kind of device is connected using BC1.2 specification (what kind of charger basically), TCPM driver does the same over USB-PD, they race together basically trying to identify how much current the device will be able to draw from the charger.
Because they are independent on HW and driver level, one of them takes it upon itself to get the information from the other, and decide what values will get more priority, and passes that over another interface to a charger driver, to make it configure the HW to not overload the power supply connected to the Type C port.
This not a driver->subsystem but basically a driver<->driver interaction.
There's a ton more communication going on between these 7 or so drivers. TCPM figures out the peer device supports Alt-display port mode, so it will tell the type-c phy to reconfigure itself for 2 SS lanes and 2 alt-dp lanes modes to support parallel use of USB and display output. Displayport CDN driver orchestrates this, but there's a problem that link training is needed for this to work well, and that needs coordination between CDN-DP and Type-C PHY drivers, because there's no standard kernel interface for it.
So, yes I see a ton of inter-driver interaction. Many of it kinda ad-hoc, over some not perfectly well matching bus like interface like extcon or direct calls to other drivers, in addition to more common interfaces at the surface of various subsystems like clock, regulator, i2c, etc.
Maybe in x86 land this is saner, but in ARM SoC land, the drivers are often not that independent. Things need to happen across drivers in right order with right timing.
Not sure if that prevents these drivers to be implemented in rust, likely not, but the api surface is not small or simple, which was what I was trying to point out.
Likely these more tightly coupled drivers won't be the first target for Rust ports, but it's worth noting that Rust <-> C FFI is well supported and not as painful as you might expect.
"Does my GPU need to talk to my mouse"? That is definitely one of those "it depends on the system". On at least one DECStation model, sort of (depending on how you define "GPU"), the mouse interface is actually on the graphics card, and is read by the CPU talking to the graphics hardware.
When they're running the DEC-provided X11 server, the mouse pointer moving on screen is done entirely within the graphics subsystem, with no need to talk to the kernel. Which leads to the hilarity of wiggling the mouse actually not being a good diagnostic for "has the kernel hung itself", as the mouse pointer will wiggle (but clicking doing nothing, and usually not changing shape as you'd expect as it goes over fields where you'd expect that).
> I have no idea what this is supposed to mean. Drivers obviously interact with each other and with the rest of the system via all kinds of in-kernel interfaces.
But the total footprint of the interfaces that typical drivers use is fairly small. When a rusty shim is provided that provides access to those interfaces, you can write pure rust drivers.
Typical device drivers with limited interface footprint also barely do anything interesting.
For example look at typical audio codec drivers. They barely have any procedural code (and when they do it's just some form of register value mapping anyway), locking/concurency and almost all resource management is handled at the higher level and the driver is not exposed to it.
Another kind of a driver that features a very limited dependency on Linux interfaces is a SoC clock/reset driver. And clock drivers are even more boring in this aspect. Code vs data ratio is probably < 0.1 in many of those.
I guess some drivers that do more complicated stuff, and deal with concurrency issues or more complicated resource management may benefit, but they'll likely need to touch much larger API surface, too.
And I dunno, even the sample driver in the patchset has things like this:
let mut data = kernel::new_device_data!(
gpio::RegistrationWithIrqChip::new(),
PL061Resources {
// SAFETY: This device doesn't support DMA.
base: unsafe { IoMem::try_new(res)? },
parent_irq: irq,
},
// SAFETY: We call `spinlock_init` below.
unsafe { SpinLock::new(PL061Data::default()) },
"PL061::Registrations"
)?;
// SAFETY: General part of the data is pinned when `data` is.
let gen = unsafe { data.as_mut().map_unchecked_mut(|d| &mut **d) };
kernel::spinlock_init!(gen, "PL061::General");
let data = Ref::<DeviceData>::from(data);
It's barely readable. I don't know what type `data` is, what `gen` is or how it's different from `data`. Variable names don't really help. `data.as_mut().map_unchecked_mut(|d| &mut *d)` also looks like some kind of WTF to the uninitiated, that is somehow retrieving some 'general part of data' whatever that is.
And what's with the two `let data`. Are they the same type or is the type now different and the first data variable now hidden? Because it looks like second `let data` gets somehow transformed from the first `data` into something else. So now we have two identically named `data` variables with different types in the same function?
Wasn't this supposed to be some simpler language to draw more people to the kernel development? At least with C each variable has a type that's quickly identifiable.
I guess after 15 minutes I see what this is trying to do. It passes some newly created spinlock object to `new_device_data` function and then calls `spinlock_init` on some part of the data returned from `new_device_data` that needs to be extracted from it via some really obtuse transformation, just to initialize the spinlock later instead of directly in `SpinLock::new` for some obscure reason, using two unsafe {} blocks in the process. But oh my god, if this kind of obfuscation is going to be the new standard for the kernel,... At least C is the devil I know. :)
Most Rust is written with the implicit assumption that it will be read in an IDE that displays types, so explicit type annotations are usually kept to a minimum. Doesn't always translate well outside of the IDE, so I wouldn't be surprised if the consensus for kernel code shifts to more annotations.
> When I look at the kernel's spinlock type [...] that doesn't really help me determine why whatever this `gen` [variable?] is needs to be passed to `spinlock_init`
That's pretty ideomatic in rust, probably borrowed from functional languages. The new definition hides the previous one, and the implication behind reusing the name is that it's fundamentally the same data, just with a different type, or different mutability (here it was converted from mutable DeviceData to an immutable Ref with the same DeviceData).
This is probably the biggest issue with Rust in the kernel; because it needs to deal with a lot of existing kernel interfaces and abstractions, you get code which is neither very idiomatic Rust, nor is terribly recognizable to someone who knows the kernel abstractions fairly well.
Now, some of that is just a matter of learning curve; once you have learned the Rust-in-Linux abstractions the above will probably be somewhat more readable.
Coming from someone who's more familiar with Rust, and less so with the kernel, here are a few notes.
* It might not be the best idea to post the example drivers with inferred types, for this familiarization reason. While there are a few types (such as those involving closures) that you can't write and can only use via type inference, or others that are too large to want to write out, these types should be fairly easy to write and it would improve readability during the familiarization process
* A lot of these things that you were having trouble with look like they are basically just dereferencing and accessing some data contained within the spinlock, and casting what is only guaranteed by the API to be an immutable (shared) reference into a mutable (unique) reference in an `unsafe` block. This is a place where trying to use simple Rust wrappers over core kernel structures which weren't designed for Rust makes things a bit more difficult to read, as you need to do a bit of an unidiomatic dance to tell the compiler "yes, the language and type don't gurantee uniqueness, but I promise this reference is unique and I won't misuse it."
* One of the small little humps to get over when learning Rust is to learn that "mut" actually means "unique", and unmarked (immutable) references means "potentially shared." But since you do need to sometimes write to objects that are potentially shared, it just means that you need to either guarantee yourself or use some kind of synchronization mechanism to ensure that at any given moment of time that data is only mutable in one place at a time.
So, I don't know the details of these APIs myself, but when I look at this here's what I see
* we're constructing some device data object protected by a spinlock
* we're providing some brief justification for why this particular usage is safe, since the API and compiler can't guarantee that it's safe
* we're extracting a mutable reference from it, for which we need to uphold the single exclusive access principle without assistance from the compiler
* we're wrapping that in a new reference to device data type which presumably provides a more idiomatic interface than what the raw primitives provide
I'm guessing that the following code which actually uses this Ref::<DeviceData> object will then be a bit more readable and require fewer uses of unsafe, as it looks like this code is what is providing that translation between some core kernel primitives which are not terribly idiomatic to use in Rust and a safer, more idiomatic wrapper.
No no no, I know that it is not idiomatic code, it says in the URL: "transpiled". I am just telling him to practice using it. I was not judging Rust at all.
That said, idiomatic Rust is difficult for me to read/understand as well. I do not expect device drivers in Rust to be readable (by me), and if they have lots of unsafe blocks then we are back to square one I would say.
I wouldn't call that code idiomatic either: it is cleaned up from the automatic translation, but it still a far cry from what Rust written by hand would look like
but it still looks weird to me (a static mut would be uncommon in a design, and not having a new-type to represent these would also be uncommon).
If you are interested in looking what Rust written from the ground up in the same domain looks like, you can maybe look at Bevy[1].
The biggest source of "noise" in Rust is explicit type conversions, FP constructs and longer types (`.into()`/`.as_mut_ref()`/`.map()`/`Result<Option<T>, E>`).
So, what we're saying here is in effect, "and all the other fields should have their default value".
Specifically, when you initialise a structure like this, the .. is "struct update syntax" and is followed by an instance of that structure from which remaining fields will be copied. Default is a Rust trait (similar to an interface if you've never seen Rust traits before) and it has the method default() which gives you a default of whatever type.
Rust has type inference, if you don't specify a type, but there is only one possibility that could work, the type inference might (and in this case does) see that and you needn't bother laboriously writing down the type. This can only happen where there was no possible ambiguity. In this case, the only type you can use for struct update syntax is the structure type you're updating of course, BufferInfo, and so Default::default() must mean the default BufferInfo.
If you found the definition of BufferInfo it probably says #[derive(Default)] which means, using a macro just implement Default for this structure in the obvious way, although it might actually have an explicit Default written out instead.
If every field of a struct impl Default, then you can you can use the `#[derive(Default)]` annotation to auto-fill that impl for that struct, otherwise you have to write it by hand.
The .. is the spread operator, and it takes every field of the value on the right and applies it to every field of the struct constructor on the left. This only works for the same type (it's not only looking at the field names).
`Default::default()` is a qualified call on the trait equivalent to `<_ as Default>::default()`, where `_` is inferred from context to be `BufferInfo`.
What that code is doing is saying "build a default BufferInfo with field buffer_usage set to VERTEX".
The spread operator has a limitation you might have intuited from the description that it can't work for types where some field(s) should always be set but not others. To get around that you can use the builder pattern
Normally in Rust, to initialize a struct, you must specify a value for every field of it, since null and uninitialized values aren't allowed. To make it a little more convenient, there's a trait called Default which can be implemented for your struct to provide an instance with every field set to a value that you think should be the default. And the .. syntax allows you to copy values for the rest of the fields of a struct from another struct instance. So calling that with Default::default() gets an instance with the default values and uses those values to fill all of the rest of the fields of the struct. Thus if you have a big struct where most values are the default most of the time, you can create instances of it with one or two fields set to something different like this without an extra few dozen lines of repetitive boilerplate for every other field of the struct.
Oh, I thought that the comment about not looking as idiomatic code was referring to the kernel::new_device_... sample in a parent/parent HN comment. I was more interested in knowing what was so unidiomatic about that.
The posted code isn't terrible, but it is a bit denser and harder to read than I'd like. Some small changes (like importing the macros to avoid a full path) makes a bit easier to read
// SAFETY: This device doesn't support DMA.
let base = unsafe { IoMem::try_new(res)? };
let mut data = new_device_data!(
RegistrationWithIrqChip::new(),
PL061Resources { base, parent_irq: irq },
// SAFETY: We call `spinlock_init` below.
unsafe { SpinLock::new(PL061Data::default()) },
"PL061::Registrations"
)?;
// SAFETY: General part of the data is pinned when `data` is.
let gen = unsafe { data.as_mut().map_unchecked_mut(|d| &mut **d) };
spinlock_init!(gen, "PL061::General");
let data: Ref<DeviceData> = data.into();
I imagine code like this would be the body of a small function that does all this unsafe preparations and gives back a Result<Ref<DeviceData>, Error> once it's done, which would change the end to be
OTOH, it may cause companies to invest in creating a well maintained Rust frontend for gcc. People have already done much of the grunt work for this, so it seems like it's just a matter of manpower, and something that acceptance into the edges of the kernel might cause more investment in.
I'd be very curious if anyone knows what the Rust team's ambitions are for supporting the full set of Linux-supported platforms? Is this something they aspire toward? If so, how are they approaching it? Working with LLVM folks? Other backends (e.g., gcc)?
Not on the relevant team, but I can tell you what I've observed from the outside:
There's no specific goal to support every platform Linux supports. We have four tiers of support: 1-3, and then not supported in-tree. The project itself only moves things forward on a given platform if there are people championing that platform's support. If folks want to get Rust in more than just drivers and into the kernel proper, and platform support is an issue, then it's up to them to drive the platform support. For example, aarch64-unknown-linux-gnu only recently gained Tier 1 support, thanks to Arm themselves driving the effort.
The rustc compiler uses LLVM, so whatever new architectures LLVM supports, rustc also gets. There are two projects going on right now - rustc_codegen_gcc which adds GCC as a rustc backend, and gcc-rs which adds a rust frontend to GCC. Both have seen considerable progress and support over the past year and are probably the most promising avenue to expanding the architectures that rust can compile to.
Will be interesting to see how much compile times suffer once rust starts being used more heavily in the kernel.
In my gentoo days I remember the compile time for firefox suddenly going up from 20 minutes to several hours which left a sour taste in my mouth with regards to rust.
Has there been much improvement in compile times in the past ~2 years?
Roughly dropped by about 30% or more? Here's a dashboard that tracks times of certain benchmark tasks by compiler version: https://perf.rust-lang.org/dashboard.html
Additionally, I would expect that the kernel would mostly avoid things that drive up compile time - generics and macros.
And as someone else mentioned, things should be able to be compiled in parallel.
Also, some crates for firefox are absurdly huge in terms of compile times - they are definitely outliers (IIUC). (Edit: I was thinking of this issue in Servo, so not definitely firefox releated: https://github.com/servo/servo/issues/1799)
That page averages all the builds across different code bases. It doesn’t specify which version/tag of which code base, nor does it talk about the hardware.
https://arewefastyet.pages.dev/ - This page tracks compile times across some common crates over all supported compiler versions, with different hardware (2, 4, 8, 16 cores). This used to be https://arewefastyet.rs but the domain expired.
The tldr - speed ups of between 25-30% in these crates over the last 3 years.
And specifically proc-macros that parse rust-like syntax. It's the `syn` library that's slow. Proc macros that have a custom parser tend to be quite fast.
Interesting, I always assumed that syn was fast enough to not be a bottleneck. Do you have some related benchmarks, eg comparing it to the rustc and rust-analyzer parsers?
This is more nuanced, as there are three separate components:
* time to compile the syn itself
* time to run syn on the inputs
* time to compile whatever syn generated
I didn’t do a super thorough studies of things, but my impression is that 2, performance of syn itself, is rarely an issue. Most of the time it is 1) (and the associated problem of decreased build parallelism because half of the crates wait for syn to compile) and 3).
I think that jump was due to lto (link time optimsation) getting activated by default for Firefox on Gentoo around that time. It looked very strange at first, because the build log just pauses while the lto passes hog a lot of cpu and memory for 20min or so. Rust has been in Firefox for about 5 years now, I think.
In my experience, compile times are more a function of project structure than anything else. The compilation unit is a crate, so keep your crates nice and small, and you'll be fine.
I think rust will also have an effect on C drivers. People will rewrite their code to match safety levels. Even without rust based driver on your system you might benefit without compilation time penalty.
Too bad that only x86-64 output is supported out of the box. There's no real reason for this, other targets work just fine but you need to place e.g. `--target=riscv64gc-unknown-linux-gnu` or even `--target=wasm32-wasi` in the compiler options input. C/C++ doesn't have this issue, it's specific to Rust.
It is the "first language" that Linus didn't object to. He hated both C++ and Ada. I am not entirely sure if he really likes rust, sees borrow checker as something important worth trying, or lost to the pressure from RESF ( Rust Evangelism Strike Force ) within the linux kernel dev community.
He is a proponent of C because he can imagine, or envision the generated assembly from C compiler. Something like a High Level of assembly. And something along the line of people who can write good C code tends to have all the scar tissue to prove it.
Would you happen to have some links about Linus hating Ada? While his stance on C++ is well-known, this is the first time that I hear he has public opinions about Ada. Thank you
>What do you think of the projects currently underway to develop OS kernels in languages like Rust (touted for having built-in safeties that C does not)?
>That's not a new phenomenon at all. We've had the system people who used Modula-2 or Ada, and I have to say Rust looks a lot better than either of those two disasters.
>I'm not convinced about Rust for an OS kernel (there's a lot more to system programming than the kernel, though), but at the same time there is no question that C has a lot of limitations.
And I asked him once I think in a private email a few years back and he manage to gave me a long detailed reply that was like an essay ( Still very very grateful for his time ). May be he doesn't hate Ada as he did to C++, but it certainly feels he doesn't like it.
I would be interested to see his opinion on Zig, which I think really fits his liking. But may be after 1.0, which is still some way off.
The article is outdated: it is pre-C++11. For example, constexpr would be valuable for kernel and is unavailable in C. Maybe "STL is stable and portable" wasn't entirely true in 2007, but it is true in 2021.
I think that his comment about "inefficient abstracted programming models" is probably the more pertinent one here.
One important factor to consider there, and for which Linux is wholly unlike the vast majority of other projects, even system projects, is that the userbase is just massive, and includes a lot of the world's infrastructure. That results in a rather different cost/benefit calculus around abstraction/performance tradeoffs than the ones for which object-oriented and functional programming were designed.
For example, I think that it's probably good for all of us that there's not really a whole lot of dynamic dispatch happening in the Linux kernel. I would generally prefer that my syscalls keep pointer chasing to an absolute minimum. But I'm hard pressed to imagine how a C++ style that avoids vtables like the plague could really have that much to offer over straight C. And it would simultaneously put extra code review pressure on the kernel team, since they would have to carefully watch to make sure contributors aren't introducing such things.
Similar concerns apply to Rust, mind. But it's much easier for me to see how Rust without trait objects could still bring something of value to the table.
> But I'm hard pressed to imagine how a C++ style that avoids vtables like the plague could really have that much to offer over straight C.
unique_ptr / shared_ptr alone would be a massive upgrade over straight C. Simply being able to enforce ownership would be huge. So would just having namespaces. Things like move semantics, static_assert, and constexprs would then be the cherry on top. There's also public/private fields, friend classes, etc... to keep people from accidentally poking things they shouldn't be.
C++ is so very much not just "C with vtables."
But C++ has a reputation, deserved or not, for being overly complex. And ultimately Linus chose the devil he knew over something that by all accounts would have been better. Nearly every other large C project has moved into C++, even from other staunch defenders of C like John Carmack eventually relented that C++ was better for large teams.
Rust gets to hit the reset button here with a fresh reputation (and some extra safety & cool tricks), and Linus' views may have changed over time.
C++ has a bunch of principles that Linus disagrees with. Actually some important C++ people aren't sold on them either, but too bad, they are principles of C++ and so you're stuck with them in C++.
Examples include exceptions as the one true error mechanism, and the use of implicit allocation.
Rust doesn't have the former, and Rust's core doesn't have the latter (but std does), one of the stipulations from Linus when this began was that Rust for Linux needed to ensure it didn't sneak in implicit allocation. Thus, while a beginner's Rust program might have text += "Some more stuff"; you won't see that in the kernel -- because it won't compile, the core::ops::AddAssign isn't implemented on these string types in the kernel because it's implicit allocation.
As to move, the move semantics in C++ are really awkward. They're a nice optimisation of course, which is why they're provided, but you would not deliberately do it that way in a language which had move semantics from the outset, as demonstrated by Rust. To make move work with existing C++ software it has to "hollow out" objects when they're moved from. Rust needn't do that, if you move a Foozle from var1 to var2 in C++, it must carefully hollow out the Foozle in var1, because in C++ var1 will eventually get destroyed, so its contents must be valid and safe to destroy. If you move the same Foozle from var1 to var2 in Rust, the Rust compiler knows you moved it, so, it's gone, var1 won't get destroyed and no further attention to it is needed.
> Examples include exceptions as the one true error mechanism, and the use of implicit allocation.
-fno-exceptions is very widely used and non-throwing variants of basically everything exist as a result. That's not a reason to avoid C++ in the kernel as a result. Especially with how heavily Linux already relies on GCC C extensions, there's obviously no hangups on being "strictly standard compliant." Not have exceptions ever been C++'s "one true error" mechanism ever. For example see iostream: https://www.cplusplus.com/reference/ios/ios/rdstate/
C++ is not, and never has been, anywhere near that dogmatic.
As for the rest of your points, you seem to be arguing Rust is better than C++. But that's not really relevant to why C++ was rejected before Rust even existed. I do think Rust is even better than C++, but I also think C++ is a huge improvement over C, especially for large codebases.
It's true that despite the fact exceptions are the one true C++ error handling mechanism in fact lots of C++ programmers religiously never use them. That ought, one would think, to set alarm bells ringing. And it does for the C++ Committee. Language schism is bad news.
However whereas you can write -fno-exceptions and get a C++ with this core language feature excised, you can't do the same for implicit allocation.
In contrast the Rust for Linux team have worked hard to produce a Rust for Linux environment that doesn't panic!() and has no implicit allocation, since these are red lines for Linus.
> It's true that despite the fact exceptions are the one true C++ error handling mechanism in fact lots of C++ programmers religiously never use them. That ought, one would think, to set alarm bells ringing. And it does for the C++ Committee. Language schism is bad news.
You're still trying to force C++ to be dogmatic. It never has been. It's always tried to be accommodating to all types of usages. It's always been multiparadigm. 'new' wasn't even defined to throw std::bad_alloc until C++98, which also introduced 'std::nothrow_t' to have it return null instead.
> you can't do the same for implicit allocation
Just like with exceptions this really isn't true, either. There's no implicit allocations in the core language, only in the STL. Saying that the "new" operator does implicit allocation is like saying "malloc" does implicit allocation. Regardless, placement new has always existed, and the non-placement 'new' operator has always been overridable. And all of the STL is optional & replaceable.
Being freely able to do X or not do X is "multiparadigm". But being told you must choose to either do X, or not do X, and either way obviously you lose access to all the software that expects the opposite, also by the way there are several more choices like this to make - is not "multiparadigm" it's schismatic.
C++ has become lots of little silos of people using dialects that are in practice mutually unintelligible with other C++ dialects. Hence as I said the concern of the committee about this problem. What's the purpose of a "standard" which doesn't standardise anything?
But to be sure, Linus could create yet another silo.
And for implicit allocation, like I said, Rust for Linux did the work. C++ practitioners didn't. Linus isn't going to take something that doesn't exist just because its fans say if it did exist it would be suitable.
There's no equivalent of Rust's core library that's in C++ though. So if you don't use the C++ STL you need to rebuild an entire library for all basic types. You also can't declare that your library doesn't need the STL.
> But I'm hard pressed to imagine how a C++ style that avoids vtables like the plague could really have that much to offer over straight C.
I don't follow this argument. In my experience, use of vtables is relatively rare in modern C++ development even when you aren't trying to avoid them, many of which are elided by the compiler anyway. Regardless, in these infrequent cases it is straightforward to code equivalent functionality without vtables, since it is syntactic type-safe sugar over the code you'd have to write in C anyway. They are like C bitfields in that every systems programmer knows how to implement the equivalent functionality manually for cases when the language feature is underspecified for the purpose.
Modern C++ without vtables is only marginally different than modern C++ with vtables, the code you'd write and the features you'd use would be nearly unchanged.
In fact, most of the C++ standard library doesn't have virtual methods. It's also not a coincidence that the areas that feature them most prominently (iostreams and locales) are widely considered to be very dated designs that weren't particularly good to begin with, and didn't age well.
> For example, I think that it's probably good for all of us that there's not really a whole lot of dynamic dispatch happening in the Linux kernel.
Between this and the other comment where you say Linus dislikes OOP I kinda get the feeling you ought to read some Linux kernel code. Because it's practically completely object-oriented and almost everything is done through a vtable for dynamic dispatch.
The linux kernel is crawling with dynamic dispatch, but because it is in C they are the most inefficient dynamic dispatch mechanisms possible, with arrays of function pointers populated at runtime that 1) the compiler can never optimize, because they aren't visible at build time, and 2) that have been the causes of numerous vulnerabilities, because it's not always obvious at commit/review time that the author has screwed up the assignment of function pointers to dispatch slots.
C++ would be a huge win all over the kernel, the only barrier to it is Linus' ignorance.
C++ has loads of inefficient abstracted programming models but rust doesn't?
C++ has moved on from the days when that argument was made. This may have sufficed for an argument against C++ then but it should also apply to rust today which has similar infrastructure like generics and traits that allows a level of abstract programming that C doesn't provide.
It’s truly the only low-level language that could actually be used for systems programming.
C#, Java? Managed code with JITs, not supported on a kernel level no-way no-how. Ditto for Python, PHP, JavaScript, and other high-level languages without pointers and all that jazz.
Go would be close… but it requires bundling a bunch of libraries in with each binary and wasn’t designed for low-level development. Again unsuitable.
C++? Linus Torvalds and the kernel developers hate it for being unnecessarily complicated and only making things harder to maintain while bringing little to the table at this point (they’ve made it this far without classes, and C++ doesn’t bring anything new like memory safety).
> Go would be close… but it requires bundling a bunch of libraries in with each binary and wasn’t designed for low-level development. Again unsuitable.
Go has a garbage collector and doesn't allow for manual memory management which would rule it out entirely. This adds a requirement for the go runtime, which is a further nonstarter.
> It’s truly the only low-level language that could actually be used for systems programming.
No it is not. There is Ada/SPARK and has been for a long time, and it is for critical systems. I recommend checking out https://blog.adacore.com/ for examples, there is plenty of that. For one: "An Embedded USB Device stack in Ada". There are also many articles about SPARK. For example: "Enhancing the Security of a TCP Stack with SPARK". These are just one of the recent articles.
The person you're responding to is wrong (there have been lots of languages before and after Rust that have been used for systems programming), but there are also lots of legitimate reasons why Ada was never considered a serious contender: GCC's mainline support (via GNAT) still didn't fully stabilize until the late 2000s, and GNAT (AFAIK) remains the only unencumbered implementation that's broadly available. Rust has the same single-implementation issue, but is actively working on different codegen backends to resolve it.
For better or worse, the ship has probably sailed on Ada/SPARK making its way into mainline Linux.
I know. That is one of the major issues: lack of hype. Most Rust enthusiasts have not even heard of the language, or they did and they have misconceptions about it. If they truly care about safety, they should definitely check Ada/SPARK out.
Ada/SPARK allows formal verification of many more properties than just memory safety. Ada on its own does not have the same kind of memory safety guarantees as rust though.
I was asking the opposite question, I think. I know of GNAT, but I was asking whether there's any Ada/Spark that has a GPL (or whatever non-proprietary) license.
It's long been done fairly successfully in business domains like aerospace and firmware development.
Where I'm less convinced that it's an option with a high likelihood of success is in open source. It's one thing when you've got a $250,000 salary riding on you ability to scrupulously follow best practices. But results may vary when the work is being done on an amateur basis.
If by successful you mean where changing one line of code takes 2 weeks of change request, 2 weeks of requirement analysis, 2 weeks of system specification translation, 2 weeks of test specification, 2 weeks of test case implementation, 1 day of implementation and finally 2 weeks of code review, and so on. Then yes, you are completely right.
In aerospace you probably want all this anyway, but for others, where 99.99% is enough, there needs to be a cheaper and more efficient way.
I'm doing embedded development day and night, so am aware how prolific C is. That however does not make it memory safe. 'Best practices' really means using humans as compiler preprocessors and hedges on humans being infallible.
>you've got a $250,000 salary riding on you ability to scrupulously follow best practices
Sounds like a great way to have a toxic workplace that either yells at people for bugs, or never gets anything done because people are paranoid about committing bugs (or both!). Meanwhile, with a different language you get the compiler yelling at you instead. Automated enforcement uber alles.
I don't think open source vs commercial dev really matters that much.
> But results may vary when the work is being done on an amateur basis.
Much, if not most, of Linux kernel development is done by employees of various companies, who are being paid for their work. Typically, that's in the form of device drivers, which also happens to be where I've seen the shoddiest, buggiest code in the kernel [1].
[1] Think "driver A abusing the 'private' pointer of driver B for its own purposes" kind of buggy.
Rust also relies on best practices. It is best practice not to use `unsafe` blocks unless absolutely needed. That does not mean Rust is an unsafe language.
1. The corollary is that code that does not contain unsafe code is guaranteed to not contain memory safety bugs, assuming that you trust and/or have audited unsafe usage in your dependencies.
2. unsafe blocks are explicit and therefore visible during code review. Memory safety violations in C++ are not always visible, simply because they are implicit.
3. Thanks to (2), unsafe usage in a codebase can be easily quantified and, if needed, audited.
> 1. The corollary is that code that does not contain unsafe code is guaranteed to not contain memory safety bugs, assuming that you trust and/or have audited unsafe usage in your dependencies.
Doesn't this assume that there are no bugs in the Rust compiler (for "safe" code)? Isn't that still somewhat of a big assumption at this point? I get that is indeed a very core goal of Rust, but has this been proven for any/all released versions of the Rust compilers?
It ultimately depends on how you define your TCB (trusted computing base). I am not aware of any formally verified compilers for any language, but assuming one exists, wouldn't you also require said compiler to run on a formally verified OS? Also, how do you tackle the infamous "Reflections on Trusting Trust" compiler attack? It's turtles all the way down :)
Long story short, for the vast majority of use-cases, developers can assume that the compiler and stdlib are trusted. This is also how Rust defines "safety", which leaves you to focus on your code and its dependencies.
Not talking about situations where the compiler or the OS are compromised - of course all bets are off in those situations.
But "guaranteed" that there can be no security holes in code marked "safe" due to an inadvertent compiler bug? I would be very surprised if that is actually the case.
Less likely than C/C++? Yes of course. Guaranteed? Probably not.
An inadvertent bug in the compiler can be modeled as a compromise. So, as before, it depends on how you define “safe” in your model. In the case of Rust, the compiler is safe and is considered to be within the safety boundary.
Will the Rust compiler have bugs in practice? Of course. Do you - as a developer - have a different safety model than the Rust language (i.e., one that does not include the compiler)? If the answer is no, you can “ignore” compiler bugs from a safety perspective.
Firstly, the difference I was trying to emphasize is "guaranteed to be visible due to the unsafe keyword" vs. "potentially not visible".
Secondly, quality standards rely on the human element, which we know is fallible and never 100% reliable. Compiler-enforced memory safety checks are (in theory) infallible[1].
[1]: Infallible when all of your dependencies are also verifiably memory safe. In reality, this is not the case, but I believe that Rust can reliably get much closer to 100% than usual.
Ensuring memory safety in C++ is an endless game of spinning plates. It's possible, but the "best practices" for doing so certainly don't feel like best practices. I've watched people repeatedly compile a program, changing a single line and running it through Valgrind for hours on end. With Rust, you actively have to fight the compiler to cause memory leaks.
Addendum: yes, that last line is what pisses a lot of people off with Rust, and honestly, rightfully so. I've written quite a bit of Rust by now, and getting your code to work can feel like a bit of an escape room. Having C as an option on the kernel alongside it is a great idea though, because there are situations where I'd happily pick up C over Rust, and they're not just 'edge cases' where Rust falls short. Ideally, they're two different tools that can be applied depending on the situation at hand.
> With Rust, you actively have to fight the compiler to cause memory leaks.
Memory leaks are not considered unsafe in Rust[1]! They're actually an important part of Rust's lifetime semantics: intentionally leaking memory is both safe and equivalent to producing a `'static` out of thin air.
You probably shouldn't leak memory as a matter of practice, of course, but Rust will happily allow you to do so in 100% safe code.
If you correctly model ownership with RAII, which also enforces strong class invariants, and make sure all operations satisfy basic exception safety, then leaks are not possible.
Those are the most elementary rules that one must follow when writing C++.
Object graphs where there's no clear owner are not automatically "incorrect" - whatever they're modelling might be non-hierarchical itself - yet would result in a leak without a GC.
Graphs where no single object is a clear owner happen all the time in real world, so I would argue that any object model that can't accurately model that is just not good enough.
And what basic guarantees are not enforceable without ownership?
'best practices' for memory safety in C++, even modern C++, are still so convoluted as to not be reliably followable in practice, either by humans or humans assisted by static analysis.
Rust has semantics which are designed to allow those best practices to be watertight and enforced by the compiler. C++ has semantics which are designed to fit around many years of backwards compatibility with unsafe practices. Attempts to make static analysis for C++ which mirror rust's borrow checker have thus far failed. There is not an automatically enforcable set of rules for C++ in any version which give you the same safety guarantees at rust (and the non-automatically enforcable set of rules is not followable by humans in practice, since we keep getting memory safety bugs in C++ code).
In theory, C++ programs use std::optional, just as Rust programs use std::option. In practice, such C++ programs are extremely rare, whereas such Rust programs are majority.
There's an even clearer example here: dereferencing a C++ std::optional is undefined behaviour. Dereferencing a Rust std::Option is a compile time error.
In theory, in practice when using debug libraries, undefined becomes defined as a runtime check that will lead to an assertion error, or a static analyser failure.
They're saying that the Rust compiler will reliably enforce these things... and that it's completely trivial to surface any cases where the programmer thinks that they are smarter than the compiler, e.g. for extra code review, outright rejection, etc.
no, he's saying the C++ best practices are too convoluted, (implying the Rust ones aren't), when they're essentially the same, just enforced differently.
They might be referring to how easy it is to accidentally/unwittingly invoke the copy constructor in C++. `Foo x = Foo();` used to be the famous example of that, although more recent versions of C++ may guarantee that the copy is elided there.
Another good example is implicit comparison conversions, e.g. some class defines `operator==` as non-explicit and any code that does `x == y` where `typeof(x) != typeof(y)` may find itself going though a possibly-allocating constructor.
Temporaries may or may not be heap-allocated; the C++ spec doesn't specify.
But even beyond that: you can have a non-trivial constructor that requires heap allocations internally. So even relying on (sane) compiler-defined behavior won't save you in the general case.
Are there any C++ implementatons that actually heap-allocate temporaries? The only context where I can imagine that happening is an async (as in co_await) function, where all autos might need to be heap-allocated for obvious reasons.
But that aside, it just seems like a very weird choice that would have profound perf implications - to the point of removing pretty much the sole remaining advantage of the language - for no gain whatsoever, since it's not even simpler.
As for non-trivial ctors - at that point you might as well worry that every function call made in pure C code can also heap-allocate, no?
>C#, Java? Managed code with JITs, not supported on a kernel level no-way no-how. Ditto for Python, PHP, JavaScript, and other high-level languages without pointers and all that jazz.
C# has pointers and can be compiled to native code. I think the main problem is GC which is required for most of the standard library (you can avoid heap allocations but that would be unidiomatic)
>you can avoid heap allocations but that would be unidiomatic
AFAIK until recently (with the introduction of the Span<T> types), the only way to avoid heap allocations for arrays or any other kind of collections in C# was with stackalloc, which is unsafe.
You still have to use stackalloc to allocate on stack. With Span<T> you don’t have to use the unsafe keyword and you can pass the allocated array to other functions easily.
I don’t think GC would be a problem in a general case (there are OSs written in Java, so there is that), but yeah, it probably wouldn’t be a great fit for the already existing linux code base. Especially that one kinda important function of the kernel is to GC other resources either way :D
Free Pascal too. An example would be Ultibo[0] which, despite the completely awful site (in terms of learning what it is - it feels like a developer's cynical overjaded idea of how a modern site that tries to market something is supposed to look like), is an OS for Rasberry Pi written in Free Pascal. It is meant to be used as a base for purpose-built projects instead of a general purpose OS, but it does show that it is possible to make one using the language (it already supports pretty much all "standard" features like networking, threads, 2d/3d graphics, a few filesystems, etc).
Of course it doesn't make much sense to use it in the Linux kernel but it is a low level (or low level capable) programming language that can be used for systems programming.
As an historical note, Pascal was very popular in the late 70s and 80s for systems programming. The entire Apple Lisa/Macintosh platform started out in Pascal, for example. The limits of Pascal led Apple to team up with Wirth and develop Object Pascal, which is basically the language Free Pascal implements now.
The gap between Pascal with a couple extensions and C, or between Object Pascal and Objective-C, is very small, anyway. Most semantic elements have a one-to-one correspondence. There were even UNIX clones written in Pascal.
Very true. Many people don't realize how Pascal evolved into Object Pascal, and how it's such a strong alternative to C++, C#, Objective-C, Java, etc...
Insofar as they don't bundle a runtime (unlike the languages mentioned by GP), yes. But they don't really offer much above C in the area of resource tracking / safety that rust's novel ownership+lifetimes system provides.
Nobody really proposed these in earnest. Also, it’s a kernel, where diversity of programming languages is a nightmare. It’s not the kernel’s job to support every language people want - it’s just that Rust has enough going for it that it’s finally being considered worthwhile after years of investigation.
Zig isn’t substantially older than Rust, and is arguably less compelling. Certainly fewer resources backing an attempt at use as a 2nd kernel language.
D I think suffered from a proprietary compiler and wide use of GC. But I’m not really familiar with it.
Zig is actually quite a bit younger than Rust. The first release of Zig predates the 1.0 release of Rust by a few months, but that was also a pre-release version. The earliest versions of Rust predate Zig by around 4 years.
Yes. Zig started in 2016 Feb, Rust started in 2006 and pre-Alpha 0.1 release in January 2012. ( Needless to say the design Rust and borrow checker is way more complicated )
It's not the only one, but if we're going to finally start moving away from C we have to pick one. And Rust is trendy right now, so it has more support than all the other options. It's a good option, and in another 30 years, just as the last .c file gets replaced from the kernel, we can re-evaluate again...
IIRC C++ is dangerous because of how it does exceptions, and you might have exception-inducing code creep in unnoticed, esp. given that the standard library uses exceptions. I could be totally wrong about this.
You could (and would) disallow exceptions in a Linux kernel use of C++. This could be enforced with something almost as dumb as grep. Exception-less C++ is still perceived as having more cons than pros. You likely would not use the userspace stdlib — much like how Rust’s core/alloc crates are significantly limited here.
Eh? A lot of modern C++ development explicitly disables exceptions, they are unnecessary whether or not they should be used in practice. For more low-level systems-y code, there are good reasons to avoid them.
While there are arguments over when and how to use exceptions, I think this is the first time I've seen them positioned as a killer feature of C++.
> I think this is the first time I've seen them positioned as a killer feature of C++.
It's not a killer feature of C++, it's a requirement, without exceptions you can't have faillible ctors meaning your object lifecycle suddenly becomes a lot more complicated and full of additional traps.
> there are other ways to design programs, and to allocate memory.
I covered this option:
> your object lifecycle suddenly becomes a lot more complicated and full of additional traps.
because without ctors you don't have object invariants as you have to split instantiation and initialisation, or you have to implement bespoke and unusual strategies drawing you further and further away from anything which could be described as "standard" C++, thus making the advantages of C++ dwindle further into the distance. And all that with, as far as I know, no standard tooling to support you.
Model fallible instantiation with factory methods. e.g. Node::make(Args..) -> Result<Node>. It won't always be as efficient or ergonomic but it works. And hey it's c++, you've been conditioned to accept a ton of ceremony to get things done. Famously the bulk of google's code is written without exceptions. And from what I know of it (not much), factory methods are how they model instantiation fallibility.
As a related but separate thought, sometimes adding invalid states to an object is useful. For example, take a look at std::thread::detach. Sometimes you see rvalue qualifiers on such methods (indicating the object has been consumed).
If you consider memory allocation to be infallible, there is a lot of stuff that can be written as pretty normal c++ without exceptions. The STL even pretty much works without throwing given this. If you consider allocation to be fallible you're going to have a bad time with "standard" c++ too.
It's widely accepted by the C++ community that Google's C++ standards are bad. They're more concerned about using a simple subset of the language anyone with minimal C++ training can understand than about writing good C++.
And detaching a thread is pretty much always a bad idea.
> Model fallible instantiation with factory methods. e.g. Node::make(Args..) -> Result<Node>. It won't always be as efficient or ergonomic but it works. And hey it's c++, you've been conditioned to accept a ton of ceremony to get things done. Famously the bulk of google's code is written without exceptions. And from what I know of it (not much), factory methods are how they model instantiation fallibility.
The problem is not the ability to do it but the ability to enforce it. "Just be careful" has not worked, and will keep not working. Google's coding standard has not stopped C++ from causing them problems.
> As a related but separate thought, sometimes adding invalid states to an object is useful. For example, take a look at std::thread::detach. Sometimes you see rvalue qualifiers on such methods (indicating the object has been consumed).
You know what's actually useful and good?
For the object to actually be consumed and not be accessible at all anymore.
> If you consider memory allocation to be infallible
Which you can't in the kernel.
> If you consider allocation to be fallible you're going to have a bad time with "standard" c++ too.
I write ultra high-throughput database kernels. We use just about every feature of C++17 and beyond, particularly since so much of the implementation is generated via metaprogramming scaffolds. I don’t use exceptions at all, mostly because it is a poor abstraction for this type of code. Exceptions aren’t all upside.
Clearly, modern C++ has very little to do with exceptions, or I would have required them or found them useful at some point. I’m not saying exceptions are not useful, but their centrality to the language is empirically false given the vast number of modern C++ code bases that don’t use them and lose nothing by it.
Database kernels do virtually no heap allocation as a matter of good design. They also rarely use stdlib containers or anything else that dynamically allocates on the heap outside of the bootstrap context. Virtually all of this memory has to be dynamically pageable to storage (and you can’t use mmap() or virtual memory) which largely precludes using parts of the stdlib that require allocations. They work within a tightly bounded memory footprint plus the stack, and these resource reservations are pretty strongly guaranteed at bootstrap. Even the stack often has analyzable bounds. It is hard to design good schedulers, whence most optimization comes, without these constraints.
If your process doesn’t survive allocation in bootstrap then it is badly misconfigured. If it survives bootstrap, then the software has been able to make hard guarantees about its resource constraints; the runtime will optimize to those constraints. Many years ago, given the designs of the time, you had to worry about blowing out the stack memory occasionally in edge cases, but even that isn’t a real problem these days due to changes in how it is done.
You would design things this way in any language. The benefit from a software design standpoint is that you can derive an enormous number of invariants from these constraints that can be used to guarantee that the core processes will never experience resource exhaustion. This eliminates a massive number of edge cases that would have to be accounted for if resource exhaustion was detected by testing limits.
Also when speaking about kernels, it isn't as if the other languages being discussed (including C), make use of their respective stdlib in kernel space.
Linux tries pretty hard to use panics as an absolute last resort: there are a lot of invariant failures that are better handled by disabling drivers or subsystems, marking an impossible response as illegal and just trying to muster on, etc.
Having exception-worthy invariants become panics would probably be a nightmare, at least for idiomatic C++: every nontrivial ctor (including copies and moves) might be a panic site.
While I disagree with disabling exceptions, modern C++ has all the goodies as Rust (minus pattern matching) for using return errors instead of exceptions.
Probably because the older languages did not bring enough to the table versus C.
I've notice that the trend of new compiled language is fairly recent with Nim (2006)/Go (2009)/Rust (2010). The previous trend was more oriented toward interpreted languages Python (1991)/Ruby (1995)/Javascript (1995).
So maybe, 10 years is the maturity age and we will see more of those language used for Linux kernel development.
Many other potential systems programming languages are object-oriented (C++, D), which at the very least counts them out because Linus and perhaps the rest of the core team dislikes OOP. I can't say one way or the other whether their assessment is objectively correct, but another data point here is that Windows, while mostly written in C++, also sticks to C for the kernel, and even the C++ code sticks to C-compatible APIs at module boundaries, for maintainability reasons. Complexity may only arguably be a shortcoming of object-oriented languages, but compatibility is absolutely a problem.
Others, like Go, use garbage collection. That may not be universally unacceptable in OS kernels, but it's certainly not something anyone wants to see happening in the Linux kernel, which sees use in applications with low-memory and real-time constraints.
And the rest are dead or dying languages such as Pascal and Ada.
I can't speak to Rust's merits, I don't know it well enough, but if some reasonable subset of Rust makes it easy to stick to C-compatible ABIs without sacrificing most of the language's practical differences from C, then I would say that, yeah, it may truly be the first good - or even reasonable - candidate.
I wouldn’t say that Linus et al. dislike object orientation - the entire kernel is built on a system of object abstractions which just happen to be written in C rather than C++ or similar.
The biggest reason why Rust is, in my mind, a contender for the kernel is that it genuinely adds a new capability that C++ does not: true and guaranteed memory safety, with an unsafe core that is much easier to audit. Drivers written in Rust will be much less likely to contain memory safety problems, and that’s potentially a big win for kernel maintainers and developers. And yeah, it’s also interoperable with C to a large degree, which certainly helps.
I'm no kernel developer, but I would guess that even putting some yellow caution tape around the risky stuff, and knowing everything that isn't taped off is less risky, could be a win for maintainability. The current status quo is presumably that people need to be able to grok 100% of the code deeply enough to be able to evaluate all proposed changes for memory safety implications at all times.
But can't you at least surmise that, as long as you can independently show that the code in the unsafe blocks only affects safe code in the intended manner, you can reason about the unsafe behavior in a more modular way?
Kind of a memory safety equivalent to what structured programming did for control flow.
TIOBE ranks Delphi/Object Pascal above Go and Rust.
I don't quite get this odd behavior of so quickly labeling a language as "dead" and devoid of any evidence to back up the claim. In the case of Pascal and Ada, various commercial interests want them to be "dead", because both represent a threat to their financial interests.
Pascal became Object Pascal, which is very much around in the form of Delphi/Embarcadero, Free Pascal/Lazarus, Oxygene/Rem Objects, PascalABC, etc...
Ada for sure is not going anywhere for many years to come because of its use in the military and aerospace industries.
I think that isn't fair to Linus. C++ with minimal use of templates, nearly no dynamic allocations and never calling any function that could possibly throw an exception is pretty dang close to C.
“I will not use C++ and programmers who develop projects instead of C are kicked out, lest they mess up the projects I’m involved in”, “C++ ended up with a bunch of terrible, unmaintainable garbage”
So how could C++ ever be possible in the Linux kernel?
The problem isn't technical, and to come back to your C++ example, there are plenty of OSes using C++ at the kernel level.
macOS being one of them via IO Kit, Windows since Vista, or that maker board loved by everyone, named Arduino.
Probably to Linus dismay, all Project Treble drivers (which run in userspace) on Android, are mix of C++ and Java, classical Linux drivers are considered "legacy" in what concerns AOSP project.
Unix came from AT&T, who then their people created C (primarily to be used with Unix) and then C++. All other languages didn't really have a chance.
The only other language that could of had a chance of derailing the use of C on Unix and related was Pascal, because it was quite popular in many universities at the time and of similar capability. Consequently, people working for AT&T or connected to C development, viscously attacked Pascal in various papers and there was a lot of bad mouthing in the American media to help make sure it would not become such a threat. With AT&T so strongly pushing C and Unix, they got the result that they wanted.
Niklaus Wirth, the creator of Pascal, didn't defend his creation because he was preoccupied with Modula-2 and later Oberon. Even so, he was just a university professor, and any defense that he might have tried would have likely failed anyway in the face of AT&T money and connections.
We have to rule out any language with a garbage collector. C++ adds about as much negative as positive to be worth while. That doesn’t leave many other options.
It’s fine to have opinions, but you’re stating them as if they’re fact. The Linux kernel maintainers disagree with you. They’re experienced engineers, and ultimately, they make the rules.
C++ is the industry standard, C is a retrocomputing language. Everyone is using C++, except for legacy codebases and some recalcitrant FOSS maintainers. Of course Linux is maintained by Linux maintainers, I am just pointing out they are against industry consensus.
Mac drivers (which run on kernel space) are written on a C++ subset.
Windows has been slowly migrating to C++ since Vista, when VC++ got kernel support for C++ code, naturally one just doesn't move something like Windows to C++ just like that.
> First off, let me point out that this library is used to implement large parts of the OS. There are hundreds of developers here who use it. So unlike, uh, some other things that get tossed onto github, this project is not likely to wither and die tomorrow.
I would say for lower level code, when C++ is used, it's a C-like dialect of C++ that is used. This is even true with something like clang and llvm.
Certainly the /more/ low level you go, the less C++-isms you see.
These days, sure you can use a highly restricted form of C++ in systems close to HW, but you might as well be using the industry standard in those environments, which is C, compiled with a C/C++ compiler.
Fuchsia is a micro kernel. The core kernel bit is written in C + ASM as Rust wasn't ready at the time. The various ancillary pieces that make up the OS (e.g. filesystem, drivers, etc) are written primarily in Rust. C++ I believe had been used but as Rust has matured and evolved that's gone away for the most part.
Rust makes up the majority of the current Fuchsia code on the main branch (2.6M lines), at least according to `cloc`, but there are more lines of C++ code (1.6M) than plain C, assembly, and C/C++ headers combined (0.95M). And that's very generously assuming all those headers are compatible with C and not just C++.
-------------------------------------------------------
Language files blank comment code
-------------------------------------------------------
Rust 11813 339160 541958 2661720
C++ 9207 344592 198973 1632916
Go 2906 81163 138138 608871
C/C++ Header 7544 156294 209786 559658
C 1680 49614 52841 260405
Assembly 322 18112 11620 133536
-------------------------------------------------------
Header + C + Assembly 9546 224020 274247 953599
It would be wise not to confuse C++ at Google with C++.
Google has extremely strong internal coding standards. Those standards restrict which C++ features can be used, and how and where those C++ features can be used.
Without compliance based upon paycheck and performance reviews, I'm not sure that I'd want to try to make that kind of selective enforcement work.
Oh huh. It must have moved on significantly from when I last looked at it years ago. It started of based largely on LK work.
If you look at the old code [1], it has a C++ extension & is likely compiled with a C++ compiler but the code itself is really just C. That's why cloc sometimes isn't an accurate tool for these. I don't have a sense of what the policy is but they could still be sticking with the C convention w/ C++ only utilizing things like constexpr & various other compiler pieces. They do seem to be making use of templates so it looks like there's been a lot more modernization done there.
I think the main issue with C++ is just that C++ is not one thing. Everyone has a different idea of which of the incredible number of features added over the years are good, and which are footguns. Obviously many of the features of C++ are incompatible with kernel development, since C++ is designed to be used for applications, not kernels.
Exactly. Very well said - in other words: C++ is NOT ORTHOGONAL, whereas C is.
C++ is also an evolving monster that everyone uses different subsets of, and many developers only know or use subsets of, either for lack of expertise or out of conviction (see how polarised the discussion of exceptions in C++ is here).
And every five years, even more complexity gets added, making the C++ story a modern PL/1-like story.
For many decades, there were also substantial differences in compiler maturity, forcing many to rely on subsets artificially.
But C is so small that every project will use their own under-specified conventions with no documentation, no tooling, etc.
At least C++ language features will have proper documentation, not this “if we place a struct inside this one at a specific place, we can use it as a linked list for no added memory cost” shit that one must know for each and every project (and is error prone).
Ok, now it's my turn: C++ would be a clear loss on net. Anyone thinking otherwise, even if they are Linux developers, are wrong. Now that was productive.
People are talking about Linus' opinions of some other languages, but more importantly the Rust folks have been prepared to put in a lot of work to meet Linus' concerns. You're now seeing all sorts of other language advocates pop up with D, Zig, and so on, but they've not put in the same work to convince Linus that anything else are real options.
There are not many older languages that specifically targeted low level programming and gained some level of traction, which I cannot think out of C++ (and perhaps D? but it's not really popular).
Rust has many qualities, but it's not a simple language.
The problem with complex languages is that they are ill suited for low level stuff.
Sometimes it's better to not have high level language features, because it allows everybody to understand the code without having to know every feature of the language.
Simple is better. There are too many developers who prefer complexity.
Simplicity is not that simple :) Brainfuck is an objectively smaller and simpler language than C, but it's not any better.
Certain problems have an inherent irreducible complexity. This complexity has to go somewhere. If the language doesn't tackle it, then users of the language will have to.
This is especially clear in the case of memory errors in C. C is too basic to express ownership of memory, valid scopes of pointers, and thread-safety of their usage. This doesn't make these problems disappear, only requires users to handle it manually and get it right without compiler's help.
In Rust you have to specify ownership and thread-safety of everything. It is complex, but not because Rust is complex, but because safe low-level memory management and thread safety are difficult problems.
Not that difficult problems most of the time, but in practice the problems are more or less ignored and it often works, in most cases anyway. Not good obviously.
Problem I have with Rust is that it uses certain patterns to make it work against the unforgiving borrow checker. I am not convinced that we will stay at this point.
Rust disallows some operations where the developer can know the operation to be safe, but the borrow checker does not. So you need to accommodate the language.
Yes indeed, in Rust you have to do things the Rust way. But that's not much different than pains of trying to write imperative code in an OOP language, or mutable code in a pure functional language. Rust is closer to ML than the C family of languages, and the differences become apparent if you try to use it like C/C++.
It's not a blocker in practice, because you can fudge things with `unsafe` if you know you're smarter than the borrow checker.
Sure it's simple in a sense there are not many keywords or features.
And you can learn it over a weekend and start programming in it.
But when you factor in undefined behavior, straight up user hostile choices done by common compilers in the name of performance, common patterns/library functions (that are all over books and stackowerflows) to avoid etc. C is not that simple anymore.
When you throw in concurrency, programming in C is quite hard.
I write this because I used to be an embedded C programmer, but that was over 10 years ago. I can still read code, sometimes even add simple patches (not to kernel ), so you could say I know the language. But it would take me several months of focused training before I would even dare to touch something as linux kernel.
Not saying that Rust is simple, just saying C isn't as simple as some people make it.
>Simple is better. There are too many developers who prefer complexity.
This is going to be unpopular opinion because it is more philosophical and completely unscientific.
I have been thinking something along that line for quite some time and not just on technology or programming languages. I think it has a correlation with age or how far along your journey in life. Simplicity has to click, it cant be told or learn just by reading. And more often that not, you can only understand simplicity after you try and fail to "fully" embrace complexity.
The added features and complexity of Rust may be a worthwhile tradeoff for the memory safety it provides compared to C. You're allowing a lot of mental overhead of "is this safe?" to be offloaded to the compiler, whereas with C you have to be extremely vigilant. Even the most talented programmers frequently make memory-related mistakes in C.
If you want a truly simple language you should have a look at Go.
Yeah, it is sure better to pass around shitty c strings and iterate over them 402 times at each place to know its length because the language is unable to put a proper abstraction over them.
Also, all of these are low level languages. There is no real difference between C and C++ in that way. C just lacks expressive power compared to C++.
not OP, but: Presumably the impression that there is a Rust hype squad that's pushing it everywhere and swipes away any counter arguments.
I'm not that opposed to Rust and think its an interesting language, but the more annoying parts of its fanbase really haven't done it any favors. The sort that comments on every new project not in Rust "why is this not in Rust" (this specific thing seems to have died down somewhat), ignores any progress in other languages or Rusts weaknesses when making comparisons and thinks "Everyone still writing C++ should be tried in Den Haag" is a perfectly fine contribution (yes, that's a real example. sadly)
C++ has nearly as many rabid fanboys as Rust. I am curious why the community reacts differently to them. I am not saying "just write it in Rust" is a good thing, I think anyone claiming someone else needs to rewrite their program is automatically wrong.
This thread is full of "Linus hates C++" as if "the subset of C++ that can be used in the kernel is basically C" is an outright fabrication.
I may be oversampling on the fact that nearly every article about programming languages here is filled with people wondering what the point is though shrug.
> "the subset of C++ that can be used in the kernel is basically C" is an outright fabrication.
It's IMHO a good example of "ignore any progress in other languages", because it was a lot more true back when the question of C++ in the kernel first came up than it would be today. Which is not to say that the kernel should add C++, Linus is fully allowed to still think c++ isn't a good choice or to personally dislike it.
And I'm very certain I've seen less intrusive pro-C++ spamming than I've seen for Rust. C++ hasn't been cool or hype in a long time, it's mostly on the defensive in such arguments. Even Go wasn't as bad IMHO, and that also had some hype factor.
EDIT:
> This thread is full of "Linus hates C++"
Ok, I skimmed the thread. There is one comment saying that, one saying "Linus wasn't a fan at the time". "Full of", sure.
You haven't seen many Hacker News pages involving a GC language and the topic of performance the number of "just write it in C++ like everyone else" is frustrating.
Nothing against C++ just telling someone else to change languages is unhelpful.
I wrote "Less religious missionaries. Also, less funding by FAANG."
To answer your question, I honestly don't have a strong opinion about Rust. I had to deal with it years ago when I needed to recompile parts of Firefox and back then it seemed OK but still a bit messy, like all new programming languages.
But I cannot shake the feeling that Rust has some very loud fanboys. The original mailing list entry is about Rust being merged into Linux-next. C++ and Java are already in there, too [1]. So while that is a great preparatory step for Rust, it's not the big breakthrough that it is presented as. By now, I've read so many articles that overstated the benefits of Rust (e.g. "zero cost abstractions" - which then fail to optimize) that I immediately become suspicious if something is news purely because it relates to Rust.
And w.r.t. my second statement. The original article thanks Google for their generous funding. Which is true. Rust did receive generous funding from many sources, because companies who need a safer alternative to C / C++ tend to like it. And I believe that is a factor which contributes to Rust being merged into linux-next while other languages weren't. Rust has the necessary funding to do the preparatory work.
You clearly misunderstood the link you gave about Java support if you think it's relevant to how the kernel is implemented.
The kernel has extensible support for execve-ing files the kernel doesn't directly understand as binaries by forwarding them to another program that behaves as an interpreter, the binfmt-misc machinery. The docs you linked just give an example of configuring that for various types of Java files.
I think Rust is like Wayland (and systemd before that); it's a new technology that probably is much better than the thing it intends to replace, but it does so by completely changing how things work, raises the lower bar on complexity (yes, I understand that it also lowers the ceiling), happily discards things that its proponents view as unimportant edge cases (I can use C to build an X client on NetBSD on Alpha), and - very importantly emotionally speaking - frequently is first noticed when it breaks something (Why did my graphical session break? Wayland. Why did my Python cryptography module break? Rust.).
Where did I say 'I didn't like Rust'? I'm sure my comment is directed specifically at the 'Rust hype squad'.
It is them who push the 'Rewrite it in Rust' message on projects without assessing the consequences of doing so; and they continue to use the above excuses of justifying it.
I do not hate Rust at all as I am completely free in choice of my tools. Reading HN however leaves me with the impression that Rust public is often on a holy quest to declare anyone using languages other than the Rust as infidels.
I'd like to put forward a controversial opinion, if I may: I do not think adding Rust to the Linux kernel is a good idea; and I suspect this project will struggle as soon as the financial backing is removed.
First, a bit of background: I am mainly a C++ and C# user space developer, so presumably despised by all camps involved. Secondly, I have no experience of kernel development whatsoever. Thus, take my comments with a large grain of salt.
My argument is as follows. Every decision you make on a software project is akin to an intersection on sets of programmers. When you choose C, you take a small slice on all possible programmers; when you choose kernel development, you take a slice on the set of competent C programmers, and so on. The intersection of programmers that are competent in Rust _and_ in C _and_ in kernel development - including the interaction between the two languages, at a very low-level (because kernel-space is special) - must be astonishingly small. And herein lies an important problem. Let us posit that the kernel does gain a significant amount of code in Rust; the interaction between these two languages (read: friction) will become significant. Either the Rust people will break things for the C people, or the C people will break things for the Rust people. Those who like C but not Rust - presumably a large subset of the Linux C programmers, else one would assume they would be developing a kernel in Rust - will become alienated. Similarly, those who like Rust will be forced to spend a lot of time doing non-rusty things just to get Rust to work, and they too will not enjoy the experience. Finally, many of the sweeping code clean-ups the kernel experiences periodically will apply differently to Rust, if they apply at all - meaning it will be a second class citizen, barring some major investment to compensate for this.
In the end, I do not think the problems will come from the languages involved per se but due to the complex technical and especially social interactions that this will create.
It is, however, an extremely interesting project technically and I suspect that Rust (and maybe even the kernel) will benefit from it, though not necessarily in ways one would expect. As far as research goes, it is pretty cool.
If they stick to core and basic Rust features, then I can see this project succeeding.
Types more advanced than Vec<T> should be avoided IMHO. Rust is a beautiful language when you can contain it's complexity to a minimum. Them it reads like safe C, just like it should.
"Types more advanced than Vec<T>" are inevitable. You need them to write safe abstractions for the rest of the kernel to use. Simplicity is a virtue, of course, but not to the point of limiting Rust to the equivalent of "safe C".
It keeps the code flow, which lives in the C file, easier to read. You can assume the function does the appropriate thing for the given configure situation without having to keep track of all the compile-time conditionals as you scan the implementation.
Anyone know what they are referring to by "... and simplified pointer wrappers"?
I'd also be very interested to hear a Rust for Linux perspective on the current state of async Rust. I've been avoiding diving into it for years now, but I'm always looking to be convinced I should.
> Would a transpiler written in GCC compatible C, compiling Rust to C, solve this issue forever?
Solve what issue?
If you're talking about the problem of, "Linux is available on more systems than the Rust compiler has been ported to," then I'm gonna say "no," because many (if not all) of the core issues that make Rust difficult to support well on various architectures (and hence why said support isn't in official Rust today) would also have to be solved by any so-called "Rust-to-C transpiler."
That's very much a [citation needed]--all previous coverage I've seen of Rust support in the Linux kernel has indicated that the mood of the Linux developers generally is at least "cautiously supportive."
What makes you think Linus won't learn Rust? I believe he's said that accepting Rust into the kernel will involve key maintainers learning Rust, and that so far they seem cautiously open to that.
However, in reality Linus wont necessarily review the patch line by line. Mostly the reviews will happen on mailing lists. Then if Linus' lieutenants that handle the specific subsystem like what they see, they will probably give a general thumbs up for the change. Finally Linus will take an executive call. He may offer some technical suggestions (as he is technical) and send it back or accept the patch or conditionally accept the patch.
The point is that the finer details will have already been dealt with by the time it lands on Linus' desktop.
But given that this is going to affect a lot of things for years to come he is probably going to spend a lot more time on it than other features.
Also I think Linux may be interested in having Rust in the kernel. See here https://lkml.org/lkml/2021/4/14/1099 . He says "on the whole I don't hate it".
It's clear to see that there is a lot of momentum behind Rust. Allowing Rust in the Linux kernel will help bring a lot more talent and energy into the kernel community (other than the inherent merits of Rust which I think are easy to see).
Linus has already learned Rust and has been actively participating in reviewing previous rounds of patches. For example[1]. His general consensus so far: "I don't hate it"[2].
> Rust's big barrier to entry is learning to write code that passes the borrow checker, but that's not a concern when doing code reviews.
Rust code is actually easier to review in that regard because (unless the code contains unsafe blocks) the reviewer can skip thinking about most aspects of memory management.
Time moves on, its of value to be conservative with new languages but it also dogmatic to say it should always only be one. Yes having one has value, but having more then one also can have value. Rust has reached a level of acceptance and adds huge value that it is worth it for the future.
There are people doing that as well. There are many, many people in the world, working on one project does not preclude others from working on other projects too.
I wish they would start creating a new kernel from scratch
in Rust.
It is needed and will be more cohesive and take more advantage of
the new language than stuffing it into Linux.
There will be a lot to learn from it though.
It will test the interop and cohabitation of Rust and C.
Lessons learned will be unbelievably valuable for other efforts to
mix and match them.
I refuse to believe that WindowsNT, NextStep/macos, Linux/UNIX/BSD
is all we can hope for in an operating system.
WindowsNT is by far the most modern.
Linux is a not terribly good copy of an operating system made to run on a PDP11.
WindowsNT owes quite a bit of internal architecture from Vax.
Just like there is constant experimentation and development of new programming languages there should be constant experimentation and development of new operating systems.
Limits, metaphors, hardware, has changed to an extreme degree, there is
no reason to stick with that made sense decades ago. :)
Where is the Tesla Operating System, (well in something a bit new and fresh with some good ideas and a lot of problems)
Could you give an example or two of something concrete that you don't think is sufficiently addressed by today's operating systems?
In my day job I do data engineering, and I'm a long-time linux user. Multimedia stuff aside, I think it's been 8 years since last time I felt limited by the operating system.
Granted, I don't do anything particularly low-level. But as a high-level "consumer", I experience the main issues with today's operating systems not to be technical capabilities, but rather user friendliness, flexibility, and standardised APIs.
But I'm curious what the issues are from a lower-level point of view!
>A few notes for people who haven't been tracking the Rust-in-kernel situation. Drivers especially have been targeted for a few reasons. First, drivers are isolated, with abstractions that don't require interaction with the rest of the system. Second, Linux is available on more systems than the Rust compiler has been ported to. Because drivers are inherently system-dependent, it is not an issue to introduce a dependency on rustc on compatible systems. Third, bugs tend to arise more in drivers than other parts of the codebase, so Rust can do the most good starting there.
[1] https://www.reddit.com/r/rust/comments/rb50yn/comment/hnmshs...