
Supporting Linux kernel development in Rust - dochtman
https://lwn.net/SubscriberLink/829858/281103f9c6fd0dc2/
======
jasonhansel
I'm concerned about the gradual move from GCC to LLVM. The lack of copyleft
protections on LLVM means that it's much more dependent on corporate
sponsorship, and means that there's a risk that major improvements to LLVM
compilers will only become available as proprietary products.

People underestimate the role of copyleft licenses in preserving long-running
FOSS products like GCC, Linux, etc.

~~~
woodruffw
> I'm concerned about the gradual move from GCC to LLVM. The lack of copyleft
> protections on LLVM means that it's much more dependent on corporate
> sponsorship, and means that there's a risk that major improvements to LLVM
> compilers will only become available as proprietary products.

As someone who works within LLVM professionally, I don't think this is
particularly likely -- compilers are _massive_ and complicated beasts, and the
"moat" for proprietary improvements is small: they're either niche and
therefore not of interest to the majority of programmers, or they're
sufficiently general and easiest to maintain by releasing back upstream (where
Apple, Google, and Microsoft will pay a small army of compiler engineers to
keep them working).

Your concern is the one that kept GCC from stabilizing its various
intermediate representations for decades, which is why virtually all program
analysis research happens in LLVM these days.

Edit: To elaborate on the above: neither Apple, nor Google, nor Microsoft
wants to _individually_ maintain LLVM. Microsoft appears (to this outsider) to
be actively looking to replace (parts of) MSVC/cl with the LLVM ecosystem,
because they're tired on maintaining their own optimizing compiler.

~~~
pjmlp
I don't consider the improvements that ARM, Apple, Sony, CodePlay, NVidia,
AMD, among others, don't upstream due to IP considerations or revealing of
hardware secrets, niche.

~~~
_ph_
But if they have IP considerations or trade secrets to protect, they wouldn't
contribute them to any other open source project either. This way, they can at
least upstream everything, which doesn't fall under these restrictions.

~~~
pjmlp
They don't upstream everything, while reducing their development costs, that
is the point.

Personally I don't care, but I bet many FOSS anti-GPL advocates will
eventually care.

~~~
adwn
> _They don 't upstream everything, while reducing their development costs,
> that is the point._

_ph_ meant to write _" This way, they can at least upstream everything which
doesn't fall under these restrictions."_ (note the deleted comma before
"which"), i.e., they can at least upstream _something_ instead of _nothing_.

~~~
_ph_
Yes, thanks :)

------
cbmuser
If we get the Rust backend for gcc finalized and merged first, it would be
much easier to get Rust code into the kernel:

> [https://github.com/philberty/gccrs/](https://github.com/philberty/gccrs/)

~~~
gok
That's a front end?

~~~
Spivak
Compile C to Rust! Easiest language migration ever.

~~~
fractalb
That would eliminate the need for Rust. Just imagine, if your C code can be
converted into Rust, would there ever be a case to start a fresh project in
Rust? It will permanently become an Intermediate language.

~~~
adrianN
Converting C to Rust is not that hard. Converting C to _safe_ Rust is the
difficult part.

~~~
fractalb
Rust without safety isn't Rust at all!

------
puzzledobserver
One of the challenges to building an entirely new kernel is the vast amount of
hardware support in the form of drivers and the features the existing kernel
exports to userland in the form of syscalls.

Would it be possible for a hypothetical new kernel, presumably written in
Rust, to run an existing kernel such as Linux in a VM, just to tap into its
drivers and emulate its syscalls? As more drivers are ported to the base
kernel, the reliance on the donor kernel would shrink over time.

Edit: This was an aside. Of course I realize that the conference was about
adding Rust code to the Linux kernel, and not about rewriting any large,
important, and functional codebase in language-of-the-day™. Hence my
speculative language: "hypothetical" new kernel, "presumably" written in Rust.

~~~
JoshTriplett
The point of this LPC session was how to incrementally introduce Rust in the
existing Linux kernel, with all its existing drivers and syscalls and similar.
A rewrite would indeed be a daunting and problematic proposition.

There are kernels written in Rust, such as Redox, but those are separate
projects, and that's not what this conference session or article are talking
about.

Standing reminder: the Rust project is not a fan of rabid Rust over-evangelism
(e.g. "Rewrite It In Rust", "Rust Evangelism Strike Force"), and sees it as
damaging and unhelpful. We discourage that kind of thing wherever we see it.
We're much more in favor of a measured, cautious approach.

~~~
monoideism
Unless the OP changed his comment, this is an overreaction to a harmless,
hypothetical, technical question. I've seen a lot of annoying Rust evangelism,
but this ain't it.

BTW, I'm not a Rust dev, and it would appear the OP isn't a Rust developer
either. They don't even seem aware of Redox. Seems odd to accuse them of "Rust
evangelism".

~~~
JoshTriplett
I was not accusing the OP or anyone here of such evangelism; rather, because
the OP was talking about writing/rewriting a kernel in Rust, I was stating
that _we_ weren't looking to "rewrite it in Rust" or similar. We've seen a lot
of people suggesting that the work on putting Rust in Linux was trying to
"rewrite the kernel", and it seemed appropriate to provide such a disclaimer.

------
ogoffart
> The ubiquitous kmalloc() function, for instance, is defined as
> __always_inline, meaning that it is inlined into all of its callers and no
> kmalloc() symbol exists in the kernel symbol table for Rust to link against.
> This problem can be easily worked around — one can define a
> kmalloc_for_rust() symbol containing an un-inlined version but performing
> these workarounds by hand would result in a large amount of manual work and
> duplicated code.

That's why there exists crates like
[https://docs.rs/cpp/](https://docs.rs/cpp/) which allow to embed C++ (and
thus C) directly into the rust source code, simplifying the manual work and
reducing the duplicated code.

~~~
geofft
Yup - that got a very brief mention in the article as "Google is working on
automated C++ conversions"; there's also
[https://github.com/google/autocxx](https://github.com/google/autocxx) which
builds on top of it.

C++ is a richer language than C and has more information about ownership
(e.g., if you return a std::string you know how ownership works; if you have
two arguments that are a char * and a length it's much less clear), but
additional annotations in the kernel's C headers to convey this level of
information in a machine-parseable way would be awesome.

~~~
ogoffart
The cpp and cxx crates are two different crates with different goal.

The cpp crate is about convenience, but require the use of unsafe within the
rust code.

The cxx crate do extra checks making the rust code safer but needs more
boilerplate.

In the case of C however, these type checking are not really possible, as you
say.

------
nielsbot
Sounds like they have the same problem Apple's Swift does for calling into
C/Obj-C.

Two examples: My understanding is that there's a ton of code and added
complexity in Swift itself to support Obj-C interoperation. And while you can
call C and Obj-C APIs, as-is, there was/is basically a company-wide effort to
write API wrappers (aka. overlays) for existing system APIs.

For the second part, maybe if Rust for Linux kernel programming gets to the
level of popularity as, say, TypeScript in the JavaScript community, the Linux
community will end up creating the needed API wrappers as an organic, group
effort.

~~~
why_only_15
This is mostly just a random tidbit but one of the fascinating things to me
about Swift <-> ObjC interop is that it's not just that Swift was modified for
ObjC, but ObjC was modified for Swift as well. In general it works pretty well
and I wonder how much of that is because Apple can modify ObjC whenever it
wants.

~~~
pjmlp
.NET <-> C++/COM also went through a similar process.

~~~
snuxoll
.Net is a much simpler integration, all of the support is in the CLR because
COM defines a limited ABI that can be called from any language that can walk a
few pointers in a C++ vtable given an interface definition.

One can even completely ignore the RCW/CCW support in .Net entirely and just
hack together some struct types that match the vtable layout.

~~~
pjmlp
Old style COM yes, UWP supports much more, value types, enumerations,
generics, partial support for implementation inheritance (in context of XAML).

Also you ignored that .NET was made to fully support C++ via Managed C++,
replaced with C++/CLI on version 2.0.

~~~
snuxoll
The WinRT ABI does implement some new features, yes - but they’re all still
implementable with nothing more than standard C# code with no special support
from WinRT itself (in fact, the WinRT ABI integration is being stripped out of
the runtime and cswinrt is the way forward).

C++/CLI ultimately still uses the same features the runtime uses for p/invoke
calls, though it uses slightly different IL to transition to native code since
it doesn’t have to worry about an external ABI within a mixed mode assembly.
Sure, there’s a whole compiler that supports this weird mixed mode world from
a language perspective - but it’s really boring at runtime.

~~~
pjmlp
All of that doesn't change the fact that there is tooling in place, and
language features (e.g. dynamic for COM, upcoming C#9 improved native FFI) to
ease interoperability between managed and some level of C++.

Something that no UNIX does, and we only find similar ideas in
Swift/Objective-C++, IBM language environments, and the old OS/2 SOM
(Smalltalk/C++).

While everyone else just writes glue code with C like interfaces, as if the
world has hardly changed from UNIX V6.

~~~
snuxoll
Purists may hate it, but as a toy project I’ve been working on making a
stripped-down version of the WinRT concepts that runs on Linux (well, anything
with a c++17 compiler and a dlopen function provided by a linker). The guy
behind cppwinrt started work on the Xlang project in a similar vein, but it’s
currently not being worked on so I figured I’d give it a shot.

I agree that C as the lowest common denominator sucks.

~~~
pjmlp
Good luck with it.

Project Reunion seems to have rebooted everything, which is most likely the
reason for it to have stalled.

------
aey
We built a whole BPF tool chain for rust :), although not for kernel
development.

[https://github.com/solana-labs/rust](https://github.com/solana-labs/rust)

If this kind of seems interesting please send us a CV.

~~~
RMPR
Do you have some kind of official blog for this? I'm interested in BPF and
currently learning Rust, would love to learn more about this.

------
boogies
Probably a crazy unpopular opinion, but I kind of agree with the Hyperbola
(GNU / formerly Linux-libre, in future using a fork of the OpenBSD kernel
[https://itsfoss.com/hyperbola-linux-bsd/](https://itsfoss.com/hyperbola-
linux-bsd/)) devs that Rust in Linux is not necessarily a good thing, for the
reasons they mentioned when they announced they were switching kernels
([https://www.hyperbola.info/news/announcing-hyperbolabsd-
road...](https://www.hyperbola.info/news/announcing-hyperbolabsd-roadmap/))
(Edit: and the LLVM dependency).

~~~
JoshTriplett
The vast majority of the text in that post is inaccurate and hyperbolic. Linux
is not "forcing adoption of HDCP" (it's just something Linux has a driver
for), Rust does not require internet access to use (and the trademark comments
are not particularly accurate either), the Kernel Self Protection Project is
alive and well and making new improvements in every new version of the kernel
(see Kees Cook's regular blog posts for details), systemd is not spelled
"SystemD", gettext has no dependency on Java (and the link _in that post_
explains that). And in general, decisions about what technologies to use are
made by those who show up and do the work to build viable solutions, not by
people who snark.

It's a rant by an obscure one-developer distribution that _formerly_ used the
Linux kernel. Some people are never happy no matter what you do, and if you
make the mistake of treating their rants as useful feedback, you get a list of
30 more demands before they'd consider gracing your little project with the
honor of their all-important usage again.

~~~
ColanR
I think you were being "inaccurate and hyperbolic".

> Linux is not "forcing adoption of HDCP" (it's just something Linux has a
> driver for)

What they actually said in the article:

> Historically, some features began as optional ones until they reached total
> functionality. Then they became forced and difficult to patch out. Even if
> this does not happen in the case of HDCP, we remain cautious about such
> implementations.

~~~
JoshTriplett
My comment was entirely based on the second link. Direct quote from that:

> Linux kernel forcing adaption of DRM, including HDCP.

(Which linked to a patch adding driver support for handling HDCP, with
absolutely no possibility of it somehow being forced.)

Regarding the quote from that interview: sure, and there's also no guarantee
that the Linux kernel won't drop support for every architecture except SPARC,
except of course for all the people involved not putting up with it. In what
possible universe would Linux developers decide it was a good idea to mandate
HDCP? It's completely unfounded and unsupported hyperbole and fearmongering,
without even a hint of potential truth.

It's also roughly consistent with what I'd expect from someone who thinks that
gettext daring to have support for Java and C# format strings is a horrific
problem that should be ripped out by the roots.

~~~
boogies
I don’t know about Linux, but Mozilla has mandated many things that I would
have hoped it never would have, like XUL depreciation, and especially added
many anti-features to their largest product, deciding each was not their hill
to die on
[https://news.ycombinator.com/item?id=24124954](https://news.ycombinator.com/item?id=24124954).
I left Firefox and have never regretted it. The Hyperbola devs did the same,
and I will be happy to try their OS and support them fighting on every hill
they find, no matter how tiny or unimportant it seems to larger organizations.

~~~
JoshTriplett
> like XUL depreciation

There were specific reasons for that: [https://yoric.github.io/post/why-did-
mozilla-remove-xul-addo...](https://yoric.github.io/post/why-did-mozilla-
remove-xul-addons/)

------
Thaxll
Looks like there is a lot of work involved to have anything done, remind me
the issues that the Chrome team said 2 weeks ago.

~~~
gpm
There is no doubt a lot of work here, but C <-> Rust interoperability is a lot
cleaner than C++ <-> Rust.

And for what it's worth, for almost all langauges C <-> X interoperability is
a lot cleaner than C++ <-> X. Since almost all languages (including rust)
"speak C", both in terms of compiler support and and being able to map every
important C concept to a concept in their language. The same isn't true with
C++.

One thing to worry about with adding any language (including rust) to the
kernel, is that the next good looking language will probably interoperate well
with C and not interoperate well with Rust. So there is a much higher cost the
second time you try to add a language.

~~~
JoshTriplett
> One thing to worry about with adding any language (including rust) to the
> kernel, is that the next good looking language will probably interoperate
> well with C and not interoperate well with Rust. So there is a much higher
> cost the second time you try to add a language.

This is something that Rust is acutely aware of. There have been many
discussions about the idea of a higher-level "safe ABI", such that languages
with notions like counted strings or bounded buffers could interoperate with
each other without having to go by way of an unsafe C interface.

~~~
mlindner
Is there any work currently happening on this. I think this would be huge for
many languages so we can finally graduate past the era of C ABI.

------
ghostwriter
I won't get tired of repeating the same comment in every topic that suggests
Rust to be a great replacement of C in _existing projects_ , that it isn't.

The safety guarantees that Rust provides are neither unique nor complete, and
if we discuss the amount of effort necessary for bringing new interfaces like
the one this article mentions ("one can define a kmalloc_for_rust() symbol
containing an un-inlined version"), we should compare it to other existing
solutions, like ATS [1], that are designed to be seamlessly interoperable with
C codebases without giving up on the safety side of the argument.

Just a week ago there was a link in ATS reddit channel that advertised ATS
Linux [2][3] initiative, that may be of great interest for the same people who
are interested in bringing more memory sefety to Kernel development, without
giving up on existing C interfaces and the toolchain[4].

[1] [http://www.ats-lang.org/Documents.html#INT2PROGINATS](http://www.ats-
lang.org/Documents.html#INT2PROGINATS)

[2]
[https://www.reddit.com/r/ATS/comments/ibyczp/ats_linux/](https://www.reddit.com/r/ATS/comments/ibyczp/ats_linux/)

[3] [http://git.bejocama.org](http://git.bejocama.org)

[4] [http://ats-
lang.sourceforge.net/DOCUMENT/INT2PROGINATS/HTML/...](http://ats-
lang.sourceforge.net/DOCUMENT/INT2PROGINATS/HTML/c2016.html)

~~~
gameswithgo
>The safety guarantees that Rust provides are neither unique nor complete,

Well no language can ever have "complete" safety guarantees. But I would like
to know what other languages offer the same set of guarantees without GC
overhead. This would include:

* the usual memory safety * no nulls * no undefined behavior * no data races

Does ATS guarantee those things?

~~~
Athas
Yes, ATS can guarantee all those things and more. In particular, ATS allows
you to write "unsafe" code, which you then prove safe with a mechanically
verified proof. This is in contrast to unsafe Rust, which depends on
programmer discipline, and has no (intra-language) mechanism for verification
beyond the basic type system. You could say that Rust is all-or-nothing, while
ATS allows gradual verification, as well as proving safe much more complex
code than you can in Rust. I'm not going to claim that ATS is overall better
than Rust - it's not - but it's a far more powerful language than Rust for
safe systems programming. It's just that most people are not willing or able
to pay the very high price in ergonomics. I recommend watching this
presentation for a good idea of what ATS programming is like:
[https://www.youtube.com/watch?v=zt0OQb1DBko](https://www.youtube.com/watch?v=zt0OQb1DBko)
In short, it's about as user-friendly as an overheating nuclear reactor.

~~~
ghostwriter
> It's just that most people are not willing or able to pay the very high
> price in ergonomics.

Hopefully it will get better in ATS3 - [https://github.com/githwxi/ATS-
Xanadu#project-description](https://github.com/githwxi/ATS-Xanadu#project-
description)

------
m00dy
I believe someone will show up and rewrite entire kernel in Rust.

~~~
pjmlp
Better start fresh with proper OS architectures, [https://www.redox-
os.org/](https://www.redox-os.org/)

------
egberts1
So glad that Rust at 1.38 ditched the Ada model of compiling all dependent
modules firstly.

~~~
ifmpx
Can you please expand further on what you mean by this? Are you saying that
rustc is compiling a crate before its dependencies?

~~~
pornel
Compilation in Cargo has been split into metadata pass and code generation
pass. Metadata part has interdependencies, but is relatively fast (it's
roughly like a C header generation). This allowed more codegen to be done in
parallel.

To be honest, it was a modest improvement, because compilation has been
decently parallelized already (compiler uses incremental builds, parallel
codegen, and ThinLTO even within a single crate)

------
The_rationalist
It's not about safety, if that was the real argument we should start with
obvious improvements like checked C
[https://github.com/microsoft/checkedc](https://github.com/microsoft/checkedc)
The C++ lifetime checker or Cyclone

~~~
geofft
Checked C addresses a small number of memory safety problems - it addresses
bounds checking / buffer overflows, but not use-after-frees, double frees,
data races / insufficient locking, and several other things that fall into the
general category of "memory safety," nor more general "safety" things like
type confusion.

The kernel is not written in C++ and is unlikely to use C++, for many reasons
- Rust is a more natural C-but-better for the kernel than C is.

Cyclone hasn't been maintained for over a decade; it was a research language,
and it largely served its purpose as a research language in inspiring
production-supported languages like Rust.

------
msla
What kind of performance hit is rewriting part of the kernel in Rust going to
entail when it comes to compiling the kernel? How many platforms will cease to
be able to compile their own kernels? The article says there will be some, but
doesn't give numbers.

More worryingly, the article also says that only some platforms will be
"Rusted" in this manner, splitting the codebase into "mainline Rust" and
"obscure platform C" as far as these sections are concerned; this seems like
it will fracture the development community between people who know Rust and
those who don't, and those who can work on mainstream platforms and those who
can't. Fewer eyes on this code seems like the wrong way to go.

~~~
mlindner
The point is that if you implement a driver for a device that only exists on
hardware that supports LLVM then there is no problem.

~~~
msla
No, the point is that increasing the number of languages in the kernel imposes
restrictions.

(The further point is that it's impossible to have this discussion here.)

------
chromedev
Pretty soon they'll be arguing for inclusion of JavaScript in the Linux kernel

