
Why Rust for Low-Level Linux Programming? - pieceofpeace
http://groveronline.com/2016/06/why-rust-for-low-level-linux-programming/
======
ktamura
As a long-time developer marketing person, I must say Rust is kicking ass, not
just as a language but as a community. They are deeply strategic.

1\. Clear audience target: They aren't going after C++ gurus or C magicians
but people who are new to systems programming. From Klabnik to Katz to
literally everyone in the community, they are consistent with this messaging.

2\. As part of 1, they have invested a lot in teaching systems programming 101
(heap v. stack, etc.), i.e., stuff that you learn in the first course in
systems programming in college, but many self-taught, higher-level programmers
might not know. This is a great example of authentic content marketing based
on a clear strategy working out.

3\. Their community is very inclusive. My experience (as a marketing guy who
barely remembers how to code) is that people are very helpful when you ask
questions, submit a patch, etc. This has been the case for me not just with
Rust itself but a couple of Rust projects that I've interacted with.

~~~
Bromskloss
> developer marketing person

What is that?

> They aren't going after C++ gurus or C magicians

Don't they have anything to gain from using Rust?

~~~
pjmlp
A developer evangelist. Think someone doing a talk about new Java 9 features
at a Java conference.

C++ gurus and C magicians already have invested too deep into their languages
to throw everything away and start from zero.

For example I love Rust and play occasionally with it, but for the time being
C++ is my native language on the job when I need to use a native language
outside .NET or JVM.

I know it since the C++ARM "standard" and we depend on standard OS tooling
that Rust is still catching up with.

The day will come when our customers will be able to do mixed debugging
between JVM/.NET and Rust. Or produce COM as easy as C++ compilers do.

But these are things that beginners in systems programming aren't usually
doing.

~~~
jdeeny
As a C magician, rust provides too many clear improvements over C to ignore
it. I certainly don't feel like I am 'throwing everything away and starting
from zero,' as much of my C (and other language) knowledge transfers over to
rust.

I'm not a C++ guru, but I think modern C++ is powerful enough that it doesn't
feel lacking in features compared to rust, like C does. There is less of a
draw for seasoned C++ programmers.

Rust seems to be gaining a lot of momentum and I am becoming more and more
confident that it will be regarded as a major language for embedded and
general systems programming and possibly even a successor to C.

~~~
pjmlp
For me C was already lacking when I got to learn it in 1992 , because by then
I was quite comfortable with Turbo Pascal 6.0.

Just check the feature list and type safety differences. The only advantage
from C was being less fragmented than Pascal dialects.

So I became a C++ hipster (if that would be a thing in the 90's).

We used to have the same heat from C guys that C# and other language users
nowadays have from systems languages.

Hence why I am always supportive of new programming languages that target the
same use cases.

------
jernfrost
I love C, but I think we really have to stop building all kinds of shared
libraries in C. Important code which needs to be secure and solid can't be
built on C anymore, it puts everybody at risk. Just look at the disaster
OpenSSL has been.

I think Rust would be create for building common crypto infrastructure and
things such as crypto currency. It seems risky to me to build something like
Bitcoin with C++ where millions can easily be at stake if the system doesn't
work.

I am an application programmer so I might not be the primary target, but I
started programming with Swift and although it isn't the same as Rust it has
some similarities. A lot stricter language than C++, C, Lua, Python and
Objective-C which have have used most in the past. So many bugs are caught at
compile time. I used to be skeptical towards static typing, primarily because
languages like C++ and Java made types so awful to work with. But with the
newer OOP languages with more functional inspiration, it is getting easier to
deal with strict typing.

You don't have to chose between productivity and safety so much anymore.

~~~
pcwalton
> You don't have to chose between productivity and safety so much anymore.

Exactly. If I had to sum up Rust's philosophy in one sentence, this would
basically be it. (Add "and performance" after "safety" too.) :)

~~~
infogulch
Safe. Productive. Fast.

Choose any three.

(Taking a hint from SQLite.)

------
Animats
Rather than writing for Linux in Rust, we need a new kernel written in Rust.
I'd like to see a replacement for the QNX microkernel written in Rust. It's
about 60K bytes of code, yet you can run POSIX programs on it. (You need file
system and networking, which are user processes.) The QNX kernel is stable -
it changes very little from year to year. There's hope of catching all the
bugs. This offers a way out of "patch and release" OS development.

Yes, you take a 20% or so performance hit for using a microkernel. Big deal.

At one time, you could download the QNX kernel sources and look at them.[1]
This would be helpful in getting the microkernel architecture right. It's very
hard to get that right. See Mach or Hurd.

[1]
[http://community.qnx.com/sf/sfmain/do/downloadAttachment/pro...](http://community.qnx.com/sf/sfmain/do/downloadAttachment/projects.core_os/wiki/BuildKernelWithIDE?id=atch1253)

~~~
naasking
L4.sec is already formally verified. So the microkernel is already done. You
need the user land services to provide POSIX compatibility, and that you can
possibly do in Rust.

~~~
steveklabnik
[https://robigalia.org/](https://robigalia.org/)

~~~
naasking
Looks cool, thanks. They should take the opportunity to fix a few Unix/ACL
security problems though, instead of just reproducing the same old POSIX
quagmire. Make chroot isolation complete with plan9-like private namespaces,
don't implement the traditional broken user/group security model and instead
learn from the Polaris virus safe computing prototype (they're already partway
there by using the capability secure seL4 kernel).

I would personally also want to eliminate a lot of the duplication in the
POSIX API, but that probably won't fly. Can have your cake and eat it too.

------
TheMagicHorsey
Is there any reason why embedded software for autonomous vehicles is still
being written in C/C++? This last week I was talking to a friend at a company
that makes a small autonomous vehicle. During testing their prototype suddenly
went off in a straight line. They had to pull a safety to halt the vehicle or
it would have gone straight forever into the Pacific Ocean. Turns out there
was an unsafe access to a variable in memory, which had not been caught with
their software and hardware test platform, even with thousands of virtual
sorties.

If their code was written in Rust, that sort of bug could not have occurred.

~~~
steveklabnik
There's two reasons Rust might not be ready here yet. First, while LLVM
supports a wide number of platforms, some embedded devices literally only
support the exact version of the C compiler they ship to you, sometimes, it's
even got its own custom patches. Second, we sort of assume 32 bits at the
lowest, though we have a patch in the queue that starts some work on 8/16 bit
support. This means some tiny micros are out of reach at the moment.

~~~
shepmaster
> we have a patch in the queue

If you mean this one[1], it's merged. Still lots of work to do, and even more
corners where things will shake out[2], but there's definitely progress.

[1]: [https://github.com/rust-lang/rust/pull/33460](https://github.com/rust-
lang/rust/pull/33460) [2]: [https://github.com/rust-
lang/rust/pull/34174](https://github.com/rust-lang/rust/pull/34174)

~~~
steveklabnik
Ah nice! I was unsure if the first had gotten through bors yet or not, and I
was pretty sure the second one hadn't.

------
fdr
I'd like to be able to use something like rust, and maybe I will for smaller
projects or for novelty sake, but I chafe over how slow compilation time is
relative to C (not C++!) projects last I checked.

On the hardware of yesteryear, a parallel compile could build Postgres in
about 45 seconds (750-1305KLOC, depending on measurement) , and user mode
Linux (which doesn't compile so many drivers) in about a minute.

~~~
steveklabnik
We've made steady improvements here, so depending on when you checked, it
might be much better.

The real improvements will come when incremental compilation lands. The
precursor requirements are just landing now; so it won't be immediately here,
but it will be soonish.

------
xvilka
Well, Rust is awesome, but there is a place for C too. I just don't understand
lack of the life and no improvements in C for ages. Better typing system (for
example _Generic doesn't know uint8_t, etc types - they are just typedefs),
'pure' keyword for functions without side effects, tuples support, deprecate a
lot of the things and so on.

~~~
stirner
> I just don't understand lack of the life and no improvements in C for ages.

Well, MSVC still hasn't even fully implemented C99. One of the big draws of C,
as I see it, is its wide support on many operating systems and architectures.
If you're going to abandon that by using new C features, you might as well use
a language with less cruft.

~~~
valarauca1
>Well, MSVC still hasn't even fully implemented C99

Neither does GCC. Both don't fully implement C11 either.

Most these features are either things C-Compilers don't need to support
themselves. Namely: special integer types can be placed in libraries instead
of compilers.

Also bounds checking interfaces are a performance loss and not included in C
compilers despite them being part of the C11 standard. (Well they're optional)

------
bjourne
Performance. Rust is still twice as slow as C
([http://benchmarksgame.alioth.debian.org/u64q/performance.php...](http://benchmarksgame.alioth.debian.org/u64q/performance.php?test=spectralnorm))
which is still a fair bit slower than if a skilled assembly programmer had
taken on the task.

Rust aficionados will say that their compiler is getting better, but so is C.
clang has gotten faster than gcc on some benchmarks and on some others gcc has
catched up and is now faster than clang again.

But what if you don't need optimal performance? Then you can use Rust. But
then you can also use Go, Python, SBCL, Haskell, Java, C#...

~~~
valarauca1
This difference on this test is caused by Rust not having stabilized SIMD
support. Also Rust support hand rolled assembly (on nightly) that C has.

On non-SIMD tasks Rust/C are neck and neck
[https://benchmarksgame.alioth.debian.org/u64q/rust.html](https://benchmarksgame.alioth.debian.org/u64q/rust.html)

You're just cherry picking benchmarks. In the cases you _care_ about raw
number crunching power you'll likely be using a GPU not SIMD instructions as
CPU's are roughly 3-4 orders of magnitude slower then GPU's at pure number
crunching tasks.

Not that SIMD isn't important as it's instructions also cover things like AES,
SHA1/2, Random numbers, Cache pre-loading/evacuation, memory fences, and fast
loading paths. But so few programmers worry about these things you are really
hitting a niche market.

~~~
bjourne
> This difference on this test is caused by Rust not having stabilized SIMD
> support. Also Rust support hand rolled assembly (on nightly) that C has.

Cool. Let's call that language with SIMD and inline assembly support
FutureRust(tm) to differentiate it from the currently released and available
Rust. We can have a discussion about how fast FutureRust will be vs C, but
this discussion is about Rust vs C. Or rather clang 3.6.2/gcc 5.2.1 vs Rust
1.9.0 since language performance is very implementation dependent.

> On non-SIMD tasks Rust/C are neck and neck
> [https://benchmarksgame.alioth.debian.org/u64q/rust.html](https://benchmarksgame.alioth.debian.org/u64q/rust.html)

In 5 of 10 benchmarks, C is twice as fast as Rust. In one of the benchmarks
where it is neck and neck, like pidigits
([https://benchmarksgame.alioth.debian.org/u64q/performance.ph...](https://benchmarksgame.alioth.debian.org/u64q/performance.php?test=pidigits))
it appears to be so because both the C and the Rust variant are wrapping
libgmp. GMP is written in C.

~~~
valarauca1
>In 5 of 10 benchmarks, C is twice as fast as Rust

fannkuch-redux why? SIMD

fasta-redux why? SIMD

spectral-norm why? SIMD

reverse-complement why? SIMD

N-Body why? Oh you guessed it SIMD

Seriously read the source code. Remember on HN where a lot of people
constantly say the benchmark game is really crappy. This is why. All 5 of
these tests boil down to raw FLOPS. Which C/C++ having access to SIMD
instructions wins at.

The fact that Rust/C performance difference works out to just the ability to
emit vector instructions says a lot about everything else in Rust. The fact
that Rust can dereference, pass variables on the stack, call functions, and
make decisions as fast as C renders your core point completely moot.

You are just being incredibly pedantic for no reason. And your argument holds
no water. Everything Rust does is identical to C except one barely used corner
case. They use the exact same model for computation, they both live in the
Cee-LangVM. Post compilation they are functionally identical (except Rust
makes stack manipulation easier).

Does any of that make sense to you?

:.:.:

Also Rust/C both calling the GMP without a time difference is a good thing.
The Rust->C FFI is literally non-existent in practice. Dipping into C code
from Rust (and vice versa) has no penalty. The same can't be said for HUNDREDS
of languages.

~~~
bjourne
Rust is also slower in binarytrees, regexdna and fasta. SSE is not one "barely
used corner case" because huge amounts of performance critical code takes
advantage of it.

Edit: To explain why I don't believe you when you say that "Post compilation
they are functionally identical [in performance]" is because if it were so,
you would just transliterate the C solutions to the Rust equivalents and it
would run as fast as C. Since that hasn't been done and is trivial to do, my
conclusion is that it doesn't lead to the same performance.

~~~
burntsushi
Did you know Rust was quite a bit faster than C in regexdna merely a few
months ago? It didn't get slower because of Rust. The algorithms employed are
radically different. My hope is that the regex library has already regained
performance, but until the benchmark game is updated (which is on us, not the
benchmark game maintainer), I suppose we'll have to suffer the pedants!

Or perhaps, you might look at single threaded performance and wonder, maybe
there is something more interesting going on than a naive surface analysis of
C vs. Rust! :-)
[https://benchmarksgame.alioth.debian.org/u64/rust.php](https://benchmarksgame.alioth.debian.org/u64/rust.php)

And by the way, transliterating a regex library isn't trivial. I invite you to
transliterate Tcl's regex library. Let me know how that goes. ;-) So I think
your reasoning is specious at best.

~~~
igouy
> It didn't get slower because of Rust.

Do you mean the program became relatively slower because of changes you've
made to the regex crate?

Wasn't the program relatively faster because you wrote the regex crate to use
Aho-Corasick for the matches required by the regex-dna task?

~~~
burntsushi
> Do you mean the program became relatively slower because of changes you've
> made to the regex crate?

Yes. The underlying reasoning is complex. When the regex crate got a lazy DFA
(similar to the one used by RE2), the vast majority of regexes got
significantly faster. Some got slower. This one in particular from regex-dna:

    
    
        >[^\n]*\n|\n
    

Before the lazy DFA, compile time analysis would notice that all matches
either start with `>` or `\n` and do a fast prefix scan for them. Each match
of `>` or `\n` represents a candidate for a match. Candidates were then
verified using something similar to the Thompson NFA, which is generally
pretty slow, but the prefix scanning reduced the amount of work required
considerably.

Once the lazy DFA was added, the prefix scanning was still used, but the lazy
DFA was used to verify candidates. It's faster in general by a lot, but, the
lazy DFA requires two scans of the candidate: one to find the end position and
another to find the start position. That extra scan made processing this regex
(on the regex-dna input) slightly slower.

I've since fixed some of this by reducing a lot of the match overhead of the
lazy DFA, so my hope is that it's back to par, but I haven't done any rigorous
benchmarking to verify that.

> Wasn't the program relatively faster because you wrote the regex crate to
> use Aho-Corasick for the matches required by the regex-dna task?

Aho-Corasick is principally useful for the second phase of regex-dna, e.g.,
the regexes that look like `ag[act]gtaaa|tttac[agt]ct`. (In the last phase,
all the regexes are just single byte literals, so neither Aho-Corasick nor the
regex engine should ever be used.) Performance here should stay the same.

On that note, I have a new nightly-only algorithm called Teddy that uses
SIMD[1] (which replaces the use of Aho-Corasick for those regexes) and is a
bit faster. I got the algorithm from the Hyperscan[2] project, which also does
extensive literal analysis to speed up regexes.

To clarify, this optimization is generally useful because a lot of regexes in
the wild have prefix literals. Even something like `(?i:foo)\s+bar` can
benefit from it, since `(?i:foo)` expands to FOO, FOo, FoO, Foo, fOO, fOo,
foO, foo, which can of course be used with Aho-Corasick (and also my new SIMD
algorithm).

One also must wonder how well a C program using PCRE2's JIT would fair on the
benchmarks game. From my experience, it would probably be near the top. It's
quite fast!

[1] - [https://github.com/rust-lang-
nursery/regex/blob/master/src/s...](https://github.com/rust-lang-
nursery/regex/blob/master/src/simd_accel/teddy128.rs)

[2] - [https://github.com/01org/hyperscan](https://github.com/01org/hyperscan)

~~~
igouy
> One also must wonder how well a C program using PCRE2's JIT would fair on
> the benchmarks game.

Let's hope some C and C++ programmers take up the challenge ;-)

------
Scarbutt
_Rust reduces the amount of state I need to keep track of in my brain._

I doubt it, the mental overhead of doing "safe memory programming" in Rust is
very high.

Edit: all good replies, want to clarify and forgot to mention that I was
comparing to languages with a GC, since I'm seeing Rust being used for lots of
stuff, in a general purpose programming language sense (like creating web
frameworks for example). Also, for non-very-low-level stuff I guess this
cognitive load will be less if/when they introduce an optional GC.

~~~
gue5t
Designing memory-safe programs in C requires a programmer to reason about the
same domains as doing so in Rust, but C doesn't double-check you to make sure
you get everything right. With no guard rails, C is a lot more stressful.

Re: reducing mental state for a programmer, algebraic datatypes in general
decrease the size of the state space of your program by making many illegal
states unrepresentable. Without advanced forms of dependent types (maybe
quotients), you can't make _all_ illegal states unrepresentable, but you
shrink the size of the state space hugely compared to writing everything as
product types (as you would in C). A programmer has to reason about all the
possible values their variables can take on, so it pays to minimize the
cardinality of that set.

~~~
mfukar
Why are we throwing away all the work done on static & dynamic analysis tools
for C programs in this kind of discussions? Programmers are crippled just for
picking C? Come on..

~~~
Manishearth
Because ultimatelty they're imperfect.

Sure, C+static analysis is good enough for many situations. But it can't
compare with the guaranteed safety offered by Rust.

~~~
mfukar
> Because ultimatelty they're imperfect.

Everything is imperfect, it's not a good reason to discount anything.

