
Rust and the Future of Systems Programming [video] - philbo
https://hacks.mozilla.org/2016/11/rust-and-the-future-of-systems-programming/
======
pimeys
Now I have four services running on production, all written with Rust. If it
compiles, it usually works. Of course you have these late night sessions where
you write that one unwrap() because, hey, this will never return an error,
right? And bam...

I'm seriously waiting that tokio train to be stable and a unified way of
writing async services without needing to use some tricks with the channels or
writing lots of ugly callback code. Also the native tls support is coming and
the dependency hell with openssl would be gone forever.

If you need http server/client, I'd wait for a moment for Hyper to get their
tokio branch stable and maybe having support for http2 by migrating the
Solicit library.

~~~
tmzt
Would it make sense to have a cargo/rustc flag to disable unwrap and friends
when building for production?

~~~
wyldfire
"Pray, Mr. Babbage, if you put into the machine wrong figures, will the right
answers come out?"

Unwrap implies "either this operation should succeed or we should panic".
Doing anything other than panic seems like it's terribly difficult to
determine what that should be. And I might've put unwrap() there because
that's really what I want to happen. For `mv`: don't let's dare go ahead and
do the unlink() if the link() failed!

Tangent: errors and exceptions aren't bad, they're how computers work. I've
encountered folks who, when faced with runtime errors pepper their code with
"if (!NULL)" or truly evil things like "except:pass"/"catch (...) { }" which
rarely make sense anywhere but the base of the stack and even then don't
usually. If you've ever asked yourself "but how did we even get here‽" it may
be because someone dropped something totally incongruous like that in a
related module.

~~~
rwj
I think the idea would be the program would fail to compile, and you would
need to go back and replace the unwrap with proper error handling. The goal
would be to allow the use of unwrap during development, but require the final
polish before the code goes into production.

~~~
josephg
unwrap _is_ proper error handling. It says "Try to do this. If it fails,
panic". Its like an assert on an invariant that the compiler requires.

If my script depends on a database connection, I might connect to a database
and unwrap() it so the script errors out if the database isn't available. If I
wrote that logic myself I would just be awkwardly rewriting unwrap.

~~~
kibwen
To be more accurate, unwrap can be a legitimate means of error handling when
used in an application, as opposed to a library. But if you're writing a
library, then unwrapping rather than using Result is a surefire way to make
your users hate you. :)

~~~
Nemo157
Sometimes you've proven some invariant in some other way, so you know that
unwrapping is guaranteed to not panic. Although in those cases I prefer to use
.expect("this will not fail because of blah") instead in the spirit of self-
documenting code.

~~~
kibwen
Agreed, I use `.expect("Infallible")` for such cases.

------
wyldfire
> GC pause ... sufficiently low power hw .. cheap phone

Yeah, but even high-powered hardware can take a "major" hit from a GC pause
when your application is extremely latency sensitive.

IMO it would be great to get folks who write the enormous base of existing
realtime apps driving critical devices everywhere to sit up and take notice of
Rust.

EDIT: I mean to say that many of my colleagues who write realtime software
dismiss new languages as including GC baggage by default (because so many
do!). So, hey, good that the video calls this out.

~~~
naasking
> IMO it would be great to get folks who write the enormous base of existing
> realtime apps driving critical devices everywhere to sit up and take notice
> of Rust.

Rust cannot make any latency guarantees either. Reference counting and its
lifetimes also have pathological cases, ie. worst-case, an object can
reference the entire heap which will take time proportional to the number of
dead objects to free.

Copying collection in this case takes literally zero time, but it's
pathological case is when all referenced data survives the current GC cycle,
ie. proportional to live objects.

There's no free lunch!

~~~
Manishearth
> Rust cannot make any latency guarantees either. Reference counting and its
> lifetimes also have pathological cases, ie. worst-case, an object can
> reference the entire heap which will take time proportional to the number of
> dead objects to free.

Rust doesn't use reference counting by default. Refcounting is _very_ rare in
Rust, much more rare than it is in C++. Most large C++ codebases I've worked
with have thrown in the towel and started refcounting all the things. In
Servo, for example most of the refcounting is across threads (where you
basically have no other option), and a few interesting cases in the DOM, each
with very good reasons for using refcounting.

Lifetimes are a concept at compile time and don't exist at runtime.

Edit: Oh, I see what you're talking about. A sufficiently large owned
tree/graph in Rust will introduce latency. It's predictable latency though. I
can make the same argument about for loops.

Unpredictably sized large trees in Rust are again pretty rare in general.

~~~
Animats
Trees don't have to be refcounted in Rust. Single-ownership trees are
possible. As long as they don't have backpointers. Backpointers are a problem
under single ownership.

~~~
Manishearth
Right, I never said that trees have to be refcounted. A sufficiently large
ownership tree will get deallocated all at once, which is the kind of latency
the GP is talking about.

~~~
Animats
Linked data structures in Rust get complicated, though. See the "Too many
lists" book.[1] Doubly linked lists, or trees with backlinks, are especially
difficult. Either you have to use refcounts, or the forward pointer and
backward pointer need to be updated as an unsafe unit operation. There might
be an elegant way to do this with swapping, but I'm not sure yet.

[1] [http://cglab.ca/~abeinges/blah/too-many-
lists/book/](http://cglab.ca/~abeinges/blah/too-many-lists/book/)

~~~
Manishearth
Right, so you implement them with unsafe. While you can implement doubly
linked lists safely with refcounting, you're perfectly free to implement them
with unsafe code. This is what unsafe code is _for_ , designing low level
abstractions with clean API boundaries.

(Also I don't see how this is relevant at all)

~~~
Animats
_Right, so you implement them with unsafe._

If you need unsafe code for basic operations within the language, something is
wrong with the language. This isn't about talking to hardware, or an external
library. It's pure Rust code.

(Some pointer manipulations can be built from swap as a basic operation. That
may work for doubly-linked lists. The other big problem is partially valid
arrays, such as vectors with extra space reserved. There's no way to talk
about that concept within the language. There could be, but this isn't the
place to discuss it.)

~~~
kazagistar
> need unsafe code for basic operations within the language

Building custom back-referencing data structures is not a "basic operation"
anywhere outside programming classes. Adding significant complexity to rust
Rust to make a 2% case marginally safer would be make the language worse. As
long as the vast majority of code is not unsafe, then it achieves its goal.

------
djhworld
I keep trying to learn rust but fail miserably.

They do say on their website that there's a hump that you have to climb over
before everything fits into place, which is probably applicable to everything
you'll learn, but sometimes I think that hump is too much of a hurdle

~~~
steveklabnik
How are you trying? What are you getting stuck on? I'd love to improve things.

~~~
cbHXBY1D
Not specifically about your book, but I would love if there was a quicker way
to find methods in the docs.

Right now, if I want to find the methods used by BTreeMap you have to wade
through a good amount of information until you can find how to just get the
keys. I'm currently on mobile where the issue is more prominent.

~~~
wyldfire
If I don't find what I'm looking for from the docs, I often use ripgrep on a
copy of the rust repo locally to find answers.

~~~
travv0
> I often use ripgrep on a copy of the rust repo locally to find answers

I mean, that works, but it's a workaround and not a solution.

------
harveywi
The state of Rust editors continues to evolve [1], but I would be curious to
learn more about the editors/IDEs that people are using for Rust development.
Any stories or thoughts?

[1] [https://areweideyet.com/](https://areweideyet.com/)

~~~
allengeorge
I'm using IntelliJ-Rust [1]. Doesn't go to the definition of macros, and
sometimes auto-complete doesn't work, but - it's comfortable enough to use.

[1] [https://intellij-rust.github.io/](https://intellij-rust.github.io/)

------
leshow
Rust is great. I've been following it since pre-1.0 and writing code with it
for about as long also. It's really come a long way, my favorite language for
pretty much anything except web development.

~~~
Narann
Wow! You've got my attention!

I mean, I feel many rust stuff (crates/dev/tools) seems to focus on high
performance web so, as a low level guy enjoying bitwise ops and dynarec stuff
I was wondering if rust what a good thing for me or if it would be better to
stick in C. Can you tell me which kind of project you do in rust, what was
your original language and why rust shine compared to your previous language?

~~~
leshow
I started on Java in school for software eng. After that I did a lot of front
and backend development in dynamic languages and forgot all about types.

Until I discovered Haskell and saw what a really powerful type system can do.
Rust's type system is very very similar to haskell's if you substitute the
word 'trait' for 'typeclass'. However code is imperative, so that's nice for
some applications, and is faster/lower level.

I've used the crate nom
([https://github.com/Geal/nom](https://github.com/Geal/nom)) in the past, it's
a parserc library for consuming bits/bytes. I used it to write a function that
consumes a vector of unsigned bytes, 11 bits at a time. If you're interested
in that sort of stuff, take a look for yourself and see if you like it.

As with most things in Rust nom generates parsers with little to no runtime
overhead, so you pay nothing for the increase ergonomics.

------
htaunay
Congrats to all the Rust team!

Love the zero-cost abstraction approach, love the good practice compiler
warnings, love the cargo ecosystem, love the markdown code dox that tests the
example code.

First time I actually enjoyed reading a programming language's documentation.﻿

------
CodeMage
I'd love to see someone write a game engine in Rust to compete with the "big
boys" like Cry or Unreal. C++ game code can be such a nightmare.

~~~
Thaxll
Not going to happen in the next 10 years, everything is built around C++. And
tbh I don't see what would games benefit from using Rust instead of C++.

~~~
cderwin
I've never worked on anything gaming related, but the bigger cross-platform
games certainly have crashes. I can't even tell you how many times Fallout 4
has crashed for me (likely hundreds). I obviously don't know if rust could
help, but it's not unreasonable to think it might.

------
shmerl
Does Emscripten support Rust already?

~~~
wyldfire
Yes. See [1] for details.

[1] [https://users.rust-lang.org/t/compiling-to-the-web-with-
rust...](https://users.rust-lang.org/t/compiling-to-the-web-with-rust-and-
emscripten/7627)

~~~
etherealG
Thanks so much, I was digging into this a few weeks ago eagerly and didn't
find anything. Glad it's here now!

------
anfroid555
Anyone know a tutorial course for someone only knowing high level languages.
And not c or Ruby. More JavaScript, PHP, nodejs, Python

~~~
oldsj
Rust for rubyists?

~~~
steveklabnik
It basically doesn't exist anymore; I stopped maintaining it pre-Rust 1.0.

------
mavsman
Coincidence that many of the Rust promoters have red hair?

------
user5994461
I'd like to see an analysis: "The amount of real world jobs available for
Rust, in comparison to Ada."

The numbers may be self explanatory.

~~~
gcp
What would they explain though?

~~~
wyldfire
They would explain Rust's relative youth and corresponding growth opportunity.
;)

~~~
user5994461
Ada was designed for system programming and, at a time, was also a young
language with a huge opportunity for growth ;)

~~~
staticassertion
To compare Ada's history to Rust's is silly. Even if the languages have a
common goal of safety, the projects themselves are fundamentally different in
many many ways. One of which is the fact that Ada was designed by the DoD for
the DoD, and required a verified compiler, etc etc etc.

They're totally different projects, even if the languages have a small number
of shared goals.

------
VoidAgent
IMO C++ is unstoppable now, c++ 11, 14 and 17 additions with GSL and all
things in the pipeline ...

~~~
kibwen
Rust doesn't need to stop C++ to be successful. The market for programmers is
already large enough to successfully support dozens of programming languages,
and that market will only continue to grow. PHP didn't "stop" Perl, Python
didn't "stop" PHP, Ruby didn't "stop" Python, and yet all of these languages
continue to be enormously successful. Why should we expect a monoculture on
the systems side?

~~~
user5994461
What's the place for Rust in the market?

The distributed systems have already moved to Go (when it's not Java/C# :D).

The system programming is done in either C or C++. (Depending on history and
availability).

The oldest stuff and/or most constrained is stuck with C. They're struggling
to have anything moved over to C++. Rust is entirely out of question.

~~~
takeda
Rust overlaps with C/C++ and even with Go. Go was initially trying to be a
replacement for C/C++, but hasn't succeeded, because it is too high level.
Rust on the other hand looks very promising in that area, which requires low
level access but also desires safety.

~~~
Matthias247
I wouldn't say Go hasn't succeeded. Maybe not as a general replacement for
C/C++. But still lots of new applications that would most likely have been
written in C/C++ a few years ago are now popping up in Go (unix daemons,
commandline tools, ...).

~~~
user5994461
Go has succeeded in its niche (distributed systems). There are real jobs
opportunities in Go, in the major tech hubs.

Funny thing is. Everything that is done in Go could be done in Java. Go is
succeeding because lots of dev despise Java (related to the entreprisey & the
usual culture of java companies).

~~~
donatj
As well as the fact that Go runs markedly faster and lighter (memory) than
Java, and without needing to worry about JVM versions. We've been slowly
migrating services and the statically compiled binaries are truly amazing for
ensuring it will run. Not once have I encountered an 'it runs on my machine'
situation.

------
hubert123
still compiles too slowly, a language in 2016 just cannot take multiple
seconds for me to use it add 5 crates and watch ur compile/run cycle climb to
a cool 10+ second average.

~~~
steveklabnik
We had significant compile-time improvements in the release yesterday, with
more to come in the future. Also, you might want to give incremental
recompilation a try, it's nightly-only for now, but being actively worked on.

> add 5 crates and watch ur compile/run cycle climb to a cool 10+ second
> average.

It should not recompile those crates each time, if it does, that's a bug.
Please report them!

~~~
nnethercote
> It should not recompile those crates each time, if it does, that's a bug.
> Please report them!

Depends on the inter-crate dependencies, alas. Touching one crate can easily
cause multiple other crates to rebuild.

~~~
steveklabnik
Only if you need to modify both crates; not if you're just adding them as
dependencies, because then you're not modifying them. I was assuming that's
what the parent was doing, due to their wording, but that might have been a
bad assumption.

------
deavmi
I agree. We need safer languages. Not that I know much about safety. I still
know we need it.

------
cryptrash
I'll never use rust for anything important. Too dangerous, unstable, badly
organized, toxic development community, the list goes on.

~~~
shmerl
Huh? Did you mix it up with something else? Rust community is one of the best.
And what's "dangerous and unstable" there?

~~~
noir_lord
Check his comment history and draw your own conclusions ;).

~~~
gbersac
First time I saw someone with negative karma.

------
z3t4
when you can run javascript on a pebble watch it blurs the boundaries ... Want
to concat an number and string ? JavaScript wont complain.

~~~
0xAA55
Want to make a _fast_ application on a pebble and with actual type checking? C
and Ada won't complain or blur your lines... [http://blog.adacore.com/make-
with-ada-formal-proof-on-my-wri...](http://blog.adacore.com/make-with-ada-
formal-proof-on-my-wrist)

~~~
z3t4
And _real_ programmers write in machine code. Want to get shit done ? Most
systems now a day has more then 20kb RAM! Where do you draw the line between
systems programming and non system programming ? And why not write some quick
and dirty code in say JavaScript and then do what needs optimization in
C/assembly ? Assuming you are not restricted to a CPU that cost less then a
dollar. And where does Rust come in ?

~~~
Retra
What kind of engineer goes around saying "It could be faster, leaner, and more
efficient? No! Build the biggest, flakiest, ugliest thing that gets the job
done today!"

------
quotemstr
I lament what Rust could have been had its designers not jumped on the anti-
exception bandwagon. Rust's error handling is bad and makes me prefer C++

~~~
IshKebab
Seriously? Rust's error handling is great, and frankly I'm glad that
exceptions have gone out of favour. I tried them but they never really
delivered their promise. At their best they do is give you nice stack traces.
At their worst they make error handling stupidly verbose, they erase the
context you need to properly handle errors, and they make it much more
difficult to even know which errors can occur!

Rust's solution is the best I've seen so far. Go's multiple return values are
pretty good too but I think Rusts's is better.

~~~
quotemstr
I very strongly believe exceptions are a lot closer to optimal than Result is.
Exceptions remove the need for inline error checking code and make it possible
to implement types with value semantics. You can't have reasonable value types
that own resources if you need explicit error checking.

The need for explicit error checking makes OOM handling in Rust awkward at
best. I've ranted about this side effect before.

Also, in Rust since we can panic, we need to worry about exception anyway!
Rust has the worst of both kinds of error handling.

Exceptions are _anti_ verbosity: you're supposed to let them propagate, not
catch and rethrow them. They don't erase context: they preserve it, since an
exception object is constructed as close as possible to the error site that
caused it.

I realize that I have a minority view, but I'm utterly convinced that I'm
right and that I'm living in a world gone mad. I've written a ton of code over
the years. At least hear me out and try to understand my perspective.

~~~
zubat
I think I know what this trend is, more generally. It has to do with how
explicit our code is. We've been through an era where you have some very
powerful and compact indirection constructs(event callbacks, polymorphic
objects, dynamic types, exceptions) in common parlance and the trend has
turned against these lately. Their utility in many instances is mostly to
enable technical debt, by worrying about the edge case later, and this has
scared a generation of coders who have seen it go astray too often and create
code that is hard to usefully refactor. And while exceptions aren't at the
center of that trend, they're often implicated as a contributor.

With the new trend, you pay up front and go verbose, put more logic at the
call site instead of indirecting it away. Go is at the leading edge of that:
lots of LOC for boilerplate is OK in Go-land.

I'm more pragmatic with my error handing methods: I mostly care about whether
I can eliminate a class of errors altogether, and secondarily what debugging
implications are presented. I don't have a strong opinion on verbosity
although I have followed the trend in that respect towards more call site
logic.

~~~
quotemstr
This trend represents the unlearning of very hard earned lessons. Personally,
I can't wait until people rediscover that programming can be more fun and
productive without boilerplate. A language being explicit, by itself, is not a
feature. Being explicit instead of implicit is only worthwhile if you get
clarity in exchange, and boilerplate is clarity-reducing because it's so
regular and so obscures program logic.

~~~
burntsushi
In Rust, in the common case, the boiler plate you're talking about here is
literally a single sigil. (Previously, it was `try!(...)`.)

This is of course to say nothing about the _advantages_ of having something
that signals "this operation can return an error," (at the call site) but you
seem to dismiss that out-of-hand.

~~~
quotemstr
I do dismiss this advantage out of hand: almost everything can fail, because
most things allocate memory. It's only because Rust treats memory specially
(incorrectly --- memory is just another resource) that it doesn't appear that
more functions can fail. It's much more valuable to flip the sense of the
annotation and mark the few functions that cannot fail in any way.

Allowing most code to throw just reflects reality and allows you to stop
obscuring your program logic with mechanical error handling plumbing.

All you do with "try!" is annoy readers by constantly reminding them that
things can go wrong when the default assumption should be that things can go
wrong.

~~~
saurik
(I just want to tell you somewhere that you are not alone, and that I
appreciate the time you are taking to have this conversation. The arguments
you are making are the exact same things I am often found saying, and in my
experience it takes hours of time alone in front of a blackboard with someone
to really get them to understand concept like "everything can fail" and
"memory is no different than disk space". The one thing I haven't seen you
argue with yet is that functions which are adamant they can't fail today often
find themselves in a position where they can fail tomorrow, such as by adding
authentication or storage; this makes "explicit" error handling a limiting
contract that either requires you to proactively return error codes from
everything even when they only currently return Success, never extend the
functionality of existing systems to avoid accumulating an error condition, be
prepared to break that contract in the future, or fall back really really hard
on panic.)

------
Animats
Unfortunately, this is just a promotion for a series of videos. Probably
talking head videos with PowerPoints.

Most people who do systems programming probably already know about Rust. They
may not have used it, but know it exists.

What's "Hacks"?

~~~
steveklabnik

      > Probably talking head videos with PowerPoints.
    

Almost everyone in this video is an engineer on Rust, Servo, or Firefox. The
exception is Dave Herman, head of Mozilla Research.

These are unscripted responses to questions, no powerpoint involved.

    
    
      > They may not have used it, but know it exists.
    

There are a lot of programmers in the world, and many of them don't read
Hacker News. I meet new people who have never heard of Rust every week almost.
Or may have heard of it, but can't tell you what it's for, just vaguely
remember hearing the name. Or have heard of it, but remember their first
impression from a pre-1.0 version and haven't listened since. ("How's its
GC?")

~~~
kibwen

      > but remember their first impression from a pre-1.0 version
    

Can confirm, I routinely observe people in the wild dismissing Rust based on
the existence of the `@` and `~` sigils, not realizing that we removed those
years ago. :P

~~~
wyldfire
I dismissed it when I first heard about it (prob just prior to 1.0). I saw
"Mozilla" as a primary sponsor and just knee-jerk assumed it was part of the
what-if-we-put-JS-here mania.

Thankfully due to repeated popularity on HN I gave it a try and I'm a convert
now!

------
enqk
Although I am not against improving the safety of languages we use for system
programming, the model that Rust advocates are pushing of "preventing
mistakes" as a way to make systems secure doesn't convince me.

Mistakes (and security breaches) happen. A system should be written in such a
way as mistakes are few, indeed, however it is essential to also efficiently
protect our users' assets (data) first, assuming that mistakes are being made.

Ideally as a user I should not care much, nor can I trust that all software is
"mistake free." I need the assurance and the tools to guarantee that 1)
mistakes are detected 2) my data/privacy/identity is protected.

~~~
gcp
Nothing prevents you from doing both approaches. I mean, while Mozilla is
increasing Rust usage, they're also rolling out more extensive sandboxing.

Either of those by themselves are not a good enough solution though. Just
trusting on Rust means you are vulnerable to Rust bugs or logic errors. Just
trusting on sandboxing means you're hoping that your trusted code (written in
C/C++) doesn't have any security bugs.

~~~
sidlls
It also means trusting that the "unsafe" usage in the Rust code doesn't have
any security bugs.

~~~
kibwen
Indeed, though trusting unsafe blocks in Rust is more tractable than trusting
an entire C/C++ codebase due to the fact that unsafe blocks present a
drastically reduced auditing surface.

~~~
sidlls
That's the contention, yes. In practice currently there is an incredible self-
selection bias with respect to how tractable that position is if (when?) Rust
becomes more widely used outside the circle of the True Faithful.

