
Shipping Rust code in Firefox - dwaxe
https://hacks.mozilla.org/2016/07/shipping-rust-in-firefox/
======
jgraham
Not in the same league as the mp4 parser shipping to all Firefox users, but
GeckoDriver [1], a Mozilla-authored standalone binary for interacting with
Firefox via the WebDriver protocol (e.g. using Selenium) is also written in
Rust and shipping, possibly to as many as hundreds of users ;)

Overall the experience of using Rust for that project has been pretty great;
the original requirements for a language were "able to produce static
executables with no external runtime requirement, not prone to C-style memory
safety issues, and accessible to a team with mostly Python backgrounds". That
leaves quite a few options — Go for example — but being a Mozilla project
taking the opportunity to use Rust seemed like it would align better with
initiatives like those discussed in the article.

Apart from the general niceness of Rust-the-langauge, the experience of the
Rust ecosystem has been really nice. Not only were there easy-to-install, well
documented, packages to do much of the heavy lifting (http server, command
line argument handling, etc.), it was also quite straightforward to set up
travis builds using cargo to compile releases for linux64+musl osx and cross-
compile releases for linux-arm7hf (added because someone asked about running
on raspberry pi and it turned out to be trivial) and win64. The only wrinkle
so far has been the difficulty of cross compiling to win32, meaning we might
actually need to set up Appveyor or similar.

Obviously this is a much smaller project than the original post in terms of
the number of users, and rather different as it is much higher level and could
comfortably be written in Python or similar, except for the distribution
requirements. However I think the use of Rust has been an unmitigated success,
and I would be tempted to use it for a lot more projects that I would never
consider writing in C/C++.

[1]
[https://github.com/mozilla/geckodriver](https://github.com/mozilla/geckodriver)

~~~
mkesper
How did you handle the Python to rust knowledge transition? Do you know any
helpful guides?

~~~
jgraham
I don't think I have any specific resources to recommend, sorry.

I wrote most of the initial implementation of geckodriver and mostly learnt
Rust from reading the official book, writing some small patches for Servo, and
working on this project (of course); pretty much the same advice you would get
coming from any other language.

I just chatted with one of the other major contributors to the project who
said that he was able to learn on the job by reading the existing code,
looking up concepts in the book or via web searches, and asking questions when
necessary; although some parts of Rust undoubtedly have a significant learning
curve it is empirically possible to contribute to an existing project without
a significant amount of upfront study. This arguably isn't too different from
learning most languages, although you are going to need to read up on the
rules around e.g. references and borrowing sooner than you would need to look
something up if you took the same approach to writing your first Python.

The type system and borrow checker mean that the compiler will tell you if
you're doing something wrong which is both a blessing and a curse; it can be
dispiriting to get dozens of compile errors when you are getting started, but
once you have satisfied the compiler it's possible to be more confident that
your code won't break in ways that are relatively common in Python (e.g.
missing or broken code for error handling).

So I'm not sure that I answered your question, but in practice it didn't seem
to be a major problem. Of course in a different environment — one where people
were less enthusiastic about learning a technology their colleagues were
raving about, for example — you might have a different experience.

------
nachtigall
Rust (like Let's Encrypt) is a great thing coming from Mozilla and I'd like to
thank them.

Edit: "Coming from" in the sense of bootstrapping great independent
communities around these.

------
gregwtmtno
Even if Servo doesn't pan out as anything more than an experiment (though I'm
optimistic that it will), I think Rust will bring real benefits to Firefox by
enhancing security of isolated components like this mp4 parser.

~~~
kevincox
If you look on the Mozilla bug tracker you can look at a number of tickets
open for integrating servo components into Firefox.

For example Gecko is gaining support for Servo's style engine so maybe CSS
will be the first large component merged.

I wonder if Firefox will slowly become written in mostly Rust.

~~~
gregwtmtno
> I wonder if Firefox will slowly become written in mostly Rust.

I doubt that will ever happen. Small parts of Firefox, yes, but the browser is
enormous. I think I once read that even Servo, which is a showcase for Rust,
has more C/C++ code in it than Rust code, largely because it uses Firefox's JS
engine.

~~~
bluejekyll
> I think I once read that even Servo, which is a showcase for Rust, has more
> C/C++ code in it than Rust code

Not in the core project:
[https://github.com/servo/servo](https://github.com/servo/servo)

~~~
lovelettr
Where is the feature in GitHub that would show the languages used in a
repository? That would be useful here regarding this comment.

(Note I have no doubt you are right. It would just be a useful metric.)

~~~
whateveracct
Did they just..get rid of it? I can't find it either.

~~~
szatkus
No, it works for other repostiories.

Probably Github uses cloc. I downloaded the code (over 500MB, insane!) and
cloc throws an error.

~~~
steveklabnik

      > and cloc throws an error
    

tokei is a similar program that's parallel, and written in Rust. It takes 11
seconds to run on my machine, and shows

[https://gist.github.com/steveklabnik/b4ede6f13c9d609edc61d74...](https://gist.github.com/steveklabnik/b4ede6f13c9d609edc61d7421e0af1b1)

(using a gist since the output is huge, and see Manish's comment as well)

~~~
Manishearth
This is including the cargo cache dir?

~~~
steveklabnik
I built Servo last week, I ran "git fetch upstream && git reset --hard
upstream/master && git submodule update" and then "tokei .". So, unless Servo
puts its cache in the dir, I would guess not, no.

~~~
Manishearth
I think by default servo does put the cache in the dir. So yeah, this would be
an accurate estimate.

Though not all that rust code is written specifically for servo, and a lot of
that C/++ code is winapi and skia. winapi is autogenerated, and skia isn't
used by default anymore iirc.

------
penagwin
AHHH Telemetry! I feel like there is a group of people who freak out to any
form of telemetry, I just wanted to highlight them using it to track bugs.
It's super beneficial to any form of changes, especially something like
switching an entire language for a component.

~~~
Manishearth
IIRC the telemetry involved here was only run on the nightly/developer edition
versions. Or something like that.

The restrictions on telemetry on release versions are pretty stringent. It's
possible this was measured on release, I'm not aware of the details, but I
suspect it wasn't.

~~~
kevincox
From my understanding release is opt-in only.

(My understanding is based upon whenever I install Firefox it promts me to
enable telemetry, which is disabled by default)

~~~
cpeterso
You are both correct. :)

Firefox telemetry is opt-out on the Nightly, DevEdition, and Beta channels,
but (mostly) opt-in on the Release channel. The opt-in telemetry from the
Release channel has limited use because it is not representative of all users.

Here is a list of Firefox's telemetry measurements. Those that are opt-out on
the Release channel are tagged _" releaseChannelCollection": "opt-out"_.
Adding a new opt-out measurement requires an additional privacy review.

[https://hg.mozilla.org/mozilla-
central/file/tip/toolkit/comp...](https://hg.mozilla.org/mozilla-
central/file/tip/toolkit/components/telemetry/Histograms.json)

------
radarsat1
Does this imply anything regarding rust compiler package for distros? To
support the Firefox build, all distros will have to have good rust compiler
packages now, right? (I'm sure the status is already not bad, I haven't
checked recently..)

If it becomes something dependable, infrastructure-wise, like Java, this might
mark the beginning of a more serious uptake.

~~~
steveklabnik
We've been doing a lot of work in this area, and some distros already have
packages. Debian testing, for example.

Currently, I believe that this functionality is optional, so distros aren't
required to actually ship it just yet.

~~~
snuxoll
Now that Rust can properly bootstrap from previous versions I can see most
distros getting it packaged soon(TM), unfortunately the Fedora 25 change
window ends today and since rust/cargo will require new packaging guidelines
unless someone really steps up to the plate (I don't have time today) it won't
make it in until F26.

~~~
djsumdog
SBT (scala) bootstraps itself in the same way. I think we're going to be
seeing more of that going forward.

For the produced binary packages (rpms, debs) it won't matter that much;
although it will reduce build dependencies for source packages and give more
consistent builds, even if your distribution comes with an _ancient_ version.

~~~
avita1
That's not a complete comparison though -- sbt only downloads the platform-
independent jars and minuscule launcher scripts. It's still on the user to
install a JRE globally.

An analogue for rust would be something that detects the platform at runtime
and then downloads a statically linked (?) binary that works on that platform.
It can also exploit the ivy cache as the Scala compiler is just a library as
far as SBT is concerned.

Isn't that a deal breaker? I would assume that downloading and checking in the
binary for a whole compiler for 3+ platforms is untenable.

~~~
Manishearth
Distros are okay with a single starting binary for compilers. Gcc does this,
for example. Rust would publish a single bootstrap binary, and distros would
build the next compiler with that (and publish), and build the next one with
the one that they just built, and so on. Multiple platforms would just involve
cross compiling the first time.

~~~
snuxoll
The problem, historically, with Rust is that each release was built with the
latest nightly, so you had to bootstrap each new version. Now they've changed
to using the previous compiler release to bootstrap, so you can bring a binary
in for the first build of a package then immediately rebuild with your
bootstrapped binary and continue on from there for eternity.

------
mkroman
I'm curious why rust-url still hasn't been shipped with firefox yet.

[https://bugzilla.mozilla.org/show_bug.cgi?id=1151899](https://bugzilla.mozilla.org/show_bug.cgi?id=1151899)

~~~
larsberg
There are a couple of big pieces of work here:

1) Making the "cargo vendor" story work better. rust-url has a bunch of
dependencies, and you have to get them all in-tree.

2) More security review & planning. URL parsing is scary! And we'd want to
ship & run it alongside the C++ one to check for places where rust-url is not
fully web compatible, but there are major privacy issues in reporting back
anything more than "1 failure," even for users who have explicitly opt'd in to
reporting back data.

But the team is definitely working on both of these pieces and I'd hope to see
it in the near future. No timeline / release number promises, though, right
now :-)

~~~
swsieber
It would be cool for #2 that if a difference was detected, firefox would try
to test and generate a general case (or a minimal case) - substituting out
sensitive information. I guess sort of like fuzzy testing...

~~~
9mit3t2m9h9a
Another question is that determining what is sensitive information is a bit
complicated… But there is an option of asking the user to edit URL to find
anonymous enough form of the bug trigger. Maybe after doing some basic fuzzing
(like replacing runs of alphanumerics with random runs of alphanumerics of the
same length, if possible).

------
madmax96
Super exciting!

I followed Rust at the beginning and I was pleased with the design goals of
the language. However, I wasn't thrilled with the pre-1.0 documentation, and
even after 1.0 there were breaking changes to the language.

Hopefully this is a sign of Rust's maturity. I'll have to look at it again:>)

EDIT: s/from/at/

~~~
steveklabnik
Which post-1.0 breaking changes were you worried about? This is technically
correct, but we've managed it in a way that's been hopefully painless to deal
with, and the community survey seemed to agree.

~~~
madmax96
The compiler has been fantastic emitting warnings about future language
changes and I have appreciated it. The Rust compiler's messages (in general)
are some of the highest quality I've seen, actually. The Rust Devs have done a
superb job at documenting the changes. I'm more complaining that breaking
changes to a non-beta language leave a bad taste in my mouth. Sure, they do
have to happen sometimes, but it seems that this should be rarer than it has
been so far with Rust.

So I haven't worried about anything per se, but breaking changes are mildly
irritating to me. Without a doubt the utility value of some of these changes
has been worth it (specifically RFC 1214, which was ironically one of the most
major and definitely necessary). I'm not involved enough to be able to speak
about other changes.

EDIT: I should also add that the software I work on professionally doesn't use
any language that Rust competes with directly. Rust doesn't offer me
professional utility so it's easy for me to complain about one thing and
ignore all the benefits of Rust. If I were in a situation where I was
contemplating using C++, Rust, or maybe Go, I think I would still choose Rust.
It's far ahead of its competitors and gets a lot of things right.

~~~
Cshelton
I'm curious, do you have a language in mind that has been better than Rust as
far as breaking changes go after the 1.0 stable release? An example? Rust has
been far above most languages/libraries that I've found in dealing with
breaking changes. The complaint here seems pretty empty to me. (Complaining
for the sake of complaining)

~~~
themihai
Go has better backward compatibility.(i.e. no breaking change after 1.0)

~~~
pcwalton
Rust has the same policy.

It might appear differently because we're _very_ up front about any change
that _might possibly_ break any code, even theoretically. We don't make any
changes that we think _actually_ break code, except for blatant bug fixes.

Go has made changes post-1.0 that were more aggressive than anything Rust has
done, such as changing the size of int.

~~~
themihai
I'm sorry to say but you must be delusional. Can you list the these
`aggressive` breaking changes? There were only 7 releases since go1 and I fail
to find any breaking change in the language specification. On the other side
each Rust release has a fat list with "BREAKING CHANGES". Many Rust packages
only work with specific Rust versions(i.e. nightly). The rust ecosystem(std
lib, tools etc) is also way behind Go in terms of stability. Why do you need
rustup if Rust is so backward compatible? I'm not saying that Go is a better
language than Rust but the its ecosystem and dev experience is definitely
superior.

~~~
pcwalton
> Can you list the these `aggressive` breaking changes?

Changing the size of int. Changing methods to introspect the type of their
arguments and do things differently. And so on.

> There were only 7 releases since go1 and I fail to find any breaking change
> in the language specification. On the other side each Rust release has a fat
> list with "BREAKING CHANGES".

Because we have a very specific definition of "breaking change" that is
primarily concerned with the _practical_ effect of changes we make. Go does
not consider these changes "breaking". Using Go's definition (changes to the
language definition), we have no "breaking changes".

If we were to change the size of int (something we will not do, by the way,
due to the practical effects of making such a change), then we would list it
under "breaking changes", even if we were technically allowed to do it. That's
because we care about the practical effects of our changes, not just the
letter of the language definition.

> Many Rust packages only work with specific Rust versions(i.e. nightly).

Because they are explicitly opting into unstable features that are carefully
marked as such. We can't stop packages from doing that. Nor can any other
language.

> The rust ecosystem(std lib, tools etc) is also way behind Go in terms of
> stability.

The parts of the Rust standard library that are marked stable have remained
completely backwards compatible, in both interface and implementation.

> Why do you need rustup if Rust is so backward compatible?

Because it's nice to keep your compiler up to date and to target different
platforms?

~~~
themihai
You are right about the compiler changes but even so you can't compare 1-2
compiler BK with Rust which has language changes as well. Compiler changes are
the norm in Rust.

>> The parts of the Rust standard library that are marked stable have remained
completely backwards compatible, in both interface and implementation.

This looks like a breaking change on a stable API. Am I wrong?
[https://github.com/rust-lang/rust/pull/28811](https://github.com/rust-
lang/rust/pull/28811)

~~~
Manishearth
> Compiler changes are the norm in Rust.

Compiler changes are the norm everywhere? I'm not sure what you're trying to
say here.

> This looks like a breaking change on a stable API. Am I wrong?

Yes and no.

Rust's policy on breaking changes is that changes that can be fixed by
properly qualifying an implicit path are not breaking. Otherwise, adding any
method to anything would be a breaking change. In this case, you can use the
UFCS syntax to disambiguate.

So it's an "allowed" kind of breakage because not allowing this means freezing
the stdlib.

(Go doesn't have this issue due to lack of generics, overloading, and
interface-based overloading. Edit: actually, go does too, due to inheritance,
but that is easier to avoid and isolate. In rust you can always write client
code that breaks if the stdlib adds a method, anywhere. This is true for most
typed languages).

Anything that has the chance of practically breaking things is still run
through crater (which tests impact on the ecosystem) and as you can see that
PR had minimal impact.

~~~
masklinn
> Go doesn't have this issue

Steve has pointed out that go _did_ have this issue as it has a limited level
of auto-deref, which it has changed in the past: implementations performed two
levels of auto-deref when executing methods when the spec only requires one,
the implementations were changed to only allow a single auto-deref:
[https://golang.org/doc/go1.4#methodonpointertopointer](https://golang.org/doc/go1.4#methodonpointertopointer)

~~~
Manishearth
Right. However, as I mentioned in the edit, you can still be careful about
avoiding breaking changes through inheritance and autoderef in the evolution
of Go's stdlib. It forbids very specific types of methods from being added,
and if you avoid that, you can continue to add methods as if nothing is wrong.

Rust (and C++, because SFINAE, and many other languages), on the other hand,
technically has a breaking change each time _any_ method is added to any
public type in the stdlib. It's always possible that the client lib was using
a trait method of the same name, and now has to fully qualify the method call.

------
throwawayReply
That telemetry graph looks odd, dates filtered from 2016/03/03 to 2016/04/07
but "Number of dates" is "3".

So does that mean despite a billion samples between those dates, all samples
fell on just three dates?

~~~
DCoder
If you click on the image to see the interactive graph, under "Advanced
settings" there's a switch "Date range variable: filter submissions by build
date range or submission date range", which is set to "build date range". So
those "3 dates" correspond to three different build dates.

~~~
cpeterso
The telemetry link in the article points to Firefox 45 data. Curiously, the
error rates appear to be going up: Firefox 46's error rate is 0.10% and
Firefox 47's error rate is 4.35%.

* Firefox 45: [https://telemetry.mozilla.org/new-pipeline/dist.html#!measur...](https://telemetry.mozilla.org/new-pipeline/dist.html#!measure=MEDIA_RUST_MP4PARSE_SUCCESS&max_channel_version=release%252F45)

* Firefox 46: [https://telemetry.mozilla.org/new-pipeline/dist.html#!measur...](https://telemetry.mozilla.org/new-pipeline/dist.html#!measure=MEDIA_RUST_MP4PARSE_SUCCESS&max_channel_version=release%252F46)

* Firefox 47: [https://telemetry.mozilla.org/new-pipeline/dist.html#!measur...](https://telemetry.mozilla.org/new-pipeline/dist.html#!measure=MEDIA_RUST_MP4PARSE_SUCCESS&max_channel_version=release%252F457)

~~~
moosingin3space
Wouldn't that make sense because FF 46 was not an ESR, 47 is the latest, and
most users have been pushed to 47? More users are triggering more bugs in the
parser.

Also, 45 ESR doesn't include the Rust code on Windows (I heard that's landing
in 48, if a Mozillian could please confirm that), so that's a large userbase
to not have included in testing.

------
alimbada
Are there other improvements besides the security advantages from this? E.g.
performance or memory usage?

~~~
nbaksalyar
Performance can be considered a consequence of security/language safety. Rust
code is free from data races and developers can utilize concurrency and all
available CPU cores without fear.

~~~
moosingin3space
As evidenced by Servo.

------
empath75
I'm sort of surprised that this is the first time they've used rust for
anything in the browser. I thought they developed Rust for this exact purpose?

~~~
bjz_
Rust was supported from the start by Mozilla in order to enable the
construction of an experimental browser engine (ie. Servo) that was completely
separate from Gecko and Firefox. Afaik, there was never any explicit goal at
the beginning to rebuild things in Firefox - this is just the result of how
successful the language project has been.

------
ChuckMcM
This is awesome. Now we just need a rust based flash player :-)

~~~
ffggvv
Or we should get rid of flash already.

~~~
ChuckMcM
I see it more like the Chesterson's Fence argument. Flash exists, and
continues to plague us, because it met a need for developers that wasn't met
in other ways. If that need still exists, rather than remove Flash (can the
capability with it) and replace it with the same capability but with a much
reduced attack surface.

I think that would be an excellent project for someone, if you should both
show all of Flash's capabilities implemented in Rust and that the result was
safer it would be an excellent endorsement of Rust. It could also illuminate
end user features which can never be made safe. Also good for the overall body
of web knowledge.

~~~
majewsky
> Flash exists, and continues to plague us, because it met a need for
> developers that wasn't met in other ways.

Emphasis on "met". I've yet to come across a function that can be built in
Flash, but not in HTML 5. In fact, I'm not using flashplayer at all anymore
and I don't suffer. (There are a few video sites that are still Flash-only,
but `mpv --ytdl` works around that very nicely.)

Come to think of it, this is actually not true. At work, I have to use Flash
Player for exactly one thing: Adobe Connect. I wonder why they didn't move to
Flash yet. ;)

~~~
llamadrama
Not sure if this is the case for adobe connect, but other, similar products
(such as blackboard's collaborate) are trying to move across to webRTC. This
is complicated because Apple doesn't support webRTC yet, which I'd imagine is
why Adobe hasn't moved across just yet.

------
bluejekyll
Will this include iOS devices too?

This is great news for the language as it's going to help make sure a wider
array of architectures are supported.

Awesome work!

~~~
guelo
Firefox for iOS is just a wrapper around webkit since Apple refuses to allow
any alternate engines.

~~~
masklinn
IIRC, technically alternate (rendering) engines are allowed but you can only
execute downloaded code using the system's webkit/JSC (and if you didn't you'd
be restricted to a straight interpreter).

------
darkboltyoutube
[https://www.youtube.com/watch?v=B2KXVfnw4rg](https://www.youtube.com/watch?v=B2KXVfnw4rg)
pokemon go death???

------
SShrike
Seems Rust is moving up in the world!

------
Noseshine
Am I'm reading either the headline or the announcement wrong?

As I read it, they are not shipping Rust but a component written in Rust.

    
    
        > For this reason, Ralph Giles and Matthew Gregan built Mozilla’s first Rust
        > media parser. And I’m happy to report that their code will be the first
        > Rust component shipping in Firefox.

~~~
akerro
You did not think, they would replace VanillaJS with Rust, did you? ;)

~~~
Noseshine
What does "replace" have to do with it? Why do you invent the most illogical
option possible - that I never uttered - to find a counter argument?

Assuming that they ship another language _in addition_ to Javascript would not
be completely out of the question. While it seems that with WebAssembly that
is no longer necessary, who knows what Mozilla, in search of future funding,
may come up with to open new markets. They also tried creating their own OS
for mobile - shipping Rust to further its adoption for whatever to me at the
moment inconceivable reason is no less improbable.

So yes, I wasn't sure what to expect from the headline - it said they are
"shipping Rust" after all, so something crazy was certainly within realm of
possibility.

~~~
kbenson
> While it seems that with WebAssembly that is no longer necessary

I believe they have mentioned that a WebAssembly target for Rust compilation
is something they are working towards, and IMO that would be the ideal way to
deploy Rust for a website (if it's a compiled language, there's no need to
deliver it uncompiled).

------
arc_of_descent
The headline is really wrong. Mozilla is shipping compiled and executable code
to us. Now, if its Perl, I can confidently say I'm shipping Perl code to you.

Small nitpick, I know :)

~~~
djsumdog
Their source contains the actual Rust code and it's part of the build process.
Distribution build maintainers will compile that rust code when making their
packages. I run Gentoo, so that process happens when I upgrade Firefox.

