Hacker News new | comments | show | ask | jobs | submit login
Compiling Rust to Web Assembly (medium.com)
289 points by grey-area on Jan 5, 2017 | hide | past | web | favorite | 88 comments

I wasn't a fan of having to compile the dev branch of both emscripten itself, and clang, each time I wanted to try the latest (hence the 'wait 2 hours' in the blog)

I thus asked about providing a Docker image - the person that handles the builds and the SDKs says that nightly builds are on the way: https://github.com/kripken/emscripten/issues/4682#issuecomme...

I read the guide and came here looking to see if there was a docker image. I just don't want to blindly pipe-sh on my host machine (would need to review what rustup is doing first). On top of that, 'wait 2 hours' is not something I want to do when I want to kick the tires on a Hello World.

This is a great example of docker-for-dev. Don't bother setting up the specific versions of compilers and then worry about clean-up afterward.

The guide should be 2 lines: docker pull, then docker run.

Here is an example that using Rust-native library for web development: https://github.com/tcr/rust-todomvc

Also I forked it to use handlebars: https://github.com/sunng87/rust-todomvc

Using asmjs, the compiled js file is about 20MB and can be slow in browser. While wasm32 one is just ~500KB.

Can wasm be chunked up and loaded on demand via AJAX, like JS?

Nice guide.

I did this over the holidays just to see it in action. It was a pain to get the emscripten sdk installed (especially on a slow connection) but it's quite cool to see it in action.

There's also a asm.js target which you can add like this:

$ rustup target add asmjs-unknown-emscripten

I'm not sure about browser support for asm.js but recent versions of node support it.

Every browser supports asm.js, since it's just a subset of standard JS. Some browsers do optimize it, though. https://en.wikipedia.org/wiki/Asm.js#Implementations

Is there a way to debug apps compiled to Web assembly? (Like a gdb.js :D ) I suppose using standard dev tools won't be useful.

That's one area where I would like to see web browsers cooperate a bit. At least Firefox and Chromium. This would give better tools to everyone. I won't bet on it, however.

This is great, I've actually never seen the steps to compiling WASM laid out before.

What all does the Emscripten SDK provide?

An HTML shell that can handle resource loading, plus a bunch of JS methods for providing IO to WebAssembly. In the future it will be possible to do something like https://github.com/brson/mir2wasm to go directly from LLVM to WASM (though IO functionality will need to be supplied separately).

What's the binary size like compared to C/C++?

I've read the overhead of Rust's standard library is about 100kB. Not sure if putting it through the Emscripten toolchain changes that significantly.

In general, https://lifthrasiir.github.io/rustlog/why-is-a-rust-executab...

For this specifically, I made some numbers when it first landed: https://www.reddit.com/r/rust/comments/575pp6/compiling_to_t...

Regardless, this is in a "oh my god it works" state, not a super serious best-we-can-do state. My understanding is that it's larger than it has to be at the moment. We'll get there.

>-rw-r--r-- 1 steve steve 159K Oct 12 20:00 sup.wasm

It's looking pretty good already. Thanks for the reply.

Has anyone heard of python compiled to Web Assembly ?

Python needs all the extra runtime and interpreter cannot just be compiled that easily, except with dropped features.

That reminds me of Google's GWT, which compiled Java to JavaScript and supported a limited subset of the JDK. Maintaining a legacy GWT codebase is not a very pleasant experience, because you are stuck in a Java-like abstraction layer on top of the actual browser features and the DOM.

I guess compiling a language like Rust to WebAssembly solves these issues for a part?

GWT is very different because it directly 'transpiles' Java to a high-level Javascript, you basically get the worst of both worlds ;) asm.js and WebAssembly are much lower level, they are generated from LLVM bitcode, theoretically any language that can be compiled through LLVM can be used with asm.js / WebAssembly (as long as the language runtime libraries are ported to run in a browser environment).

I don't know, but if Web Assembly allows something like dynamic linking, it would enable, for example, Oracle to supply a WASM-JVM via CDN, that JVM based languages could target.

But this would have to be a rather cut-down version of the JVM, I think, so probably not that viable.

As far as I know, if you don't want to add a runtime, you can only compile languages that compile to native code to Web Assembly. C, C++, D, Rust, Nim, etc.

So it's more for low level lanuages to make performance easier on the Web, not for replacing JavaScript in general.

C-python runs on a bytecode interpreter, so theoretically you could port the VM to webassembly, provided you remove features that depend on a native operating system. More bare-bones interpreters like Lua (http://daurnimator.github.io/lua.vm.js/repl.html) and Chibi-scheme (https://chibi-scheme.appspot.com/) which have had their C compilers and vms ported to emscripten can run natively on the web. JIT interpreters like Juajit, Java Hotspot, and pypy, cannot easily run on webassembly or asm.js due to their dependency on native chipsets with self-modifying code support. Also, most of the tooling for wasm and asm.js relies on llvm, so a good amount of work would be required for compiled languages that don't have compilers emmitting llvm bytecode

But don't need D und Nim a runtime for their garbage collection?

Good point. Maybe it's small?

I mean Nim already allows compilation to JavaScript.

Still waiting for an article named 'Compiling Go to Web Assembly' https://imgflip.com/s/meme/Ill-Just-Wait-Here.jpg

Three rust articles on the front page? Not complaining, it definitely seems like the popularity has really jumped up in the last three months.

More Rust discussion and less "future of media/ad supported businesses" discussion works for me.

Not to mention fake news and politics in general.

As far as I know it's because the Rust is filling in the gaps - two of the three articles are about relatively new features. And the next Rust release should close the stable/unstable gap by stabilizing some macro/compiler plugin stuff.

It is a marketing push. Marketing pushes don't have to correlate to massive improvements in the product.

Certainly not a coordinated marketing push. All of the recent Rust posts on HN don't come from any official venues. It's just random community members writing about Rust, and HN upvoting it.

There was a flurry of Rust posts about "why Rust" near the end of the year, which was somewhat "coordinated". In the sense that Steve wrote a post and this triggered a chain reaction where all kinds of folks wrote about related topics. Completely unintentional, just happened. Most of those posts did not make it to HN.

It's just random people discovering Rust due to increased popularity and posting about it (which has a circular effect).

No marketing plan involved, and nobody coordinates these posts.

Quite a few of us made learning rust our new years resolution in recent threads.

and what's the argument here?

Rust and Go are always on the front page.

Also Python, JS frameworks, CLR and JVM languages, C/C++, LLVM, notably obscure languages... It's like there's some kind of theme.

If someone makes a Go compiler in Rust hacker news will have to shut down. Game over, there is nothing left to do in computing.


"A Go compiler, written in Rust."


Granted, it would require writing a garbage collector in Rust...

> Granted, it would require writing a garbage collector in Rust...

Why would that be a problem? There are compilers written in C that implement garbage collectors.

Now, writing a garbage collector for Rust, that's another problem entirely

> garbage collector in Rust..


> Now, writing a garbage collector for Rust,


(mostly a toy GC, but some folks are using it)

In the future we may have http://manishearth.github.io/blog/2016/08/18/gc-support-in-r...

During my senior year of college (which ended last spring), I worked with two other students on implementing a server-side JavaScript interpreter in Rust. One of us, a grad student (not me!), wrote the garbage collector for it (https://github.com/rustoscript/french-press). We didn't end up finishing the entire language, but we got a sizable chunk of the language features implemented, and more than enough to know that Rust is very well suited to writing programming languages in, including garbage collectors. By far, the most complicated component of getting the garbage collector right was dealing with closures, but that's mostly because JavaScript is a very "unique" language, to put it nicely.

> In the future we may have http://manishearth.github.io/blog/2016/08/18/gc-support-in-r...

I read that back when you first posted it, which is exactly what came to mind when I was writing my comment. At least to my naive understanding, implementing a garbage collector for Rust is several orders of magnitude more difficult than merely implementing one in Rust.

The JS interpreter is pretty cool!

We've had problems with closures before in Servo, I know that pain :)

> implementing a garbage collector for Rust is several orders of magnitude more difficult than merely implementing one in Rust.

Well, I mean it depends on what you're aiming for. Making a safe optional GC in Rust isn't too hard; rust-gc exists and is not very complicated. Making the compiler support arbitrary pluggable GCs is trickier. GC algorithms themselves can get so complicated that the state of the art research probably eclipses whatever has been done with rust-gc and similar "GCs for rust" so far. If you want a GC for Rust that contains a world-class collection algorithm the complexity is additive but the source of most of the complexity isn't the "for Rust" part, it's the "in Rust" part (A GC for rust will also typically be in Rust).

But also "optional" GCs like those that would be useful in Rust haven't been explored much either, so world class algorithms may not even make sense in this case.

It's ... an interesting set of problems, and very hard to make judgement about what is difficult :)

Thanks for elaborating on that! I'm definitely looking forward to seeing what you all do on the GC front for Rust in the future, even if it's a bit beyond my current comprehensions

I hate to ignore the cool technical knowledge needed to implement a GC for JavaScript, but damn, "french-press" is the best possible name for that project, I love it.

There is a lisp dialect on rust https://github.com/murarth/ketos and it doesn't need GC!

Rust compiler written in Go?

TBH it's getting a bit much at this point. The rust advocates here (not sure if they're paid or just fanboys) really need to tone it down. The hyperbole, and the constant "rust solves this unrelated problem, and has no flaws!" is beginning to grate.

I've coded in it a bit and it seems like cool language, they get a lot right and have some new and interesting ideas. But it's not a panacea (or indeed "salvation" as one recent blog post put it) for everything wrong in software. We don't need to all immediately re-write everything we do rust. Not everything new needs to be written in rust. And your re-writes of programs that have already been written in other languages and work well aren't that compelling.

If you like rust, then write some new, interesting and useful software in the language, where what language it's written in isn't the main selling point. For example, I love the look of this http://nickel.rs/.

TL;DR - the corrosive nature of rust advocates on HN is starting to eat away at my patience, and I am tired of having to galvanize against them and their absurd claims.

Like burntsushi I feel you're attacking a straw man here. Or, you're attacking specifically the "salvation" article which was a bit hyperbolic on purpose (in which case you're mostly off topic in this thread)

> the constant "rust solves this unrelated problem, and has no flaws!"

Where do you see that?

> But it's not a panacea for everything wrong in software

Nobody says that. Folks generally focus on the memory safety and thread safety bits. The salvation article did say that Rust can solve all of the problems in software caused by C, which was hyperbolic (I wasn't really fond of that part either), but that's the only instance in a blog post or comment that I can recall this being said recently. There doesn't seem to be a trend as you see.

> We don't need to all immediately re-write everything we do rust. Not everything new needs to be written in rust.

Nobody says that either. If anything, I've seen folks (especially in /r/rust) advocating caution when asked about porting software to Rust.

> If you like rust, then write some new, interesting and useful software in the language, where what language it's written in isn't the main selling point.

There's a lot of this all over the rust community. Redox has reasons for existing other than Rust (https://doc.redox-os.org/book/introduction/why_redox.html). Servo similarly has a wide variety of interesting bits -- some of which have been enabled by Rust but aren't interesting because of Rust. Webrender in particular is something that could have been done without Rust. Servo's layout and style systems are pretty revolutionary, because of the way they work. Ripgrep (http://blog.burntsushi.net/ripgrep/) probably could have been implemented in Go and perhaps still win on some of the benchmarks; it's interesting because of the tricks that burntsushi used. https://github.com/google/xi-editor is also interesting, though incomplete.

There certainly was a phase where many threads about basic software (curl-type stuff, init systems, ...) had comments that felt like endless repeats of "it is 2016, stop writing new C code and just use Rust", which I personally found annoying. I either tuned it out or it has gotten better recently, and I don't think it was a organized effort, just a common pattern.

Yeah, most posts in HN Frontpage with Rust/C/C++ seemd to have a "never Write c(or the baddest: C++!!) in 2k16!1! Its useless because of Rust!"-comment.

Its kinda annoying, but i can ignore them pretty good, so i dont have more like a feel Here. It could be less in the last 1-2 Month(s).

Yes, we had a recent explosion of marketing materials that went a little over the top. That's OK. It happens.

I don't think anyone really believes Rust solves every problem, so I think you're attacking a straw man. There are plenty of trade offs at play. But the problems we believe it does solve makes it exciting for a lot of us (and for perspective users).

Plus it's not really the fault of Rust developers if https://blog.ntpsec.org/2017/01/03/getting-past-c.html gets to the front page; that's not really marketing material for the language.

Hello, I posted this because it looked interesting. I'm not a Rust advocate (I prefer GC-languages personally), the WebAssembly angle and different languages using it as a compilation target was more interesting than Rust specifically.

Right, sorry to hijack the thread, or imply you were one of the people I was talking about.

What are your general thoughts on web assembly? I am really optimistic about it - improving client side performance with a fast IR format will really open up a lot of doors, and make the web a much more viable platform for applications. I do note that many similar attempts have been made in the past though.

> the corrosive nature of rust


More seriously, I'm not sure who you mean by the paid rust advocates here. There are some core developers who work for Mozilla, but I can't imagine that a Rust-using company or individual would spend salary on an HN advocate rather than paying someone to work on the compiler or ecosystem.

You'd be mistaken. steveklabnik is a hired Rust evangelist who also writes docs.

> Wherever there is discussion about Rust to be found, be it IRC, the RFCs repo, or Reddit/HackerNews, you can be sure to find a comment or two from Steve there, explaining and clarifying the situation.


To be clear, my job description is "doc writer". I've been posting a ton on HN for _years_, and advocating for Rust before I ever had a job working on it. At the time that post was written, I was a contractor, and it wasn't clear if I was ever going to stay on with Mozilla; historically, I've told everyone this was a short-term gig, though my feelings on that have been changing a bit.

Regardless of that, if you ever hear me saying Rust is a "panacea" or "immediately re-write everything [in Rust]", call me out on it. I don't agree with either of those positions. I want people to accurately understand what's going on, not mislead them.

(I _do_ want to see more people using Rust, because I love it, but not everyone will also love Rust, and that's totally fine.)

Having idled around the same tech watering holes as you have for years, I'd think they'd have to make a significant effort to keep you from talking about it. ;)

I am a pretty active observer of rust community as the language abilities and paradigms excite and suit me. Therefore I do see steveklabnik as well as several others frequent these avenues. In all my time reading Reddit/HN/etc. I've never seen steve or the others "advocate"; they're doing exactly what that description states which is explain and clarify. The tech community brings up rust as a discussion topic and they spend their own time (possibly some of Mozilla's) to make sure that the community understands the organization (not just Mozilla), drive, and tech behind rust.

> steveklabnik is a hired Rust evangelist who also writes docs.

Nobody is paid to evangelize on message boards.

> There are some core developers who work for Mozilla


There is a difference between "someone working for Mozilla on a project also comments about it in public" and "Mozilla pays people to advocate for rust in public forums", and I don't think anyone is surprised by the fact that the first happens (arguably, it is one of the good things in communities like HN, and it's not like they hide where they work). Trying to construct this as paid shills or something is, IMHO, not helpful.

And from what I've seen, the "Why is this not written in Rust?" and similar comments are not from core Rust people, to the contrary they also explain why Rust does not fit for something (and potentially what's done to change it).

By me, for one, and I have nothing to do with Mozilla. I just follow HN for years, and have read Steve's comments even before he was working for Mozilla. He is just passionate about Rust and posts about it on HN (as opposed on the Mozilla blogs) on his own time, as he did on his own blog back in the day. As do other people, whether they work for Mozilla or not.

They are downvoted for spreading misinformation. Steve is not a "hired Rust evangelist". Steve was hired to write docs, and that's what he does. He has always maintained a large online presence in the Rust community, before and after being hired; but this is not what he was hired for.

Disclaimer: I also work for Mozilla (on Servo). Not paid to do advocacy online.

Steve is paid to write docs; evangelizing is just his hobby.

Sibling comments to yours corrected this point prior to you making this comment, which makes it harder to assume good faith about your actions here.

The part quoted heavily implied that he was paid to evangelize. Yet he was heavily down-voted for saying something relatively innocuous. I thought it was noteworthy.

What makes you and thesmallestcat even more incorrect is that this blog post is about Steve Klabnik and Yehuda Katz joining the Rust core team. This is an unpaid position with the (informally organized) Rust project, not a paid position with Mozilla Research. Mozilla employs Steve (though I think it was just a part time contract in 2014), but doesn't employ Yehuda.

The "Rust Core Team" is not a Mozilla construct. Many of the folks on it are Mozilla employees, but the core team is not a paying job.

Mozilla hired Steve to work on docs.

Steve was invited to the Rust Core Team because of his online activities, among other things.

Neither wycats nor steve were being paid by Mozilla at the point that post was written IIRC. (I might be getting my timelines mixed up here. wycats has contracted for mozilla in the past for cargo and stuff, but I don't recall exactly when that happened)

To reply to both of you at once, I was near the end of my first contract with Mozilla to write docs full-time when I was added to the core team; I then got a second contract, and finally, a "real job."

(I had to look this up in my email to remember myself, frankly.)

Nobody gets a paycheck for being on the Rust core team.

As a counter, stuff is posted to new every minute, but 90% of it doesn't reach the front page. In order to reach the front, you often have to do make up silly titles such as the ones you mentioned, which cause an emotional response (good or bad, as long as it gets clicks).

HN also has a problem with brigading, which is why you see some utterly random articles make it to the top too. Sadly not much you can do about this, but you could try browsing /r/programming or elsewhere instead.

> HN also has a problem with brigading

More than once as an employee I've been asked to upvote a company post. Obviously I immediately flagged said posts.

> And your re-writes of programs that have already been written in other languages and work well aren't that compelling.

Try ripgrep.

I didn't know how much time I spent waiting for grep until I tried ripgrep!

I enjoy reading about rust.

I don't know the language (yet), but I've been keeping my eye on it for a few years, and I think it's going to be fantastic once it stabilizes and gets improved tooling.

I especially welcome the possibility of using a safe language like rust for web development, instead of writing more dreadful javascript...

Note that Rust is stable now. IDE tooling and stuff isn't there yet (soon?!), and there are other reasons why you may not use Rust, but it is stable.

After 1.0 the Rust releases have been stable, with the stability checked against the entire ecosystem on crates.io. https://blog.rust-lang.org/2014/10/30/Stability.html


You have to explicitly opt in to unstable features. Provided you don't do that, Rust is stable.

There are bits of the standard library marked unstable, yes. This is true in almost any language/compiler -- there's always some new/experimental or otherwise nonstandard stuff that the compiler will let you do if you opt into it. Clang has its own nonstandard bits. So does GCC.

The fact that almost all crates in the Rust ecosystem work on stable is an indication that enough of the stdlib is stable for it to be useful.

The Rust stdlib will never be "finished"; there will always be new things being added. This is true for every language. Allowing folks to experiment with unstable features is not a bad thing.

That method is not yet available in stable Rust, because it's unstable.

If you run that example on stable Rust, you'll see

    error[E0554]: #[feature] may not be used on the stable release channel
> isn't finished

I wrote about this here: https://news.ycombinator.com/item?id=13277438

TL;DR: stuff starts out being added, sits in unstable for a while, then makes it to stable. No software is ever finished. Stability is not the absence of change; stability is the absence of things breaking.

> I'm curious about what "stable" means to rust advocates

You should read the blog post in the parent comment, as it explains, at length.


Define "unfinished". You're being deliberately misleading with that word.

The Rust stdlib has everything most folks need. There are some things like simd which are currently being actively pushed that are "missing", but even C++ doesn't have "stable" simd, it has compiler extensions in some compilers. Most of the unstable APIs in the stdlib exist independently in the crates.io ecosystem anyway (or their functionality can be replicated with other stdlib apis).

The fact that the rust stdlib can still be improved does not make it unfinished (or, if it does, being "unfinished" is not a bad thing). By that argument every language that still pushes out new versions is unfinished.

Your definitions of "stable" and "finished" exclude almost all the languages/compilers out there, I highly doubt these are the definitions that 99% of programmers use.


> Changing the API of the standard library on minor release versions is stable now

No it's not. A release version of rust cannot access that API. It's just there in the docs. We could possibly remove it from the release docs, or make the messaging clearer (this is being discussed), but you can't use it on a release. That is stable.

I mean, you have the same situation with https://gcc.gnu.org/projects/cxx-status.html#cxx1z . They, too, explicitly say that opting in to this may break backwards compatibility (i.e. it's unstable). http://clang.llvm.org/cxx_status.html says something similar.

In both those cases the default release compiler contains the ability to opt in. With Rust, even this isn't the case; you have to download a different compiler if you really really want those features, and even then you still have to explicitly opt in at a fine-grained level.

> Changing the API of the standard library

Adding to the API of the standard library, as SemVer specs. Backwards incompatible changes aren't made. (Except for soundness reasons, which happens in extremely rare circumstances, and are made with great care with regards to the upgrade path.)

Rust releases follow SemVer, which is not exactly an uncommon standard for those things. Code that works in released version 1.X.Y is supposed to work in all following versions. Are there languages that won't add backwards-compatible features?

> So the language is stable, but it ships with an unstable, unfinished standard library?

No, it does not. It ships with a stable standard library, but methods that are unstable are also shown in the docs, so you can see what's happening in upcoming releases. And it links to the issue where you can get involved to help bring the method to stable, if that's important to you.

> I see rust advocates being called out on this _all the time_

Called out on what, specifically?

If we ignore the unnecessary heat of the comment for a moment, there's an interesting developer experience misunderstanding in there. It may even be enough to just change the wording -- instead of "unstable", "proposed"? (Might sound too tentative.)

Yeah, totally. This would make for a very interesting RFC.

(There's also always been a disconnect between naming in this regard; that is, you can only use "unstable" things on the "nightly" build of Rust. And while we try to produce a nightly every night, it doesn't always happen, as these last few weeks have shown. That is changing, at least. But "proposed Rust" sounds stranger than "nightly Rust", so this disconnect might persist even if we decided to change "unstable" to "proposed". Side note: Ember and IIRC Chrome call their nightly versions "canary"...)

I like this. Perhaps "Proposed, nightly-only" would be even clearer?

That method is marked unstable, which means you can't use it on the stable releases of Rust. The only way to use it is to do both of the following:

1. Use nightly Rust.

2. Explicitly enable the `io` feature with a special directive in your source code.

> I especially welcome the possibility of using a safe language like rust for web development, instead of writing more dreadful javascript...

There are now a number of typed languages that have solid compile-to-js support. I've done it with OCaml (js_of_ocaml, Bucklescript), F# (Fable), and Elm. For dynamic languages, there's also Clojurescript, which is quite nice.

Applications are open for YC Summer 2018

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact