I really don't understand this article, and the claims really rub me the wrong way.
The main point it makes is, again "He perfectly demonstrates one of the points my “Oxidizing” article was making: with Rust and WebAssembly we have reliable performance without the wizard-level shenanigans that are required to get the same performance in JavaScript."
This doesn't make a lot of sense as a claim.
Why? Because underneath all that rust .... is an optimizing compiler, and it happens the author has decided to stay on the happy path of that. There is also an unhappy path there.
Is that happy path wider? Maybe. It's a significantly longer and more complex optimization pipeline just to wasm output, let alone the interpretation of that output. I have doubts it's as "reliable" as the author claims (among other things, WebAssembly is still an experimental target for LLVM). Adding the adjective "reliable" repeatedly does not make it so.
Let's ignore this though, because there are easier claims to pick a bone with.
It also tries to differentiate optimizations between the two in ways that don't make sense to me:
"In some cases, JITs can optimize away such allocations, but (once again) that depends on unreliable heuristics, and JIT engines vary in their effectiveness at removing the allocations."
I don't see a guarantee in the rust language spec that these allocations will be optimized away. Maybe i missed it. Pointers welcome.
Instead, i have watched plenty of patches to LLVM go by to try to improve it's heuristics (oh god, there's that evil word they used above!) for removing allocations for rust.
They are all heuristic based, they deliberately do not guarantee attempting to remove every allocation (for a variety of reasons). In general, it can be proven this is a statically undecidable problem for a language like rust (and most languages), so i doubt rustc has it down either (though i'm sure it does a great job in general!)
The author also writes the following:
"WebAssembly is designed to perform well without relying on heuristic-based optimizations, avoiding the performance cliffs that come if code doesn’t meet those heuristics. It is expected that the compiler emitting the WebAssembly (in this case rustc and LLVM) already has sophisticated optimization infrastructure,"
These two sentences literally do not make sense together. The "sophisticated optimization infrastructure" is also using heuristics to avoid expensive compilation times, pretty much all over the place. LLVM included. Even in basic analysis, where it still depends on quadratic algorithms in basic things.
If you have a block with 99 stores, and ask LLVM's memory dependence analysis about the dependency between the first and the last, you will get a real answer.
If you have 100 stores, it will tell you it has no idea.
What happened to reliable?
Why does this matter? For example: Every time rust emits a memcpy (which is not infrequent), if there are more than 99 instructions in between them in the same block, it will not eliminate it, even if it could. Whoops. Thats' a random example. These things are endless. Because compilers make tradeoffs (and because LLVM has some infrastructure that badly needs rewriting/reworking).
These "sophisticated optimization infrastructures" are not different than JITs in their use of heuristics. They often use the same algorithms. The only difference is the time budget allocated to them and how expensive the heuristics let things get.
There may be good reasons to want to write code in rust and good reasons to believe it will perform better, but they certainly are not the things mentioned above.
Maybe what the author really wants to say is "we expect the ahead of time compiler we use is better and more mature than most JITs and can spend more time optimizing". But they don't.
Maybe it would also surprise the author to learn that their are JITs that beat the pants off LLVM AOT for dynamic languages like javascript (they just don't happen to be integrated into web browsers).
But instead, they make ridiculous claims about heuristics and JITs. Pretending the compiler they use doesn't also depend, all over the place, on heuristics and other things is just flat out wrong. At least to me (and i don't really give a crap about what programming language people use), it makes it come off as rampant fanboism.
(Which is sad, because i suspect, had it been written less so, it might be actually convincing)
> Why? Because underneath all that rust .... is an optimizing compiler, and it happens the author has decided to stay on the happy path of that.
There are two big differences here:
1) You're comparing "staying on the happy path" of a JIT compiler in the JS case, vs. an optimizing compiler in the Rust case. With the latter you can just compile your code and see what comes out and it tends to be fairly predictable. With the former, I'm not even sure there are tools to inspect the generated JIT code, and you're constantly walking the line of JS engines changing their heuristics and throwing you off the fast path. This was one of the primary motivations for the asm.js/WebAssembly work: the ability to get predictable performance.
2) Many of the optimizations mraleph performed were tricks to avoid allocation (which is normal optimization stuff, but more of a pain in GCed languages). In JS he winds up having to effectively write C-in-JS which looks pretty hairy. In Rust controlling allocation is a built-in language feature, so you can write very idiomatic code without heap allocation.
> what comes out and it tends to be fairly predictable
Predictable as long as you stay on the same version of the compiler (yes, I know that there are crater runs to prevent regressions). Also, how does much can/does the output for different target architetures differ in performance? Couldn't that be likened to trying to optimize for multiple JS engines?
To address your question about allocations, in Rust, you always know when you are allocating on the heap vs on the stack. The way you write your code guarantees it. Stack allocations are “basically free” compared to the heap because the memory management overhead is negligible or nonexistent. There are also guarantees about when objects you allocate on the heap are freed; there is no garbage collection.
This is in contrast to JavaScript where objects are always allocated on the heap, and are garbage collected.
So Rust gives you a lot more control and flexibility about how memory is managed. This might or might not matter for your use case, of course. There are compiler optimizations and heuristics -on top of that-, to be sure, but you end up with a lot more guarantees about how your code executes, because that’s what the language is designed for.
"here are also guarantees about when objects you allocate on the heap are freed; there is no garbage collection.
This is in contrast to JavaScript where objects are always allocated on the heap, and are garbage collected."
Surely you realize that this, and what the author wrote, are basically the same ever rehashed GC vs non-GC language discussion. Performance characteristics of each is not anywhere near as simple as any of these things claim, so i'm going to leave this alone.
I see it saying things like "This program has one variable binding, x. This memory needs to be allocated from somewhere. Rust ‘stack allocates’ by default, which means that basic values ‘go on the stack’."
Can someone point me to an actual language level guarantee in a spec? I looked, and i don't see it.
I can't see anything that makes it non-conformant to build a rust compiler that dynamically allocates and places all of these on the heap, and is very non-constant time for local variable allocation.
But again, i didn't spend more than a few minutes browsing, so i may have missed it (For example, https://doc.rust-lang.org/reference/memory-allocation-and-lifetime.html says nothing here, the requirements i can find in this entire chapter can easily be met by using dynamic allocation. There are not guarantees on space or time usage that would require a real stack :P).
In fact, the vast majority of requirements here look like they could be met by a garbage collected heap/stack. It would be horribly inefficient, but ...
I'm totally willing to believe nobody has written down a good enough spec yet, just saying i don't see it ATM :)
I want to strongly differentiate between what "one implementation of rust does" and what "the language guarantees". Because if you are going to claim it's rust that makes the guarantee, as the author did, you should be able to back the claim up.
> I want to strongly differentiate between what "one implementation of rust does" and what "the language guarantees". Because if you are going to claim it's rust that makes the guarantee, as the author did, you should be able to back the claim up.
It's perfectly reasonable to use "Rust" as a proxy for the only implementation that can be realistically deployed to the Web (rustc). Everyone knows what the author meant.
> I can't see anything that makes it non-conformant to build a rust compiler that dynamically allocates and places all of these on the heap, and is very non-constant time for local variable allocation.
I mean sure, the language doesn't require that compilers don't emit dumb code, it is just designed so that it's easy to emit good code. Something with fewer static guarantees like JS makes it much harder for compilers to emit that good code. I don't think there's any ground to contest that.
In any case, if you just replace "stack allocates" etc. with "semantically stack allocates", you get the language's guaranteed behaviour (although Rust has no ISO spec or anything, so you're probably going to say that even that isn't truly guaranteed).
Mean, it's fair that the language used is quite strong, but still, spelling literally everything out with every possible caveat is a great way to have bad pedagogy.
He is a lawyer, what would you expect? Being pendandic is his job. Not that I agree with him btw. I think he just does not have enough real practical engineering experience to understand it well.
I think the author's primary point was that the work done by mraleph required _deep_ knowledge of the V8 JIT internals and low-level profiling to get those "3x speedup" results, plus the algorithmic improvements. Meanwhile, the original Rust implementation got the same "3x" results without having to do deep analysis of how the compiler was behaving. It also seems (based on the commentary) that the way JS/JIT engines treat WASM bytecode is likely to require fewer special-cases or heuristics than plain JS.
Sure, the Rust compiler and the LLVM infrastructure are doing a lot of complicated work internally... but the end users of the compiler aren't having to spend time digging through the guts of it to guess what kind of magic sequences are needed to get fairly good performance.
"but the end users of the compiler aren't having to spend time digging through the guts of it to guess what kind of magic sequences are needed to get fairly good performance."
That's precisely my point: The claims they make about this in this article that claim to prove this are demonstrably false. So that may even be true, but it's definitely not anything in this article that shows that.
If the author had just said "hey, when i wrote this version, it performed better and was easier", that'd be great, and awesome.
Instead, it seems they wanted to make more general claims, and to be honest, don't seem to have a lot of idea what they are talking about on that front, and to anyone who does have experience in this area, like i said, it comes off very badly.
I'll also point out, if you expect to not need to do profiling and algorithmic improvement to anything to get significant speedups, that's also similarly silly. It's just not realistic for any language in the real world. The only question is "which of the code you write will this be true for" not whether it will be true.
The article clearly admits that the LLVM compiler uses heuristics. You even quoted that part of the article.
>WebAssembly is designed to perform well without relying on heuristic-based optimizations, avoiding the performance cliffs that come if code doesn’t meet those heuristics. It is expected that the compiler emitting the WebAssembly (in this case rustc and LLVM) already has sophisticated optimization infrastructure, that the engine is receiving WebAssembly code that has already had optimization passes applied, and that the WebAssembly is close to its final form.
But really the point of the article is the last part.
>that the engine is receiving WebAssembly code that has already had optimization passes applied, and that the WebAssembly is close to its final form.
The compiler applies heuristics once during the compilation step and then never again. Compare this to JITs which are constantly changing and can pull the rug from under you.
>Maybe it would also surprise the author to learn that their are JITs that beat the pants off LLVM AOT for dynamic languages like javascript (they just don't happen to be integrated into web browsers).
Yes but it gets even better! If you limit yourself to a formalised subset of javascript called asm.js which gives you fine control over memory layout and allocation you can reach even the performance of C! Have you heard of it's successor? I think it's name was Web Assembly and every major browser has integrated it. It's a really cool technology that shows how sandboxed JITs can have the same performance characteristics as AOT compilers.
Allocation in Rust are opt-in, you allocate when using the `Box` keyword, when using reference-counted pointers or when using a heap-allocated data structure (vector, hashmap, etc.). If you do neither of those, you have zero heap allocation, 100% of the time, there is no heuristic involved here.
For optimizations, you're right : LLVM uses a lot of heuristic. But this happens at compile time, when you generate your webassembly blob. When you have it, it will run at a consistent speed across browsers and between several generations of the same browsers. None of this is achievable in plain JavaScript, where JS optimized for old V8 (crankshaft) won't be optimal for SpiderMonkey or newer version of V8 (Turbofan).
What you're completely missing here is that a naive rust compiler would be reasonably close in speed to LLVM's non-naive compilation. Sure some inlining helps a bit, and llvm does some really fancy things that help a bit, but even without those Rust would still be a reasonably fast language. As such you can just right normal rust, and your worst case where the compiler completely fails to help is reasonably fast.
A naive javascript interpreter is really slow, spidermoney and V8 do a ton of work to make it reasonably fast. If there's an important piece of code where the compiler doesn't even completely fail to help, but just helps less than normal, the code is no longer reasonably fast.
> Why? Because underneath all that rust .... is an optimizing compiler, and it happens the author has decided to stay on the happy path of that. There is also an unhappy path there. Is that happy path wider? Maybe
I think you've got a fair point, that all optimizing compilers and JITs have happy paths and unhappy ones, falling off the happy path for either form will result in slower code. But, the happy path being wider kinda seems like the key thing here?
Additionally, AOT has consistency, in that it doesn't depend on runtime data. This can obviously leave some performance on the table if a JIT ends up optimizing for the data that happens to be used in a given session, but also means performance usually doesn't (accidentally) depend on global state.
> Instead, i have watched plenty of patches to LLVM go by to try to improve it's heuristics (oh god, there's that evil word they used above!) for removing allocations for rust
I recall some that just told LLVM the names of the symbols Rust uses instead of malloc/free, so that it could do its standard escape analysis of completely pointless (that is rarely useful for Rust, IME), but didn't actually touch the rest of LLVM.
In any case, rustc (and the current understanding of Rust as a language) doesn't insert allocations itself, because there's no reason to, unlike JS, where the allocations are essentially semantically required. That is to say, Javascript's language model is so flexible that many things have to allocate by default (as it's the only way to get the required shared-mutation behaviour), where as in Rust, heap allocation is implemented as a standard library construct (it's not necessary for the language: e.g. libcore is a bare-metal subset of the standard library that works with no dependencies) and people won't manually put things on the heap unless they have to.
> Maybe it would also surprise the author to learn that their are JITs that beat the pants off LLVM AOT for dynamic languages like javascript (they just don't happen to be integrated into web browsers).
Are you saying that a JIT for JS beats LLVM for... JS? I don't think that's particularly surprising (and probably especially not for someone who seems to have a lot of history with writing JS engines/Spidermonkey, like the author), given things like webkit's experience: https://webkit.org/blog/5852/introducing-the-b3-jit-compiler... .
The main point it makes is, again "He perfectly demonstrates one of the points my “Oxidizing” article was making: with Rust and WebAssembly we have reliable performance without the wizard-level shenanigans that are required to get the same performance in JavaScript."
This doesn't make a lot of sense as a claim.
Why? Because underneath all that rust .... is an optimizing compiler, and it happens the author has decided to stay on the happy path of that. There is also an unhappy path there. Is that happy path wider? Maybe. It's a significantly longer and more complex optimization pipeline just to wasm output, let alone the interpretation of that output. I have doubts it's as "reliable" as the author claims (among other things, WebAssembly is still an experimental target for LLVM). Adding the adjective "reliable" repeatedly does not make it so.
Let's ignore this though, because there are easier claims to pick a bone with.
It also tries to differentiate optimizations between the two in ways that don't make sense to me: "In some cases, JITs can optimize away such allocations, but (once again) that depends on unreliable heuristics, and JIT engines vary in their effectiveness at removing the allocations."
I don't see a guarantee in the rust language spec that these allocations will be optimized away. Maybe i missed it. Pointers welcome.
Instead, i have watched plenty of patches to LLVM go by to try to improve it's heuristics (oh god, there's that evil word they used above!) for removing allocations for rust. They are all heuristic based, they deliberately do not guarantee attempting to remove every allocation (for a variety of reasons). In general, it can be proven this is a statically undecidable problem for a language like rust (and most languages), so i doubt rustc has it down either (though i'm sure it does a great job in general!)
The author also writes the following: "WebAssembly is designed to perform well without relying on heuristic-based optimizations, avoiding the performance cliffs that come if code doesn’t meet those heuristics. It is expected that the compiler emitting the WebAssembly (in this case rustc and LLVM) already has sophisticated optimization infrastructure,"
These two sentences literally do not make sense together. The "sophisticated optimization infrastructure" is also using heuristics to avoid expensive compilation times, pretty much all over the place. LLVM included. Even in basic analysis, where it still depends on quadratic algorithms in basic things.
If you have a block with 99 stores, and ask LLVM's memory dependence analysis about the dependency between the first and the last, you will get a real answer. If you have 100 stores, it will tell you it has no idea.
What happened to reliable?
Why does this matter? For example: Every time rust emits a memcpy (which is not infrequent), if there are more than 99 instructions in between them in the same block, it will not eliminate it, even if it could. Whoops. Thats' a random example. These things are endless. Because compilers make tradeoffs (and because LLVM has some infrastructure that badly needs rewriting/reworking).
These "sophisticated optimization infrastructures" are not different than JITs in their use of heuristics. They often use the same algorithms. The only difference is the time budget allocated to them and how expensive the heuristics let things get.
There may be good reasons to want to write code in rust and good reasons to believe it will perform better, but they certainly are not the things mentioned above.
Maybe what the author really wants to say is "we expect the ahead of time compiler we use is better and more mature than most JITs and can spend more time optimizing". But they don't.
Maybe it would also surprise the author to learn that their are JITs that beat the pants off LLVM AOT for dynamic languages like javascript (they just don't happen to be integrated into web browsers).
But instead, they make ridiculous claims about heuristics and JITs. Pretending the compiler they use doesn't also depend, all over the place, on heuristics and other things is just flat out wrong. At least to me (and i don't really give a crap about what programming language people use), it makes it come off as rampant fanboism. (Which is sad, because i suspect, had it been written less so, it might be actually convincing)