
Introducing the B3 JIT compiler - basugasubaku
https://webkit.org/blog/5852/introducing-the-b3-jit-compiler/
======
munificent
Really cool article. Posts like this always make me wonder what the state of
the programming would be if browsers hadn't sucked up almost all of the
world's compiler optimizers.

~~~
ajross
To be fair, GPU vendors sucked up a ton too.

But considering that optimized scalar code performance has moved, what, maybe
40% over the last two decades, I'm going to say "not much". Compilers are
sexy, but they're very much a solved problem. If we were all forced to get by
with the optimized performance we saw from GCC 2.7.2, I think we'd all
survive. Most of us wouldn't even notice the change.

~~~
munificent
> Compilers are sexy, but they're very much a solved problem.

Not for all of the other widely-used languages that still have incredibly
simple interpreters. Think how much energy could have been saved if Ruby,
Python, and PHP were all as fast as your average JS engine.

~~~
VeejayRampay
Exactly. Ruby would probably have massive adoption with JS-like speed.

~~~
chrisseaton
My implementation of Ruby, JRuby+Truffle, is as fast as V8

[http://stefan-marr.de/downloads/crystal.html](http://stefan-
marr.de/downloads/crystal.html)

~~~
pizlonator
Note that usually being "as fast as" a production JSVM means also proving that
you can start up as fast as JSVMs do. Have you done this?

~~~
ehsanu1
Search around for "Substrate VM". I see it referenced in some slide decks, and
it's designed to make JVM startup much faster.

Here's an old slidedeck that talks about it:
[http://www.oracle.com/technetwork/java/jvmls2013wimmer-20140...](http://www.oracle.com/technetwork/java/jvmls2013wimmer-2014084.pdf)

~~~
ksec
Wow thx for the reminder, I have forgotten about it already. I remember the
promise of Truffle + Graal + Substrate VM.

My god can't believe it was 3 years ago i read about it on HN.

------
cpr
Good to see the Webkit team (mostly Apple) continue putting serious energy
into JS performance. Take a bit of guts to throw out the whole LLVM layer in
order to get compilation performance...

It's also encouraging to see them opening up about future directions rather
than just popping full-blown features from the head of Zeus every so often.
(Not that they owe us anything... ;-)

(Edit: it's also damned impressive for 2 people in 3-4 months.)

~~~
MrBuddyCasino
Gutsy move indeed. Though I wonder, what was the development cost in real life
cash for gaining those 5% of performance?

~~~
pizlonator
3 months. Two people working on it (me and @awfulben).

~~~
titzer
Nice work, Fil. Looks cool.

~~~
pizlonator
Thanks! :-) Looking forward to more write-ups about TurboFan!

------
legulere
tl;dr: B3 will replace LLVM in the FTL JIT of webkit. LLVM isn't performing
fast enough for JIT mainly because it's so memory hungry and misses
optimisations that depend on javascript semantics. They got an around 5x
compile time reduction and from 0% up to around 10% performance boost in
general.

~~~
zepto
Actually the bigger reason is compile time - better optimizations based on
JavaScript semantics are a secondary advantage.

~~~
pizlonator
I think that's mostly accurate, in the sense that we wouldn't have done this
if it was _only_ motivated by specializing for JavaScript semantics. We had
gotten pretty good at having our high-level compiler (DFG) burn away the
JavaScript crazy and leave behind fairly tight code for LLVM to optimize.

But as soon as we realized that we had such a huge compile time opportunity,
of course we optimized the heck out of the new compiler for the kinds of
things that we always wished LLVM could do - like very lightweight patchpoints
and some opcodes that are an obvious nod for what dynamic languages want
(ChillDiv, ChillMod, CheckAdd, CheckSub, CheckMul, etc).

~~~
dochtman
But isn't it true that some of the things you ended up doing would make sense
for LLVM, or would most of them be invalidated by the kinds of optimization
passes that are common in LLVM?

E.g. stuff like making the in-memory IR representation better cacheable
certainly sounds like it's all-upside, and LLVM should just learn from your
project.

~~~
pizlonator
LLVM's use of a very rich (and hence not as memory efficient) IR is deeply
rooted. Phases assume that given any value, you can trace your way to its
uses, users, and owners. The LLVM code I've played with assumes this all over
the place, so removing the use lists and owner links as B3 does would be super
hard. B3 can do it because we started off that way.

------
alberth
>> "tl;dr: B3 will replace LLVM in the FTL JIT of webkit. LLVM isn't
performing fast enough for JIT mainly because it's so memory hungry and misses
optimisations that depend on javascript semantics. They got an around 5x
compile time reduction and from 0% up to around 10% performance boost in
general." [1]

Is this a knock on LLVM then?

I wonder then specifically if this brings to light any concerns over Swift
(another dynamic language, and was created by the same person who created LLVM
as well). [2]

Seems weird that the original creator of LLVM was able to make a dynamic
language such as Swift - without any problems.

[1]
[https://news.ycombinator.com/item?id=11105231](https://news.ycombinator.com/item?id=11105231)

[2] [http://nondot.org/sabre/](http://nondot.org/sabre/)

~~~
pizlonator
Swift is not a dynamic language. It's statically typed.

~~~
klodolph
This comment, and the parent comment, seem to be conflating "dynamic language"
with "dynamically-typed language".

~~~
derefr
What's a "dynamic language"?

~~~
barkingcat
Dynamic languages are (generally) not compiled ahead of time. It is possible
to have static typing in a dynamic language in the sense that the code is
interpreted at runtime (but the types are set statically in the code), and
there is no "binary" that one runs.

It used to be called interpreted language or "scripting" language - but I
think the vocabulary shifted so the word "dynamic" more encompasses what the
languages are about.

~~~
gpderetta
By that definition, AFAIK, Swift is neither a dynamic language nor dynamically
typed.

~~~
barkingcat
According to Apple itself, Swift is a compiled language, and there is no use
of the word "dynamic" or "dynamic language" on Apple's website

[https://developer.apple.com/swift/](https://developer.apple.com/swift/)

As to whether it is dynamically or statically typed - it's easy to tell - do
you need to indicate a variable is an int or a char or a string before you
start using it? and do you have to cast the variable to different types when
using functions that expect a certain type? If so - it's statically typed.
Dynamically typed languages allows you to give an character "5" to a function
that expects an integer (ie the number 5) and then automatically converts it
so for example the final result to 5+"5" is int(10). Or when you try to print
it, it gives you a string literal "1" and "0" without you having to recast it
yourself.

Obviously having dynamic typing makes things easier for humans, but the
computer has to keep predicting what the programmer is going to do with that
variable, so it has some runtime and compile time weaknesses in performance,
memory usage, as well as less strictness in compile-time checking - which
might allow certain types of errors to sneak by, whereas static typing is very
direct - you have to instruct the computer to do every casting from one type
to another, etc., and the statically-typed compiler is a sadist - it will fail
all your code over and over again until you get all the types right. The
difference is extremely noticeable even for a novice programmer.

[https://developer.apple.com/library/ios/documentation/Swift/...](https://developer.apple.com/library/ios/documentation/Swift/Conceptual/Swift_Programming_Language/TheBasics.html#//apple_ref/doc/uid/TP40014097-CH5-ID322)

says that Swift is "type-safe" (compiler is a sadist and will halt on all type
errors) but has type inference so you don't have to explicitly indicate the
typing of a variable. I would consider it to be heavily on the statically-
typed side though - since it seems the language won't let an integer variable
all of a sudden also be a string - you have to cast it properly first.

Not sure where the confusion comes from, probably from people putting buzz
words to everything that is new regardless of applicability.

Either way, you are correct, swift is not "dynamically-typed" and neither is
it a "dynamic/interpreted" language.

~~~
dragonwriter
> As to whether it is dynamically or statically typed - it's easy to tell - do
> you need to indicate a variable is an int or a char or a string before you
> start using it?

Modern statically typed languages often don't require this, because type
inference, so its a bad test.

> and do you have to cast the variable to different types when using functions
> that expect a certain type?

Modern statically-typed languages may not require you to do this (e.g., Scala
if the types involved have appropriate implicit conversions defined.)

> Dynamically typed languages allows you to give an character "5" to a
> function that expects an integer (ie the number 5) and then automatically
> converts it so for example the final result to 5+"5" is int(10).

That's the canonical example of _weak_ typing; many (maybe most) dynamically-
typed languages do not do this type of conversion.

A better test for a dynamically-typed language is what kind of error is
produced by sending a value of an unexpected tpe (including, as expected,
anything that can be handled by the applicable implicit conversion rules) to a
function: if it is a compile-time error, the language is statically typed. If
it is a runtime error, it is a dynamic language.

------
jlebar
It's worth noticing that most of the optimizations here are for "space" \--
reducing the working set size or the number of memory accesses. CPUs have
gotten much faster than memory blah blah. This is the sort of thing where
microbenchmarks may mislead you, because you WSS is probably not realistic.

I think we don't have great tools for helping with this sort of optimization.
One can use perf to find cache misses, but that doesn't necessarily tell the
whole story, as you might blame some _other_ piece of code for causing a miss.
Maybe I should try cachegrind again...

------
panic
Cool stuff! Does anyone know why the geometric mean is used for averaging
benchmark scores rather than the usual arithmetic mean?

~~~
jsnell
I would say that geometric mean is the usual way of averaging benchmark
scores. It has the property that a given relative speedup on a component
benchmark always has the same effect on the aggregate score. With an
arithmetic mean the component benchmarks with a longer runtime will dominate
the aggregate. Normalizing the results before applying the arithmetic mean
doesn't really help either -- the first X% improvement to a component
benchmark would still be valued more than the second X% speedup.

------
DannyBee
Interestingly, much of their complaints around pointer chasing, etc, are
things LLVM plans on solving in the next 6-8 months. i'm a bit surprised they
never bothered to email the mailing list and say "hey guys, any plans to
resolve this" before going and doing all of this work. But building new JITs
is fun and shiny, so ...

~~~
Joky
LLVM instruction selection is _slow_ , there is a "fast-path" which hasn't
received much attention (it is only used for -O0 in clang). The new
instruction selector work just started and will take a couple of years,
considering the tradeoff between spending 3 months on it and waiting a few
years for LLVM to be improved (without any guarantee of LLVM reaching the same
speed as what they did). See also some thoughts from a LLVM developer on
optimizing for high-level languages: [http://lists.llvm.org/pipermail/llvm-
dev/2016-February/09546...](http://lists.llvm.org/pipermail/llvm-
dev/2016-February/095463.html)

~~~
DannyBee
The problem with the path they've taken is it has a finite end.

This is why, when they compare it to v8/etc, it's kind of funny. They all have
the same curve.

Basically all of these things, _all_ of them, end up with roughly the same
deficiencies once you cherry pick the low hanging fruit[1], and then they
stall out, and get replaced a few years later when someone decides thing X
can't do the job, and they need to write a new one. None of them ever get to a
truly good state.

Rinse, wash, repeat.

The only thing these things make _real_ progress, is by doing what LLVM did -
someone works on it for years.

Let me quote a former colleague at IBM - "there is no secret silver bullet to
really good compilers, it's just a lot of long hard work". If you keep
resetting that hard work every couple years, that seems ... silly.

TL;DR If you really believe they've totally gotten everywhere they need to be
in 3 months, i've got a bridge to sell you

[1] For example, good loop vectorization and SLP vectorization is hard.

~~~
nadav256
Danny, you mentioned loop and SLP vectorization. One thing that bothered me
while watching the development of the SLP and Loop vectorizers over the last
two years was that developers who cared about SPEC added functionality that
increased compile time. I remember one change that added a second phase that
scans the entire IR in the SLP vectorizer to get a 1.4% win in one of the SPEC
programs. I hoped the FTL project would be able to enable the vectorizers, but
to my disappointment the vectorizers are too slow to justify the boost on
javascript workloads.

------
ck2
tl;dr

[https://webkit.org/blog-files/kraken.png](https://webkit.org/blog-
files/kraken.png)

[https://webkit.org/blog-files/octane.png](https://webkit.org/blog-
files/octane.png)

seriously though, dang, how many years of coding to get to that level of
expertise

------
Ecco
I'm really wondering about the politics behind all this. I mean both LLVM and
WebKit are Apple projects (even though they're Open Source). So it would have
been reasonable to expect an improvement of LLVM instead of ditching it
altogether.

~~~
pcwalton
I highly doubt there are any politics behind it. LLVM is an AOT C/C++ compiler
at its core, and the tradeoffs it makes don't always make sense for dynamic,
JIT compiled languages with extreme emphasis on compilation speed like JS.
Personally, I expected this to happen.

