

LLVM based JavaScript JIT that beats V8 on Asm.js benchmarks - officialjunk
https://trac.webkit.org/wiki/FTLJIT

======
zurn
The web page doesn't say anything about asm.js - so this is a generic JS JIT
that happens to beat V8 in asm.js?

V8 doesn't explicitly support asm.js so it's easier to beat than Firefox.

~~~
sanxiyn
Yes, it is generic.

~~~
cliffbean
It is generic, but the graphs on arewefastyet.com suggest that it isn't
actually making most regular benchmarks go faster, just some of the asm.js-
style ones.

~~~
azakai
It makes one or two of the Octane benchmarks faster, like Mandreel. But
Mandreel is quasi-asm.js as well.

A powerful JIT like LLVM will tend to do best on large long-running
benchmarks, like asm.js.

------
officialjunk
Performance benchmarks:
[http://arewefastyet.com/#machine=12&view=breakdown&suite=asm...](http://arewefastyet.com/#machine=12&view=breakdown&suite=asmjs-
apps)

~~~
joosters
All the graphs are missing the keys to identify which coloured line represents
which machine/setup. And the y-axes aren't labelled, so you can't tell whether
higher is better or not :(

~~~
sanxiyn
Quoting from "About" link:

"Some benchmarks use time (lower is better), and some use points (higher is
better). We orient all graphs so lower is better, visually."

------
sanxiyn
Note that while it beats V8, it does not beat SpiderMonkey.

------
thinxer
Does "FTL" stands for "faster than light"?

~~~
masklinn
Er… first phrase of the page:

> JavaScriptCore uses LLVM for a top-tier high-throughput optimizing JIT,
> which we call _the FTL (Fourth Tier LLVM)_.

~~~
aidenn0
And I think the double meaning is about as transparent as "Simultaneous
Peripheral Operation OnLine"

------
notthetup
Wait.. I thought V8 doesn't support asm.js? Or are they comparing it against a
fork of V8 which does support asm.js..

Because if they're saying the new webkit jsc (using asm.js) is faster than V8
(not using using asm.js), then it's not really a fair comparison.

~~~
ahoge
V8 supports asm.js as any other ES3-compliant JavaScript engine. It also has
many optimizations in place which benefit from asm.js-style code. Asm.js-style
code also simply doesn't do those expensive JavaScript-y things. That's why
that Citadel demo runs just fine in Chrome, too.

However, V8 doesn't have a separate mode. The "'use asm';" pragma is ignored.

The Octane benchmark suite does contain Emscripten and Mandreel benchmarks
though. So, they are tracking it and they also try to make this kind of code
run faster.

~~~
hyperpape
Not only that, a few months after asm.js was included in Firefox, Chrome had
optimizations done that helped a great bit with asm.js
([http://blog.chromium.org/2013/05/chrome-28-beta-more-
immersi...](http://blog.chromium.org/2013/05/chrome-28-beta-more-immersive-
web.html)). Possibly a coincidence, but they certainly seem to care about
asm.js performance above and beyond generic JS performance, even if they're
not going to support the pragma.

~~~
magicalist
> _but they certainly seem to care about asm.js performance above and beyond
> generic JS performance_

That doesn't follow. They certainly care about it; it's now in the Octane
benchmark suite, which means they think it's representative of code that
should be optimized for, but it's only _part_ of the suite, which means it's
just one more style of code that should be able to run efficiently in a modern
JS engine.

~~~
Alphasite_
You're oth saying the same thing for the most part, I dont think he's trying
to imply that its their focus, just that its important enough to optimise
specifically.

------
marijn
Does anyone have numbers handy on the amount of size linking LLVM would add to
a browser? Installing llvm-3.2-dev gives me over 75mb of static libraries, but
I don't know how many of those would need to be shipped to support such a JIT.

~~~
joakleaf
Did you try to strip the libraries for debug symbols?

~~~
sanxiyn
Assuming llvm-3.2-dev refers to Ubuntu package of the name, it is already
stripped of debug symbols.

------
stuaxo
Now feed the output of that into enscripten ...

------
mantrax5
To have a meaningful "victory", you want to have one of those:

    
    
      1. A generic JS JIT that runs generic JS faster than another generic JS JIT.
    
      2. ASM.JS JIT that runs ASM.JS faster than another ASM.JS JIT.
    

What we have here is something else:

    
    
      3. A generic JS JIT that runs ASM.JS faster than another generic JS JIT.
    

If people are not familiar with ASM.JS, the somewhat misleading (to the
uninitiated) headline sounds more interesting than it is actually.

It's not a significant milestone, as SpiderMonkey's ASM.JS implementation
outperforms this one, and V8 isn't implementing ASM.JS in any way. As in _at
all_. So of course it'll be slower.

The purpose of ASM.JS is to allow a separate ASM.JS runtime to run ASM.JS
code. The reason it happens to be a JS subset, is so you can have a fallback
for backwards compatibility.

Not to say this LLVM engine doesn't show potential. But implementing ASM.JS in
LLVM is not the hard part of implementing a generic JS JIT.

ASM.JS is entirely static, so it's comparatively trivial to JIT, it's
literally an ASM to ASM transpiler.

~~~
sanxiyn
Edit: the particular quote I objected to was edited out by parent.

I object to "all the dynamicness of the rest of the JS is what V8 excels at".
V8 excels at a subset of dynamic JavaScript (namely those benefit from
Crankshaft) and is not very good outside of the subset.

Typescript benchmark included in Octane benchmark suite is a good example. In
Google's own words: "This test measures the time TypeScript takes to compile
itself and is a proxy of how well a VM handles complex and sizable Javascript
applications".

In particular, Crankshaft is not effective for this benchmark. (Check yourself
by running it with Crankshaft disabled: the score actually improves because
you save compilation effort.) And as you can check yourself or check on
AreWeFastYet, JavaScriptCore beats V8 by 2x on this benchmark.

I admit Crankshaft subset of JavaScript is large and interesting (certainly
compared to asm.js subset) but it is not true that V8 excels at dynamic
JavaScript, for sufficiently large value of dynamic. JavaScriptCore does.

~~~
mraleph
TypeScript benchmark is one of the Octane benchmarks that does not do warm up,
so Crankshaft does not have chance to fully shine [which is totally fair given
how TypeScript compiler is used: you don't warm it up, you just spawn it again
and again]. It also has an extremely flat profile. So you can't simply say
that "Crankshaft is not effective for it". Though potentially you can say
"benefits of Crankshafting do not compensate for the cost of Crankshafting".
If you enable warmup for it then score jumps by a factor of 2 for V8.

JSC might be warming faster or simply have a better cost / benefit ratio at
lower tiers. I doubt FTL kicks at all for this benchmark given how short
running it is. V8 also deopts quite a bit on it, so it might be some
pathological case... This is definitely something worth exploring and fixing
on V8 side.

