
Ask HN: Have tracing JIT compilers lost? - bakery2k
The title refers to the famous 2010 thread &quot;Have tracing JIT compilers won?&quot; [1] on &quot;Lambda the Ultimate&quot;. There, it was hypothesized that for dynamically-typed languages (JavaScript with Firefox&#x27;s TraceMonkey and Lua with LuaJIT), &quot;tracing JITs are turning up as the winners&quot; and that per-method JITs, such as V8, may not &quot;be able to keep up&quot;.<p>Since then, however, Firefox has left TraceMonkey behind and switched to a succession of method JITs, and in general, the major JavaScript engines have converged on an implementation strategy of a fast interpreter and one or more JIT compilation tiers (V8&#x27;s TurboFan [2]&#x2F;JavaScriptCore&#x27;s Baseline &amp; DFG &amp; FTL [3]&#x2F;SpiderMonkey&#x27;s Baseline &amp; IonMonkey [4]&#x2F;ChakraCore&#x27;s Simple &amp; Full [5]).<p>Notably, none of these JavaScript JITs use tracing. Other dynamic language JITs that have appeared since 2010 (e.g. Julia, Pyston) have also avoided tracing. The only remaining major tracing JITs seem to be LuaJIT (which is no longer receiving new features) and PyPy (which uses a unique meta-tracing approach).<p>Contrary to what was predicted in 2010, it appears that for dynamic languages, method JITs have won over tracing JITs. For what reason(s) have tracing JITs been superseded? Have method JITs proven:<p>- Less complex to develop and maintain?<p>- Faster in the best&#x2F;average case?<p>- To provide more consistent performance, i.e. less slowdown in the worst case?<p>Do you see any future for tracing JIT compilers, or do you expect future dynamic language JITs to only use the per-method strategy? Does tracing have <i>any</i> advantages - are there any situations in which tracing may be a good choice, or have tracing JITs proven to be a dead end?<p>[1] lambda-the-ultimate.org&#x2F;node&#x2F;3851 | [2] v8project.blogspot.com&#x2F;2017&#x2F;05&#x2F;launching-ignition-and-turbofan.html | [3] webkit.org&#x2F;blog&#x2F;3362&#x2F;<p>[4] blog.mozilla.org&#x2F;javascript&#x2F;2013&#x2F;04&#x2F;05&#x2F;the-baseline-compiler-has-landed&#x2F; | [5] github.com&#x2F;Microsoft&#x2F;ChakraCore&#x2F;wiki&#x2F;Architecture-Overview
======
tarmit
> To provide more consistent performance, i.e. less slowdown in the worst
> case?

AFAICT, this is the reason for the decline of trace-based JIT compilers. When
Mozilla introduced JaegerMonkey, their first method-based JIT (which went on
to make TraceMonkey obsolete), their reasoning was "tracing tends to interact
poorly with certain styles of code" \- specifically "the downside of the
tracing JIT is that we have to switch back and forth between the [slow]
interpreter and the machine code [...] a lot"
[[https://hacks.mozilla.org/2010/03/improving-javascript-
perfo...](https://hacks.mozilla.org/2010/03/improving-javascript-performance-
with-jagermonkey/)].

For an example of a situation in which tracing performs poorly, consider a hot
trace through an if-else statement. The trace will only contain one of the
statement's two branches - leading to reduced performance if the "off-trace"
branch is frequently taken.

> Faster in the best/average case?

The issue with tracing isn't best-case performance - again, from Mozilla:
"when we’re able to “stay on trace” TraceMonkey wins against every other
engine" [ibid]. Also, it seems tracing can compete in the average case against
somewhat-advanced method-based JITs: "type inference was added to
JaegerMonkey, which made it faster on average"
[[https://blog.mozilla.org/nnethercote/2011/11/23/memshrink-
pr...](https://blog.mozilla.org/nnethercote/2011/11/23/memshrink-progress-
report-week-23/)].

> Less complex to develop and maintain?

In fact, I believe a good method-based JIT is probably _more_ complex than a
trace-based one. When compiling at method-granularity, with incomplete
information about types, complex features are required e.g. On-Stack
Replacement (if the current method becomes "hot", it needs to be replaced by a
compiled version), Inline Caches, Hidden Classes and Deoptimization.

In a trace-based JIT, all these features can be replaced by "just compile
another trace".

~~~
tarmit
> Does tracing have any advantages?

In summary, I think the advantage of tracing is its ability to give very good
best- and average-case performance (albeit with poor worst-case performance),
with less complexity than method-based compilation. This is a good trade-off
for LuaJIT (a few people supporting ~50kLoC and specifically targeting a small
footprint), but not for the leading JavaScript engines (who can afford large
teams and millions of LoC).

