
How does LuaJIT's trace compiler work? - eloff
http://www.freelists.org/post/luajit/How-does-LuaJITs-trace-compiler-work,1
======
cloudflare
From the post:

    
    
        A recent example illustrates the power of this approach:
    
        Cloudflare's WAF (web application firewall) basically generates
        Lua code for the (highly non-linear) maze of firewall rules. An
        incoming attack triggers certain rules and the corresponding paths
        are turned into linearized traces. These can be heavily optimized
        by LuaJIT, much more so than you could ever hope to do with a
        static compiler.
    
        When a different attack wave is started, it'll spawn different
        traces -- i.e. the generated code dynamically adapts to the
        specific attacks.
    
        The result is a much higher firewall throughput than you might be
        able to achieve with a data-driven or a static compilation
        approach. This directly equates into money saved on machines
        needed to handle the load.
    

Added to that is the fact that the WAF configuration (which is automatically
generated Lua code) is different for each customer and so we rely heavily on
caching and JITing to get the performance.

The bottom line is that LuaJIT is very fast and very flexible.

~~~
JoeAltmaier
Wow I'd need some evidence to believe JIT can do a better job than a 'static'
compiler. What can JIT do to optimize 'much more so'? How about a little more?
If you know some common conditional jump stats you can do slightly better. Is
there anything else?

~~~
rayiner
A static compiler has to generate code that handles every possible path
through the code. This tends to pessimize code downstream of a control flow
merge point, because such code has to be compiled assuming it can be reached
via multiple paths. A trace compiler can compile only the paths that are
actually taken on a given workload, and compile the (dynamically) uncommon
cases into exits to the interpreter.

The firewall rule example Mike Pall gives is a pretty good one. You have code
that has many different paths through it, but only specific ones are
triggered, dynamically, by a particular attack. Those can be compiled into
traces that assume the other paths are not taken (except to guard the possible
control flow splits with trace exits). This can allow an optimizer to make
much more aggressive assumptions on the actually-taken paths.

~~~
JoeAltmaier
My point exactly. This slight optimization is all you get. I think this
technology is being oversold if that's all they got.

------
mikemike
Err, the title of this item is a bit misleading, although it's the subject of
the posting on the mailing list.

One cannot explain it all in a single post. I just answered some specific
questions, omitting most details.

------
pygy_
Later in the thread:

    
    
            From: Mike Pall
    
        ...
    
        [I refuse to contribute to stackoverflow anymore, due to some
        recent incidents. I can't even edit *my own* answers to correct
        them, without some anonymous fool rejecting my edits. Ok, so maybe
        they don't know I wrote the damned thing. But then they really
        shouldn't be allowed to moderate Lua-specific questions. This is
        just plain unacceptable.]
    
        --Mike
    

This really is a shame ( _edit: the SO situation, not Mike 's decision. He's
the one who should have mod powers_).

~~~
Tobu
He's being a prima donna. Edits by a third-party get reviewed, there is no way
you could accept that edit (which was from a different account) without
allowing other random people to edit his posts in the same way. You can't
blame stack overflow for erring of the side on respecting the author's intent.

~~~
pygy_
FYI, he's participating in this thread.

------
ajtulloch
Somewhat tangential - the LuaJIT source [1] is an absolute masterpiece of
software engineering in C. The detailed comments (especially on the
optimizations - "luajit/src/lj_opt...") are incredibly interesting reading.

One of my favourite quotes is at
[https://github.com/LuaDist/luajit/blob/master/src/lj_opt_nar...](https://github.com/LuaDist/luajit/blob/master/src/lj_opt_narrow.c#L155)

    
    
      ** [A recursive backpropagation algorithm with backtracking, employing
      ** skip-list lookup and round-robin caching, emitting stack operations
      ** on-the-fly for a stack-based interpreter -- and all of that in a meager
      ** kilobyte? Yep, compilers are a great treasure chest. Throw away your
      ** textbooks and read the codebase of a compiler today!]
    
    

[1]: [https://github.com/LuaDist/luajit](https://github.com/LuaDist/luajit)

~~~
malkia
I've borrowed the way Mike writes his enum macros.

------
eloff
Also of interest, are trace compilers suited for static languages?

[http://www.freelists.org/post/luajit/Are-trace-compilers-
sui...](http://www.freelists.org/post/luajit/Are-trace-compilers-suited-for-
static-languages,1)

~~~
rayiner
Some of the original work on trace compilation was done in the context of
Java, a static language:
[http://static.usenix.org/event/vee06/full_papers/p144-gal.pd...](http://static.usenix.org/event/vee06/full_papers/p144-gal.pdf).

What trace compilation buys you is: 1) elimination of method call boundaries
in analysis; 2) elimination of data flow merges.

Dynamic languages benefit particularly from these characteristics because
their semantics are replete with method calls and data flow merges (e.g. after
any generic operation). But static languages can have these qualities too.

~~~
mikemike
To give credit, where credit is due: the original work on trace compilation is
much, much older. The paper you cited is an application.

The fundamental papers to hunt for are Joseph A. Fisher's publications on
trace scheduling (sadly, his PhD thesis from the 70ies is nowhere to be found
online) and the Multiflow reports from the 90ies. The Dynamo paper built upon
that foundation ten years later in '99 (get the full HP report, not the short
summary). A related research area is about trace caches for use in CPUs with
various papers from the 90ies.

AFAIK there's no up-to-date comprehensive summary of the state of research on
trace compilers. Most papers don't even scratch the surface of the challenges
you'll face when building a production-quality trace compiler.

~~~
chrismonsanto
> AFAIK there's no up-to-date comprehensive summary of the state of research
> on trace compilers. Most papers don't even scratch the surface of the
> challenges you'll face when building a production-quality trace compiler.

In general, there are few good + comprehensive resources for advanced
compilation techniques. I would gladly fork over $$$ if you wrote a textbook
on tracing compilers aimed at people who have expert-level knowledge in more
traditional compilation methods (aka, not JIT).

~~~
sitkack
I second this, Mike Pall should write a book. I will buy any book Mike Pall
writes.

~~~
swah
I'll read any book Mike Pall recommends.

------
rombix
Somewhat related discussion about a possibility of building tracing JITs for
other languages on top of the LuaJIT code base:

[http://www.freelists.org/post/luajit/To-which-extent-
LuaJIT-...](http://www.freelists.org/post/luajit/To-which-extent-LuaJIT-is-
specific-to-Lua)

