
Multi-Level Intermediate Representation Compiler Infrastructure - matt_d
https://github.com/tensorflow/mlir
======
KenoFischer
Congrats to the MLIR team for getting the source out. I know it must have been
a lot of work. MLIR seems like (or at least aspires to be once complete) a
nice step up from LLVM and addresses a lot of the issues that LLVM has around
extensibility. Looking forward to see where this goes. In particular, it'll be
interesting to see to what extent re-use of passes will work out between
dialects in practice.

I'm also looking forward to digging into the polyhedral compilation stuff
more. It's about time that somebody attempted a ground up re-think of that, so
I'm looking forward to understanding it better. I've talked to Chris a number
of times about the rest of the design, but we never got into the specifics of
the polyhedral compilation.

That said, I think it's a bit unfortunate that this ended up in TF rather than
LLVM. TF of course needs good compiler tech more than most projects at this
point, which is why MLIR came from that direction, but MLIR is much more of a
general compiler project. I worry that putting in TF will discourage the rest
of the ML industry from adopting it as eagerly (everybody is already using
LLVM, so that branding would make that much easier). Also, looking at the
code, MLIR looks structurally a lot more like an LLVM-heritage C++ project
than a TF one (hell, you build it by cloning llvm and putting MLIR in the
projects/ directory). Perhaps moving it under the LLVM umbrella is still in
the cards - I'll keep advocating for it at least ;).

~~~
chrislattner
Thank you for the kind words Keno.

We don't have a great description of what the polyhedral dialect is, but we do
have a discussion of what it isn't, here:
[https://github.com/tensorflow/mlir/blob/master/g3doc/Rationa...](https://github.com/tensorflow/mlir/blob/master/g3doc/RationaleSimplifiedPolyhedralForm.md)

There are other docs in the same directory that you might find interesting.

We don't have all the details sorted out, but Google would like to contribute
at least some part of MLIR to the LLVM.org project, but you have to ask LLVM
for acceptance, and that requires putting the code up somewhere first.

All things in time, I hope to catch you at EuroLLVM next week where we are
giving the keynote and a tutorial on MLIR.

-Chris

~~~
KenoFischer
Yep, I've been reading the docs with great interest. I'm also looking forward
to getting the XLA/HLO dialect available. I'm thinking that might be a neat
way to get immediate use out of MLIR in the julia world. There's a bunch of
extra HLO passes I need to write for TPU performance and I figure might as
well try to do that using MLIR.

Good to hear that getting this into LLVM is still in the cards - totally
understood that there are organizational challenges, just want to keep pushing
in the right direction :). My schedule doesn't allow me to be at EuroLLVM this
year unfortunately, but I'm sure we'll get an opportunity to catch up before
long. Now that MLIR is out it'd be fun to dig in in detail.

------
CalChris
Well, MLIR is getting its star turn at the LLVM meeting later this week in
Brussels. I've flipped through the MLIR rationale doc and the earlier CGO
presentation [1]. Correct me if I'm wrong but MLIR is meant for higher level
optimizations which are then lowered to a more LLVM-like dialect for actual
codegen.

My question is does this mean anything to non-TensorFlow-like backends? Is the
aarch64 backend going to change at all?

[1]
[https://drive.google.com/file/d/1hUeAJXcAXwz82RXA5VtO5ZoH8cV...](https://drive.google.com/file/d/1hUeAJXcAXwz82RXA5VtO5ZoH8cVQhrOK/view?usp=sharing)

~~~
chrislattner
The MLIR project has no plans to reimplement X86 or ARM backends. The LLVM
Code generator could be improved of course (e.g. through the GlobalISel
project) but there is no need to do that in the context of MLIR - MLIR
supports codegen through LLVM.

------
lostmsu
Honestly, I am not sure what purpose does that serve. I read the motivational
docs, and they seem to be clear, but what they describe is like a
representation of IR at a specific compilation step, which has some predefined
metadata for a specific use case + restrictions on data flow.

IMHO, the hardest part is obtaining that metadata and converting normal code
into that form, not compilation of that form further down into the machine
code.

~~~
jcranmer
The entire compilation process is generally a process of first converting code
into a very high-level representation, and then lowering it through stages
that progressively lose the original language semantics and gain hardware-
specific semantics, with optimizations happening along each level of the path.

Traditional LLVM contains essentially three different levels of IR: there's
the mid-level IR that most people think of, which is (to a very, very rough
approximation) a machine-independent (but generally not portable)
representation of C/C++ (with extensions) semantics. This is lowered to a
legalization IR (either SelectionDAG or GlobalISel), which contains the same
general class of generic operations, but things such as integers larger than
native machine types (e.g., i128) are converted into native machine types [1].
The legalized IR is then lowered to a machine-specific IR, where the opcodes
and operands are specific to the relevant ISA, and the target-independent
passes are generally restricted to dealing with very high-level summaries of
operations (e.g., "this instruction is commutative").

This project is about an IR that is higher level than LLVM IR that can, to
some degree, abstract over the different kinds of such IRs that already
exists. (As the presentation notes, Clang _should_ have built such an IR,
especially now that it needs to do a lot of that work for static analysis or
constexpr anyways).

[1] More specifically, this is the IR that the legalization process is
actually done in. How much of it has been legalized depends on where you are
in the process.

