
Intel Haskell Research Compiler - mpweiher
https://github.com/IntelLabs/flrc
======
tetrep
In essence, a big reason why the Intel team was able to do this was because
they're getting optimized IR from GHC's front-end[0], and then applying
further optimizations with their compiler (HRC) and then converting it into
yet another IR (MIL) used by yet another compiler (FLRC), designed by Intel
for arbitrary functional programming languages, for some further optimizations
and ultimately into C where Intel's C compiler does the rest of the work.

It's interesting that HRC makes relatively significantly optimized programs
(up to 2x), but that GHC's runtime (which HRC is not using) is so well
optimized that the performance of HRC programs is roughly on par with those
from GHC, despite the programs themselves being more performant.

I find stuff like this to be a testament to the practicality of great design
in functional programming language ecosystems: Even the compilers are
composable!

[0]: For those unfamiliar with what GHC does/how it's designed, this talk
gives a great overview of the Core language that is the optimized IR of GHC:
[https://www.youtube.com/watch?v=uR_VzYxvbxg](https://www.youtube.com/watch?v=uR_VzYxvbxg)

~~~
nickpsecurity
That first paragraph was a fun summary. It sounds like they're just throwing
proven tools and tactics at a tough problem getting some good results. They
start with output of one, high-quality tool. Like prior projects, they find an
intermediate point in the transition that lets them add or experiment with
optimizations. Then, they feed it into another high-quality tool to get even
more out.

Looks like good research mixed with good, engineering tradeoffs.

------
georgewsinger
For context, here's a quote from an abstract elsewhere on the web:

> The Intel Labs Haskell Research Compiler uses GHC as a frontend, but
> provides a new whole-program optimizing backend by compiling the GHC
> intermediate representation to a relatively generic functional language
> compilation platform.

and later:

> For certain classes of programs, our platform provides substantial
> performance benefits over GHC alone, performing 2x faster than GHC with the
> LLVM backend on selected modern performance-oriented benchmarks; for other
> classes of programs, the benefits of GHC's tuned virtual machine continue to
> outweigh the benefits of more aggressive whole program optimization. Overall
> we achieve parity with GHC with the LLVM backend.

Source: [https://www.semanticscholar.org/paper/The-Intel-labs-
Haskell...](https://www.semanticscholar.org/paper/The-Intel-labs-Haskell-
research-compiler-Liu-Glew/d90a018d01c5865f57db29bfb8d80ae46e7fcd61)

------
larsberg
It's wonderful to see this released! There was so much great research going on
here that overlapped with work we were doing in Manticore and that was going
on in other optimizing compilers for statically typed high-level languages,
but the details always required talking to Neal or Leaf.

I can't wait to dig in and finally learn more than I could at after-conference
drinks :-)

------
amelius
Note:

> FLRC is open sourced as is. We at Intel Labs are no longer actively working
> on this compiler.

~~~
dbcurtis
Indeed. Fire and forget. Sad.

Some number of years ago I worked in Intel's software labs. It really drove
home to me the point that software development is a long attention span
activity, and chip development is a short attention span activity. Mixing both
at the same company is difficult. A chip design has a short shelf life, and
zero field upgradability, so to be successful building chips you drive hard to
tape out, and then move on to the next iteration as soon as possible.
Software, OTOH, is a many-years-long process of incremental improvement and
field updates.

Trying to do software development in a company where much of the management
are "sand heads" (physical chemists that came up through the chip world)
requires dealing with a huge cultural communication gap.

~~~
DigitalJack
The chips I build are in the field for decades. Compare that to the JavaScript
flavor of the hour. Seems more like ADD.

Common Lisp is about the only software I would consider declaring long
attention span.

The fixed nature of a chip requires far more attention than something you can
push an update to.

"Ship it and fix it later" was pioneered by the software industry. That is the
essence of short attention span, unconcern about details.

Maybe we mean different things by that.

~~~
antonvs
> Compare that to the JavaScript flavor of the hour

That's a rather selective and misleading comparison. The Javascript language
is over 20 years old.

The uses of Javascript you're presumably referring to are mainly in user
interfaces, a deceptively challenging problem which hardware companies are
notoriously bad at.

> Common Lisp is about the only software I would consider declaring long
> attention span.

C is much older than Common Lisp. C++ is about the same age as Common Lisp.

But perhaps you're thinking of Lisp in general, including the predecessors to
Common Lisp, in which case you should probably also mention Fortran.

Looking at other widely used programming languages today, Python is over 25
years old, only about 7 years younger than Common Lisp. Java is also more than
20 years old.

All of these languages are still widely used, in many actively used
applications that themselves are 20+ years old, in every conceivable
application domain.

In short, the distinction you're trying to make isn't at all clear.

~~~
kazinator
ANSI CL: 1994 ANSI C: 1989

"Much older" here denotes five years.

The ANSI C committee was formed in 1983. The Lisp one in 1986; however, by
then Common Lisp was already a thing with Guy Steele's book published (1984).
That all started around 1982, supposedly. (Wikipedia: _" Work on Common Lisp
started in 1981 after an initiative by ARPA manager Bob Engelmore to develop a
single community standard Lisp dialect.[6] Much of the initial language design
was done via electronic mail.[7][8] Guy Lewis Steele, Jr. gave at the 1982 ACM
Symposium on LISP and functional programming the first overview of Common
Lisp."_)

ANSI CL is rooted in some Lisp dialects which precede the entire development
of C.

------
aurelius
How ironic is it that the Haskell Research Compiler is written in Standard ML,
and not Haskell? Joking aside, SML is a great language, and often overlooked,
so this is good to see!

~~~
aseipp
Word I heard was that it was originally designed as a compiler for both
Standard ML _and_ Haskell. I think the Haskell part only came later.

~~~
drtodd13
Haskell was the second frontend. Don't think we ever did a SML frontend. We
can't really discuss what the original frontend was so it was scrubbed from
the release.

~~~
nickpsecurity
Is that because it became an internal product with competitive advantage or
something more boring? I've abstracted away from detail enough so you could
hopefully answer.

~~~
drtodd13
It's not an internal product but I wouldn't call it boring either. There were
some new ideas and some non-technical reasons why they couldn't be used at
that time but they could still be viable and potentially something we wouldn't
want public.

------
framp
This is the original paper from 2013:
[http://www.leafpetersen.com/leaf/publications/hs2013/hrc-
pap...](http://www.leafpetersen.com/leaf/publications/hs2013/hrc-paper.pdf)

GHC can use this intermediate code (a-la LLVM) to produce executable. The idea
is to create an intermediate code which is better optimised for their CPU.

------
winter_blue
I feel like other functional language compiler writers could target some layer
of _GHC / Intel HRC / FLRC_ compiler stack, as a target (instead of say,
generating LLVM bitcode or even machine code directly). As someone interested
in programming language design (and as a big believer in strong, static
typing), I'll consider targeting one of the layers of this existing compiler
stack, if I were to build a compiler for a new language.

Also, the related paper for this project is quite interesting and insightful:
[http://www.leafpetersen.com/leaf/publications/hs2013/hrc-
pap...](http://www.leafpetersen.com/leaf/publications/hs2013/hrc-paper.pdf)

------
nitrix
Wait... what? "To actually compile a Haskell program, we'll also need a
patched version of GHC. See Building and Using GHC with HRC for more
information."

~~~
tikhonj
Think of this as closer to an optimizing backend for GHC than a completely
brand-new compiler. It uses GHC to parse, typecheck and desugar Haskell to an
intermediate form called Core and uses that to do its own optimization and
code generation.

A core design principle of Haskell is that while the whole language has gotten
relatively complex thanks to all its language features and extensions, almost
everything can be simplified to a _really_ small and elegant core language.
This core language is a typed lambda calculus that looks _a lot_ like a subset
of Haskell except with a few changes like no type inference and different
rules for strictness.

GHC then uses this pared down version of Haskell (appropriately called Core)
for the rest of its optimization and compilation. This means that once the
first pass is done with type inference, type checking, typeclass resolution
and a lot of other high-level transformations, the rest of the compiler
doesn't have to worry about them at all. This makes all of GHCs optimizations
easier to implement and maintain, and it lets us add features to Haskell
without needing to change the backend.

~~~
jl6
Is there any chance they might miss an opportunity for optimisation by
throwing out the sugar, thereby discarding information about the prgrammer's
intent?

Or does the sugar carry no such information?

~~~
tikhonj
There is, but it's pretty low because language extensions are _designed_ with
Core in mind and the desugaring tends to be relatively straightforward. In
most cases, the semantics of the "sugar" is defined in terms of a simpler
subset of the language, so there is very little the compiler could do
differently.

More importantly, any modern compiler is very far from the producing the most
optimal possible programs. Overall performance could be improved pretty much
anywhere, and the tradeoffs involved make operating on the "whole" language a
far lower priority than pretty much anything else.

