
A Brief History of the BEAM Compiler - jxub
http://blog.erlang.org/beam-compiler-history/
======
ryanianian
I'm always surprised and delighted to see so many "first implementations" of
languages in prolog. It was described to me in college as "just a novelty" but
in fact it's used extensively in the JVM internals [1] and apparently is the
starting-point impl for other languages.

But prolog still seems so..awkward. I wonder why langs like Haskell or OCaml
aren't more de-facto for these purposes; they seem to have similar expressive
power for parser/grammar things and with less inside-out paradigms (imho).

[1]
[https://docs.oracle.com/javase/specs/jvms/se8/jvms8.pdf](https://docs.oracle.com/javase/specs/jvms/se8/jvms8.pdf)

~~~
segmondy
"seem so..awkward"? Sure because you don't understand prolog. it's very easy
to implement languages.

Look how simple I can compile down to asm code in 35 lines of pure prolog
code.

[https://github.com/segmond/PrologThingz/blob/master/clause_a...](https://github.com/segmond/PrologThingz/blob/master/clause_and_effect/compilation/cg_flatten.pl)

    
    
      /*
      	c := 1;
      	r := 1;
      	while c < n do
      	     (c := c + 1;
      	      r := r * c)
      */
      program(1, (
          assign(c,1);
          assign(r,1);
             while((c < n),
      	  (assign(c, c+1);
                 assign(r, r*c))))).
    
      ?- compile(1).
        movc(1,r(1))
        stm(r(1),c)
        movc(1,r(1))
        stm(r(1),r)
      label(_G3638)
        movm(c,r(1))
        movm(n,r(2))
        cmp(1,2)
        bge(1)
        movm(c,r(1))
        movc(1,r(2))
        add(r(2),r(1))
        stm(r(1),c)
        movm(r,r(1))
        movm(c,r(2))
        mul(r(2),r(1))
        stm(r(1),r)
        br(_G3638)
      label(1)

~~~
triska
This is great, thank you for sharing this!

Your Prolog compiler is also notable because it can be easily made so general
that it not only compiles, but even _generates_ programs.

For example, if I use pattern matching to generalize the first two clauses
via:

    
    
        cg(i(I), R, [movc(I, r(R))|Z]-Z).
        cg(a(A), R, [movm(a(A), r(R))|Z]-Z).
    

then we can already use the same code to generate programs and compiled
instructions:

    
    
        ?- cg(Prog, N, Asm-[]).
        Prog = i(_5354),
        Asm = [movc(_5354, r(N))] ;
        Prog = a(_5354),
        Asm = [movm(a(_5354), r(N))] .
    

The other clauses can be also generalized, using integer constraints to make
them usable in all directions.

For example, if we write:

    
    
        cg(X+Y, R, C0-C2):-
                cg(X, R, C0-C1),
                R1 #= R + 1,
                cg(Y, R1, C1-[add(r(R1),r(R))|C2]).
    

then we can compile an _addition_ whose operands are not even yet fully known:

    
    
        ?- cg(i(X)+i(Y), N, Asm-[]).
        Asm = [movc(X, r(N)), movc(Y, r(_8850)), add(r(_8850), r(N))],
        N+1#=_8850.
    

Still, we can inspect the general pattern that is generated as a result of the
compilation, and therefore write for example test cases that subsume many
possible concrete cases!

On a general note, one reason why Prolog is so popular for writing compilers
is that the language has a built-in mechanism for writing _parsers_ , and in
fact describing all kinds of sequences (for example, of tokens, of
instructions etc.) in a very convenient way.

For example, the above three clauses can be written equivalently as a so-
called _definite clause grammar_ (DCG):

    
    
        cg(i(I), R) --> [movc(I, r(R))].
        cg(a(A), R) --> [movm(a(A), r(R))].
        cg(X+Y, R)  -->
                { R1 #= R + 1 },
                cg(X, R),
                cg(Y, R1),
                [add(r(R1),r(R))].
    
    

This makes it very easy to see the key aspects of the code. At the same time,
the description is so general that it can be used in all directions, to
generate, parse and test instructions.

Sample query:

    
    
        ?- phrase(cg(i(X)+i(Y), N), Ls).
        Ls = [movc(X, r(N)), movc(Y, r(_2288)), add(r(_2288), r(N))],
        N+1#=_2288.

------
spinningslate
Prolog and Erlang in the same thread - is it Christmas already? Two of my
favourite languages, both hugely undervalued.

I remember the moment I first "got" Prolog. Having only programmed in C/C++ to
that point, it took some mental contortion. But the "oh!" moment was amazing.
Backward chaining & logical variables are just so incredibly elegant. I still
use transitive closure as an example of just how powerful the language is:

path(A, B) :- link(A,B).

path(A, B) :- link(A,I), path(I,B).

That's it. Stunningly simple.

As for Erlang - as others have noted - it was conceived right from the start
as a language for concurrency. It's had 20 years to mature and robustly solve
problems that inherently single-threaded languages have barely even recognised
yet (C/C++/Java/C#/python/javascript/rust/...). Supervisors / error handling
and threading are two notable examples. It makes me sad to look at the spread
of "async". An inelegant sticking plaster that adds yet another concurrency
construct and turns it into a tactical decision - on _every method_ \- and at
the call site too.

Erlang shows that a single, well-conceived and executed construct is both
possible and preferable. Lovely.

~~~
simplify
Can you explain what transitive closure is and how your prolog snippet
accomplishes it?

~~~
carapace
[https://en.wikipedia.org/wiki/Transitive_closure#In_graph_th...](https://en.wikipedia.org/wiki/Transitive_closure#In_graph_theory)

> In computer science, the concept of transitive closure can be thought of as
> constructing a data structure that makes it possible to answer reachability
> questions. That is, can one get from node a to node d in one or more hops? A
> binary relation tells you only that node a is connected to node b, and that
> node b is connected to node c, etc.

That snippet defines a function path(A, B) by specifying two cases:

1\. a link(A, B) exists

-or-

2\. some element I exists, such that a link(A, I) exists and path(I, B)
exists.

------
nathan_long
If you'd like a high-level discussion of why the BEAM was designed as it was,
check out my recent post: [https://dockyard.com/blog/2018/07/18/all-for-
reliability-ref...](https://dockyard.com/blog/2018/07/18/all-for-reliability-
reflections-on-the-erlang-thesis)

The TL;DR is: reliability was the main goal, and using isolated processes
which can react to each others' failures, even when running on separate
machines, was the way they found to get reliability. This turned out to be
nice for scalability also.

------
misterbowfinger
When I say this out loud, it sounds like a stupid question, but I'm still
curious...

Outside of the fact that different programming languages run on BEAM and the
JVM, what's the difference between the two?

I believe BEAM ships with supervisors, which is why Elixir can utilize them...
what else is different?

~~~
davydog187
The BEAM is a preemptive VM. That means that no one long running process can
hog resources.

The BEAM was built on a principal of responsiveness since it was originally
designed for telephonics.

Here's an in depth article on preemption in the BEAM that I shared with a
friend this morning [https://hamidreza-s.github.io/erlang/scheduling/real-
time/pr...](https://hamidreza-s.github.io/erlang/scheduling/real-
time/preemptive/migration/2016/02/09/erlang-scheduler-details.html)

~~~
dozzie
> The BEAM is a preemptive VM.

The BEAM Book shared by armitron disagrees:
[https://happi.github.io/theBeamBook/#_scheduling_non_preempt...](https://happi.github.io/theBeamBook/#_scheduling_non_preemptive_reduction_counting)

Though an Erlang programmer never sees this in practice, because compiler
inserts yield points appropriately.

~~~
davydog187
Maybe on the definition of "preemptive", but it should be treated as a
preemptive VM in practice.

~~~
derefr
Nah. You can screw with the BEAM’s reduction-scheduling even in pure Erlang
code: just write a really long function body consisting only of arithmetic
expressions (picture an unrolled string hash function.) Since it contains no
CALL or RET ops, the scheduler will never hit a yield point, even after going
far “into the red” with the current process’s reduction budget.

You just never see this in real Erlang code, because who would code that way?
If you want to be idiomatic, use recursion. If you want to be fast, use a NIF.
Why ever use an unrolled loop body?

But it _can_ happen, and therefore, the BEAM does not _guarantee_ preemption,
even in Erlang. Reduction scheduling isn’t a “kind of” preemptive scheduling.
It’s its own thing.

~~~
davydog187
I mean, like i said, it should be treated as a preemptive VM in practice. In
reality its not, but for most practical cases, its easier to understand it
terms of preemption.

~~~
derefr
As a developer, yes. As an ops person trying to figure out why your deployed
release isn’t making its deadlines, no.

------
jerrycurly
BEAM is phenomenal but one area that's still preventing people from adopting
it is pure raw performance.

People have to write NIFs far to often to obtain the raw perf they need and in
doing so, negates all the tremendous benefits of the BEAM.

I hope more people push BEAMJIT development forward.

[http://www.erlang-factory.com/sfbay2017/lukas-larson.html](http://www.erlang-
factory.com/sfbay2017/lukas-larson.html)

Maybe Facebook would fund this effort given their huge BEAM investment from
WhatsApp use.

~~~
qaq
I think it depends on project domain. perf. is fine for most web apps for
example.

~~~
jerrycurly
>> "perf. is fine for most web apps for example."

That same argument is made for Ruby and look at how many people leave due to
slowness

~~~
jxub
The thing is that MRI Ruby is effectively single-threaded (unless you use
things like concurrent-ruby which aren't that performant anyway) and threads
can be blocked waiting for I/O, in fact they do most of the time, and is in
I/O where Erlang really shines thanks to BEAM's nature of lightweight
preemptive processes.

~~~
jashmatthews
CRuby is multi-threaded with a GIL. CRuby also supports light-weight
cooperative threads called Fibres, which are used to build things like:
[https://github.com/postrank-labs/goliath/](https://github.com/postrank-
labs/goliath/)

JRuby has no GIL, JIT, and is significantly faster than BEAM.

