
Compiling a Lisp to x86-64: primitive functions - tekknolagi
https://bernsteinbear.com/blog/compiling-a-lisp-4/
======
lionsdan
There's also Bones, an R5RS Scheme that compiles to x86_64 assembly (from the
Chicken Scheme creator).

[http://www.call-with-current-continuation.org/bones/](http://www.call-with-
current-continuation.org/bones/)

[https://bitbucket.org/bunny351/bones](https://bitbucket.org/bunny351/bones)

~~~
pankajdoharey
Adding to that Eli Bendersky has a very good 4 part tutorial on writing a JIT
from scratch [https://eli.thegreenplace.net/2017/adventures-in-jit-
compila...](https://eli.thegreenplace.net/2017/adventures-in-jit-compilation-
part-1-an-interpreter/) So you can try converting other interpreters to native
compilers.

------
tekknolagi
This is part of a series I'm writing about compiling a small Lisp directly to
x86-64 machine code. Let me know what you think!

~~~
smabie
This is great. Most Lisp implementation involve building a trivial
interpreter, but rarely does anyone show how to build a real native compiler.

Great work, I especially like the use of C instead of a higher level language.

~~~
tekknolagi
I think that's because it's more finicky and a longer implementation. Of
course, if you build a small interpreter that you can then use to write a Lisp
compiler _in_ Lisp... you've got it made!

~~~
jamiek88
I love how we can bootstrap.

I’m getting interested in bare metal, I.e. no OS, booting direct to app,
development.

These compilers are a goldmine of info.

------
p4bl0
For those who like me were not sure that the `calloc` call would necessarily
allow to tag the returned pointer, I found the explanation in a previous post
of this article series:

> _It’s important to note that we can only accomplish this tagging scheme
> because on modern computer systems the lower 3 bits of heap-allocated
> pointers are 0 because allocations are word-aligned — meaning that all
> pointers are numbers that are multiples of 8. This lets us a) differentiate
> real pointers from fake pointers and b) stuff some additional data there in
> the real pointers._

It's what I suspected, but I didn't know that it was actually safe to assume
that.

------
giancarlostoro
I always wondered why we rarely see more efforts like an LLVM based Lisp or
Scheme language, especially when some projects like Racket are looking to
improve performance by switching their runtimes (and I assume other benefits
that come from consolidating underlying programming efforts across multiple
projects).

~~~
afranchuk
I've been working on a lisp that's strongly typed and basically only has
functions that are the llvm C API, including JIT. With that you can build
macros, type systems, etc. Basically much of the code you write as a user can
be what's typically implemented (in predetermined ways) in the compiler in
other languages.

~~~
giancarlostoro
This sounds interesting is any of this public or publicly blogged about?

~~~
afranchuk
I'd love to make it public when it is a bit more thorough. Right now it's just
local. I really wanted to get to the point that it's self-hosted before going
public, but naturally that's a little involved since it does involve building
everything up from assembly (llvm IR). But it's fairly straightforward, albeit
time consuming, to build simple abstractions on top of each other to implement
features typically present in high-level languages.

------
timonoko
#METOO from 1982:

    
    
        C:\>nokolisp                              
                                              
        1> (comp-debug t)                         
        t                                         
                                              
        2> (ncompile ’(plus a 1))                
                                              
        $O9CB:$7149: MOV AX, [$01BA]              
        $O9CB:$714C: CALL $0F1D ; CALL NUMVAL     
        $O9CB:$714F: MOV BX,$01                   
        $O9CB:$7152: ADD AX,BX                    
        $O9CB:$7154: CALL $05C0 ; CALL MAKNUM     
        $O9CB:$7157: JMP $1DA7                    
                                              
        (subru: eval=$7149, compile=$3BB0)

~~~
sitkack
How does the stack work?

I assume $1DA7 is the outer interpreter loop?

What is $3BB0 for?

~~~
timonoko
The stack is regular 16 bit CPU-stack. Unlike most or all others, i was using
the CPU stack for Lisp-nodes. The garbage collector made many errors when
marking data from CPU stack. But it does not matter, with 16 bits you can have
65000 nodes, enough for all eternity.

JMP $1DA7 is basically return-from-closure.

(subru: eval=$7149, compile=$3BB0) means that compiled functions are treated
differently, when called from intrepreter and compiler. It was direct machine-
code CALL in compiled code.

------
mtreis86
I am really enjoying this series, thanks. I've read PAIP by Norvig which
includes a compiler, but this is a whole different perspective, being straight
to machine code.

------
pfdietz
Generating code from Lisp is a well-solved problem.

I'd like to see the capability of recompiling functions while they have live
stack frames, and this just working (within defined limits).

~~~
macintux
This could have started an interesting discussion about the second sentence if
you hadn’t led off with the first.

~~~
pfdietz
I don't think we'll see radical improvements in Common Lisp compilers (the
Lisp dialect I am most familiar with) for conventional AoT compilation. Sure,
there are incremental improvements that can be made.

Perhaps you could start an interesting discussion on how you see that first
sentence being wrong?

~~~
macintux
The first sentence wasn't wrong, but it came across as snarky. Very little on
HN is original research; that doesn't mean anyone who creates something they
find interesting should be slapped down.

