
Lightening run-time code generation - ingve
https://wingolog.org/archives/2019/05/24/lightening-run-time-code-generation
======
chrisseaton
Guile is a really interesting story in that they've spent ages doing some
fantastic optimisations, of the kind you usually don't do when your final
execution engine is just an interpreter. I used to read Andy's blog posts
about compilers unaware that the pipeline didn't actually end in machine code.
Hopefully that makes the template JIT they have here enough to get good
performance as the input should already be in really good shape.

I wonder why Andy didn't chose to write the JIT in Guile itself, with a
binding to Lightening, rather than in C. Guile is designed to be embedded and
have bindings written for it.

~~~
bjoli
What I love about the guile optimizer (and some lisp optimizers in general) is
that you usually have a first step of source to source optimization. That
means I can write some code and see what gets inined, hoisted, unrolled and
removed without having to read the bytecode assembly.

That is especially nice when writing complex macros. I remember writing a loop
utility and realized I couldn't run the optimizer on code where everything was
known at compile time, since guile usually optimized the loop away. (for/sum
((i (in-range 5)) i) expanded into 20 lines of code, but optimized to the
integer 15 :)

~~~
chrisseaton
The tricky think about a source-to-source transformation, is where do you
store the meta-data to map the new source back to the old source, for the
purposes of error reporting, debugging, and deoptimisation? You can't just map
from one list to another as your optimisation function.

~~~
bjoli
I think it is the same system as used when doing macro expansion. Macros use
syntax objects with embedded metadata.

Both chez and guile usually produces proper error sites, so it is done in some
fashion.

------
ncmncm
It makes me obscurely happy to see Guile evolving to remain useful, despite my
misgivings about obligate-GC languages in general.

~~~
stcredzero
_despite my misgivings about obligate-GC languages in general_

In terms of performance, you have a point. However, there are many ways to set
the performance dial, depending on your particular cost/benefit situation.

------
neilv
Am building Guile 2.9.2 right now, to try it out.

When Racket (nee PLT Scheme) added JIT on x86, via GNU Lightning, there was a
very noticeable practical performance bump for some of my work (dynamic Web
serving).

~~~
bjoli
For 2.9.1 I saw a performance improvement of between 1.5 and 4x across all
workloads. It is still not as fast as racket, but iirc Andy said there are
some low hanging fruit left.

The peg parser and my implementatio of clojure's transducers get the biggest
performance improvements. My simple CSV parser got over 3x faster without any
changes on my part.

~~~
fatcow
Hi! I'm struggling to lean peg in guile - care to share csv-parsing code
please?

~~~
bjoli
Sure. Ill put it online whenever I have access to my computer next. Are you in
#guile on freenode? Ivan ping you there when it's up.

It is slow as molasses though. A 5mb CSV file takes roughly 12s on 2.2. A
properly implemented streaming parser should be several orders of magnitude
faster. I want to see, just for shits and giggles if it is possible to beat
pythons CSV parser with a pure guile module. I think it is. With guile 2.9 it
should be a walk in the park.

~~~
fatcow
Sorry for delay - will raise hand on #guile when online next

------
fithisux
"It's written in C and not C++" hope to stay this way. I really like guile. My
problem is that I am on windows.

------
peetle
Lightening /= lightning

~~~
NikkiA
The article explains that (Guile's) lightening is a fork of (GNU) lightning,
so in this case, it kinda is, but isn't. Anyway, it's not a typo in the
headline as might be surmissed by your comment.

~~~
ncmncm
Lightning does fork.

