
RaptorJIT + Studio: Analyzing the evolving C heap of a JIT compiler (2017) - BobbyVsTheDevil
https://github.com/lukego/blog/issues/20
======
lukego
Author here with a little update from experience.

Generally I am quite happy with this approach, but I am not wild about the
cost/benefit of writing tools in Smalltalk. I've spent a lot of energy on
treading water and trying to keep things working while the environment is
being rewritten underneath my feet.

I have also found that Smalltalk works best when data is imported into the
Smalltalk heap, and represented by a dedicated model classes, and also
represented by dedicated view objects (Bloc.) That's a damned lot of classes
to write and the whole image can grind to a halt if I reasonably want to
inspect millions of objects. This makes me miss simpler and more scalable data
structures like R data frames.

Going forward I plan to take a more perl-inspired "there's more than one way
to do it" approach and complement the Smalltalk code with e.g. R, Lua, etc. I
expect to spend more time writing code generators (a la SWIG) rather than
hand-crafting everything in Pharo.

~~~
patrec
I really hope RaptorJIT or one of the other luajit forks gains some traction,
because luajit is such an impressive feat of engineering and so massively
superior to more mainstream dynamic language implementations (ffi, startup
time, execution speed, code size...) that it would be tragic if it just slowly
fades into obsolescence.

I also think your focus on making performance more predictable and reducing
the mental burden on devolopers who want to write _reliably_ fast code is
exactly the right one; even a version of luajit that's half as fast on average
but more predictable could be more useful in practice to most people.

Having said that, to keep some already precarious and fringe technology alive
by adding as core part of the toolchain an app written in an even more fringe
and decidedly more fiddly[*]/unstable language like Pharo, and one which has
basically zero community overlap with luajit users seems... bold.

As does moving forward by adding R/codegen targetting pharo to the mix.

~~~
lukego
That's a fair point.

I'm viewing the tooling as a separate project. RaptorJIT is one application
that it supports and Pharo is one UI.

Codegen is appealing for being able to use the same data for more things e.g.
also generate gdb/rr extensions for debugging the RaptorJIT VM, or generate
Lua code for a simple console-based analysis tool, etc. There are already
other people writing LuaJIT tooling in Python and Javascript and ... so I want
to keep an open mind about target environments for the sake of potential
collaboration.

(You could also think of codegen as a bridging strategy in case the tooling
should be ported to some more conventional environment(s) once it is mature
e.g. CLI, web, emacs, etc.)

EDIT: Should also note that the Studio tooling can be run via Docker in pure
text mode, converting the binary diagnostic data into ascii that's similar to
but richer than the classic LuaJIT '-jdump'. So users don't need to use the
GUI, or setup any of the dependencies, or even learn different ways of
working.

~~~
patrec
> I'm viewing the tooling as a separate project

If the quality (and acceptance) of the tooling is pretty important to the
success of RaptorJIT and the tooling is unlikely to be used for anything else
than RaptorJIT (e.g. pypy or at least another luajit fork), they are not
really _that_ separate.

I don't think codegen is in itself bad, but R -> Pharo so you can work on some
low level lua JIT stuff – you're mightily reducing the set of people who don't
need to learn extra tech to be able to contribute (from an already small pool
of people who have the relevant skills and inclination to improve a JIT
compiler).

~~~
lukego
> the tooling is unlikely to be used for anything else than RaptorJIT

Here we are talking at cross-purposes. I view the tooling as an extensible
framework like Emacs or Eclipse. I am talking about my intentions for
extending the diagnostic tooling to support other projects beyond RaptorJIT.

Over time I would anticipate reducing the number of moving parts in the
RaptorJIT tooling.

The value of a "there's more than one way to do it" approach is to make it
easier to support new projects besides RaptorJIT using whatever tools that
project's community are comfortable with (which, as you say, probably won't be
Pharo but might be R/python/perl/emacs/etc.)

------
vkaku
Neato.

