
Clasp – A Common Lisp with LLVM back end and interoperation with C++ - drmeister
http://drmeister.wordpress.com/2014/09/18/announcing-clasp/
======
eudox
>Programmers can expose C++ classes as well as functions and class methods
with a single line of code that provides the Clasp name and a pointer to the
function/method.

Hellooooo Qt without Smoke bindings.

~~~
pnathan
It would be really shiny to have a cross-platform GUI library in CL without
grief. I've prodded that space repeatedly in the last 6 years and simply have
not come away happy about any of it except LispWork's library.

------
phkahler
Sounded good until I saw this:

A faster Clasp compiler is coming soon – the current Clasp compiler generates
slow native code that is about 100x slower than highly tuned Common Lisp
compilers like Steel Bank Common Lisp.

No mention of the expected speedup, but 100x is a huge performance problem.

~~~
drmeister
The reason is that Clasp doesn't do Lisp language level optimizations like
escape analysis yet. All bindings are stored on the heap and the
stack/registers are underutilized. LLVM is a great library for implementing C
and C++ but more work needs to be done to support Lisp features like closures
and first-class functions. We are working on that now. The goal was first to
"make it correct" and now we will "make it fast". Once we have a faster
compiler (give us a couple of months) I don't see why it couldn't approach the
speed of SBCL (a tall order).

~~~
phkahler
Glad to hear it. I've been wondering when new languages would be developed on
top of LLVM and it's starting to happen (Julia, now this). C++ is getting a
lot of more advanced features, but the syntax is so clunky compared to
something like Python for example. I also like the notion of easy C++ library
use for things like QT.

------
wtbob
This is extremely exciting, particularly if it can be made to work with
emscripten. Common Lisp in the browser, here we come!

~~~
aidenn0
Almost certainly can't. The closest you can get is using clicc or ECL or
something similar to generate C code, and then compile that with emscripten.

This invokes llvm at runtime, and AFAIK emscripten isn't ported to emscripten.

FWIW, I've tried other lisps under emscripten:

Most lisps generate machine code, store them in RAM, and then execute that
RAM. This is not possible under emscripten.

Clisp is a good candidate, since it's byte-code interpreted rather than
generating machine code, but clisp makes so many assumptions about how the
machine works (in particular it strongly wants a C style stack and does manual
stack-pointer manipulation). I actually got fairly far into the bootstrap
process under emscripten, but the minimal lisp interpreter it compiles
generated bizarre errors.

~~~
drmeister
I disagree. Clasp could do this because it compiles Common Lisp to LLVM-IR
bitcode files (using COMPILE-FILE) as well as directly to native code using
LLVM's MCJIT engine (using COMPILE). emscripten
([https://github.com/kripken/emscripten](https://github.com/kripken/emscripten))
says that it compiles LLVM-IR to JavaScript. I haven't used emscripten, but I
believe everything I read on the internet :-) and thus Common Lisp
--[Clasp]--> LLVM-IR bitcode files --[emscripten]--> run within browsers.

~~~
eudox
I'm not an expert in Common Lisp internals, but there are some things that
don't look like they could be entirely put in an executable without bringing
the compiler and runtime along. I'm not just talking about things like a
library that outright uses eval, but the more high-level things like macros
that call compiled functions and reader macros.

Then again, you might be able to compile the compiler to LLVM IR -> emscripten
-> JavaScript, and use that to compile CL code.

~~~
drmeister
Agreed, but it's more of a "tree-shaking problem" than anything else. Clasp is
written in C++ and Common Lisp. Clasp compiles Common Lisp to LLVM-IR and
Clang compiles C++ to LLVM-IR. So theoretically you could compile everything
to LLVM-IR and feed that to emscripten. Granted, it's going to be a _huge_
LLVM-IR file. Then you shake out the functions and globals that aren't needed.
If the compiler isn't needed then it will shake out (or not be compiled in in
the first place). The question for me is "what problem does it solve". I
assume there are problems that I'm not aware of that it would solve, otherwise
why would someone develop emscripten? Common Lisp is a fantastic language that
is really underutilized, it's fun and so expressive. I fell in love with
Common Lisp three years ago so deeply that I wrote a new Common Lisp to solve
my scientific programming challenges while still being able to make use of
powerful C++ libraries. I think everyone should use it everywhere - it's
awesome.

------
mcmancini
Exciting work. CL on the LLVM has been needed.

Is this project something that you're doing to support your own research goals
in computational chemistry? If so, can you share how this project fits into
the bigger picture?

------
yvdriess
Finger crossed then that Azule can get a proper moving GC to work with LLVM
then, until then a CL on LLVM (even Julia) are stuck with kinda bad GC.

I wonder why start from scratch for an LLVM backend, can't SBCL be used to
generate LLVM code?

~~~
ioddly
Is the compacting garbage collector mentioned in the article not a proper
moving GC? I am not familiar with it.

~~~
drmeister
Yes, the Memory Pool System by Ravenbrook
([https://www.ravenbrook.com/project/mps/](https://www.ravenbrook.com/project/mps/))
is a proper, moving garbage collector. It uses precise GC on the heap and
conservative GC on the stack, as does the garbage collector in Steel Bank
Common Lisp on x86 chips. I need it because I need my code to run on 100,000
CPU supercomputers with a controlled memory footprint to develop organic nano
machines (seriously).

~~~
ioddly
To be honest I was just kind of thinking "oh great another language
implementation," before I read what you're actually doing and why you needed
to create clasp. I appreciate the difficulty of writing precisely GC'd C/C++.
It's pretty awesome that you were able to use clang to (I assume this is
mainly what the analyzer does) track roots in C & C++ code.

Best of luck.

~~~
drmeister
Thanks - yes the analyzer tracks roots through about 300 C++ classes. It also
finds global variables and builds C++ code to interface with the MPS library.
I exposed the Clang libraries to search the AST and describe the AST in Common
Lisp and then wrote the static analyzer in Lisp. I shudder at the thought of
doing this all in C++ and I write a lot of complicated stuff like Common Lisp
implementations in C++ :-). Common Lisp is the language of trees and pattern
recognition. Common Lisp is the perfect tool for this job.

------
masukomi
I would point out that MOCL [https://wukix.com/mocl](https://wukix.com/mocl)
pretty much provides this functionality only with full Common Lisp
compatibility. It compiles the CL to C and adds methods to expose your
functions to Obj-C / C code (if you want). Works just fine. Builds with Xcode,
which as I understand, does everything on/via LLVM.

~~~
lispm
mocl is different from many other implementations: it is a batch compiler for
whole programs to C. The generated code does contain a limited evaluator and
no compiler. Some dynamic features are gone...

------
JabavuAdams
> Clasp exposes the Clang AST library and the Clang ASTMatcher library. This
> allows programmers to write tools in Common Lisp that automatically analyze
> and refactor C++ programs. Clasp can be used to automatically clean-up and
> refactor large C++ codebases!

Niiice. I want to program C++, but not in C++.

~~~
drmeister
Yes, I'll post more on this and provide example code in the coming weeks on
the blog. It's really exciting and it's something that Clasp can do right now
even though the Clasp compiler isn't a highly optimizing one yet. Google uses
automated Clang refactoring to clean up the googleplex (100 megaLOC). Clasp
puts those capabilities into everyones hands now.

------
tempodox
Wow, this news almost makes me drool. I will keep my eye on that. I'm on SBCL
right now, but I can't wait to see what Lisp can do with LLVM...

------
fithisux
I hope it turns out a good implementation since there is a lot of C++ code out
there and I do not know C++. But I can comprehend Lisp.

------
tunnuz
Maybe not-so-related, but "clasp" is also the name of a pretty popular Answer
Set Programming (ASP) solver (link [http://www.cs.uni-
potsdam.de/clasp](http://www.cs.uni-potsdam.de/clasp)).

------
edwintorok
The name is already taken unfortunately: [http://www.cs.uni-
potsdam.de/clasp/](http://www.cs.uni-potsdam.de/clasp/)

    
    
      Package: clasp
      Version: 3.1.0-1
      Description-en: conflict-driven nogood learning answer set solver
      clasp is an answer set solver for (extended) normal logic
     programs.

~~~
chc
It was also already taken by a device for fastening things. And my name was
already taken by at least three people before me. Somehow we all get along.

~~~
edwintorok
Package names in distributions have to be unique though, so this one will
probably have to be called clasp-llvm or something like that.

