Hacker News new | past | comments | ask | show | jobs | submit login
OpenLDK: A Java JIT compiler and runtime in Common Lisp (github.com/atgreen)
211 points by varjag 5 days ago | hide | past | favorite | 61 comments





Hello -- I wrote this! It's still very much a work in progress.

Very cool! I'm curious whether you're using it (or planning to use it) yourself for anything serious or if it's just to see if it could be done. Are you planning to support classes beyond Java 8 at some point?

I'll definitely use it when it becomes usable. And, yes, it needs to support newer versions of Java. Crawl.. walk.. run. But you are right.. part of my motivation was just to see if it was possible. Many years ago, I worked on gcj, which was similar in concept, except that it mapped Java classes to C++ classes (as implemented by g++). I've been on a Common Lisp kick for the past few years, so it seemed like a fun challenge. The fact that the OpenJDK class libraries are free software now makes this more practical.

I definitely get the Common Lisp kick. I dabbled with it before, but then got sidetracked by other languages and back then the REPL didn't click for me. After I understood the power of the live image concept though (and after switching to Emacs), I just want to do every personal project in CL only.

Are you planning to do a Futamura projection?

No. It's a simple transpiler for now.

@atgreen - Would love to hear what key lessons / takeaways you've learned while working on this project?

A pragmatist will point to this and say, why not Clojure? Can you share more about what keeps you using SBCL instead of Clojure?

> why not Clojure?

Common Lisp implementations like SBCL have static typing, performance, instant (and effortless) compilation to native machine code, and so on. No need to deal with Leiningen or copy-pasting a deps.edn / build.clj template across projects. If I want to experiment with SIMD intrinsics with the comfort of a REPL, I can't do that with Clojure, but I can do that with SBCL and a built-in library (SB-SIMD).

Note: I have used Clojure professionally and still use it for some personal projects, but I try to be aware of the trade-offs when deciding to use Clojure / Common Lisp / any other programming language.


do you have any reference libraries/apps/projects you could refer me to? curious to see what a healthy and idiomatic project looks like.

Or for that matter, Armed Bear Common Lisp (real CL implemented in the JVM rather than Clojure which is more like Scheme mixed with its own ideas). Still, it's a neat trick, running Java classes on a Lisp that isn't in the JVM itself.

I'm sure they would, but I have the freedom to not be practical! The journey is the destination.

Common Lisp is fun to code in, and the tooling is top-tier.


Could you share some of what is great about the cl/sbcl tooling?

If you are writing services for a platform like kubernetes, the inner-loop development cycle is unmatched. You just connect your editor to the lisp image running in k8s, and with a couple of keystrokes, your code changes are shipped to the target container, replacing what's running live -- as native machine code. You can debug in a repl, handle conditions (exceptions) interactively with restarts, etc etc.

Yep makes sense. I worked with clojure for many years so it’s familiar to me. Iterating via the repl is a great way to develop software. I was wondering if sbcl has good tooling for code navigation, completion, debugging.

I am a hobbyist Scheme programmer, and I can chime in and say that Clojure stack traces are among the worst I’ve ever seen in my life. Chez errors aren’t the best in the world, but anything beats those JVM ones, imo.

I thought Clojure already had JVM interop?

Clojure runs on the JVM; this is the opposite, Java in a CL compiler.

Besides, there's more to Lisp than Clojure.


Well, for one thing Clojure is not Lisp. It’s a neat language (and is informed by experiences with Lisp), but it’s a different thing.

What's the biggest thing your mind that makes Clojure not Lisp?

eadmund probably means that Closure is not Common Lisp. Common Lispers often refer to Common Lisp as just "Lisp". This makes sense in Common Lisp circles because it's obvious which Lisp they're talking about.

However, some Common Lispers will refuse to code switch[1] when around non-Common Lispers, with the implicit assertion that Common Lisp is the only true Lisp. This is, of course, just gatekeeping snobbery.

When normal people say "Lisp" they're referring to a group of languages which includes Scheme, Clojure, Racket, Janet, Emacs Lisp and of course, Common Lisp, and probably a few others.

To be clear, not all Common Lisp folks are like this. The vast majority are very nice people. But there are enough bad apples that you'll inevitably run into this "correction" if you discuss Lisp dialects publicly for long enough.

[1] https://en.wikipedia.org/wiki/Code-switching


I doubt you can find a single Clojure-is-not-Lisp person who denies that McCarthy's LISP 1 and 1.5, MacLisp, BBN Lisp, Interlisp, Lisp Machine Lisp, ZetaLisp, Portable Standard Lisp, Emacs Lisp, XLisp, VAX Lisp, ISLisp, EuLisp and even AutoLISP ... are Lisp.

I made a Lisp. It's not Common Lisp, yet I've never had anyone tell me it's not Lisp.

I believe Clojure is better than AutoLISP.

I wouldn't use McCarthy's LISP 1.5 for anything; it mainly of historic interest. But it is undeniably Lisp.

"Lisp or not" is not the same thing as "good or not". It's more like "different or not". Difference is what makes certain words apply to some things and not apply to some other things.

This is not some gender issue of inclusivity, like what is a woman. It has nothing to do with being nice or not. It's a technical matter of using the correct term for the things to which it applies.


> I doubt you can find a single Clojure-is-not-Lisp person who denies that McCarthy's LISP 1 and 1.5, MacLisp, BBN Lisp, Interlisp, Lisp Machine Lisp, ZetaLisp, Portable Standard Lisp, Emacs Lisp, XLisp, VAX Lisp, ISLisp, EuLisp and even AutoLISP ... are Lisp.

Please tell me you aren't making the argument that things with "Lisp" in the name are Lisps.

If only Steele and Sussman had thought to call it SchemeLisp instead of just Scheme, we could have avoided this whole semantic debate.

> I made a Lisp. It's not Common Lisp, yet I've never had anyone tell me it's not Lisp.

Me too! I've never had anyone tell me anything about the Lisp I made, actually.

> This is not some gender issue of inclusivity, like what is a woman. It has nothing to do with being nice or not.

Respectfully, I disagree. Being pedantic isn't nice, and this "Lisp refers to Common Lisp" idea was explicitly about politically excluding Scheme from the community all the way back when Naggum started it[1].

> It's a technical matter of using the correct term for the things to which it applies.

The problem with what you're saying is that words apply to concepts based on usage, not based on what a small group of pedants think they should apply to.

De facto people apply the term "Lisp" to both Scheme and Closure, therefore the term applies. That's how the English language works, words that are widely applied to a thing are taken to mean that thing. And sometimes the meanings of words change over time because common usage changes. Like it or not, common usage moved beyond "Lisp" meaning... whatever you want it to mean. People refer to Clojure as a Lisp now.

[1] https://web.archive.org/web/20250000000000*/https://groups.g...


Clojure people don't know anything else, and say things like [I'm paraphrasing typical rhetoric]: "Lisp provides a functional paradigm based on operations on immutable sequences represented by persistant data structures".

Their use of Lisp is not big enough to count as English mainstream. Neither is any other use of Lisp in computing.

English mainstream "Lisp" is the speech artifact.

When the following groups use the word "Lisp", they all mean something else: (1) hackers working in some classic Lisp; (2) Clojurians; (3) speech therapists.

The Clojurian "Lisp" does not include anywhere near everything from the classic "Lisp", and vice versa.

The Clojurian "Lisp" is mainly just a synonym for Clojure, and in a broader sense refers to Clojure-like dialects like ClojureScript, Babashka and such.

Clojurians use the word "Lisp" because the Clojure website and documentation tells them to. If the Clojure website and documentation spelled it out that Clojure takes some ideas from Lisp, but isn't Lisp, then they wouldn't do that.

The author of Clojure despises classic Lisp and Lisp hackers, so he perpetrated that on purpose, and gleefully enjoyed it when some Lisp programmers become indignant about it.

Problem is, it does cause confusion. It's not as simple as these people have their "Lisp", these others have theirs, because the usage is in a very similar area. We see situations like people not wanting to try a classic Lisp because they think it's a functional language for manipulating immutable data structures due to either their exposure to Clojure or to Clojure-related advocacy. That could work the other way; people who don't like classic Lisps (e.g. passing familiarity from school years) may be reluctant to try Clojure thinking it's the same, since it calls itself Lisp!

In the Perl community, calling a certain different, new language Perl 6 created confusion. So the creators did a kind thing and invented a different name: Raku. This is because they didn't despise the Perl user base; they liked Perl users and wanted to be helpful.


> Clojure people don't know anything else, and say things like [I'm paraphrasing typical rhetoric]: "Lisp provides a functional paradigm based on operations on immutable sequences represented by persistant data structures".

Wow, you've gotten really specific with your straw man argument here.

I'm sure you can find some wacko out there who specifically is saying that's what a Lisp is, but that's certainly not what I'm saying a Lisp is. The Clojure front page says:

"Clojure is a dialect of Lisp, and shares with Lisp the code-as-data philosophy and a powerful macro system."

Maybe try responding to a more mainstream opinion like that instead of your straw man.

> Clojurians use the word "Lisp" because the Clojure website and documentation tells them to. If the Clojure website and documentation spelled it out that Clojure takes some ideas from Lisp, but isn't Lisp, then they wouldn't do that.

Wow, life must be really hard if you're triggered this easily.

Alternative hypothesis: Clojure docs refer to Clojure as a Lisp because it is obviously a Lisp, not because they're picking on poor persecuted Lispers. You're not a martyr. Nobody in the Clojure community cares about your feelings enough to try to hurt them. In fact, for the most part, the Clojure community doesn't know you exist.

If you can find a spot where Clojure website or docs orders Clojurians to refer to Clojure as a Lisp, I'll be pretty surprised. My guess is, all it does is refer to Clojure as a Lisp. It's not about you, so maybe try and cope by not taking it personally.

> The author of Clojure despises classic Lisp and Lisp hackers, so he perpetrated that on purpose, and gleefully enjoyed it when some Lisp programmers become indignant about it.

I very much doubt Rich Hickey "despises classic Lisp and Lisp hackers", given he wrote a language which you have to admit is at least heavily based on Lisp.

I won't speak for whether Rich Hickey feels glee because I don't make shit up like you are, but I feel glee when pedants get triggered, and I think that is pretty justified. :)

To be clear, I don't hate Common Lisp programmers--I like Common Lisp and most of the people who write it.

Rich Hickey did say that a lot of Lisp communities are really toxic and he wanted to distance the Clojure community from that and... yeah, you're being a great example of his point.

> We see situations like people not wanting to try a classic Lisp because they think it's a functional language for manipulating immutable data structures due to either their exposure to Clojure or to Clojure-related advocacy.

Maybe people don't want to try a classic Lisp because they start exploring the possibility and all the communities are polluted with gatekeeping pedants.

And let's be explicit: is Scheme a Lisp? I know you're just parrotting Naggum's arguments against Scheme, so let's be clear: this isn't about Clojure, this is part of a gatekeeping culture that started before Clojure existed.


Clojure is a Lisp, precisely, a language from the List family of languages.

Other known examples of Lisps are Emacs Lisp, Common Lisp and Scheme.


Reminds me of Greenspun's tenth rule of programming (also counts for Java, but this most probably predates it):

Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.

https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule


Ritchie's Revenge: And this half is still faster than Common Lisp.

Only when adding UB exploits done by optimizing compiler backends.

I am old enough to remeber the days any junior Assembly programmer could easily outperform any machine code generated by the C compilers of the day.


SBCL (the most popular Common Lisp implementation) is pretty darn fast, being comparable to Java, Go or OCaml.

https://benchmarksgame-team.pages.debian.net/benchmarksgame/


But still slow compared to a non-GC'd language. Walking over all the references of your program while its stopped is slow.

> Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.

...including Common Lisp.


Haha... It would be strange to run this inside https://abcl.org/...

A Java runtime... running on a Java runtime... Hmm, maybe not. But could be possible.


Have you seen:

https://winworldpc.com/product/ibm-visualage-for-java/10

QUOTE: VisualAge for Java is based on an extended Smalltalk virtual machine which executes both Smalltalk and Java byte codes. Java natives were actually implemented in Smalltalk. Later versions were implemented on regular Java.

Obviously not the same....


Yes, I'm familiar with that! It was created by the OTI team in Canada, which was acquired by IBM. I think some of them are still around, working on IBM's OpenJ9 java implementation. They reach out to me now and then on libffi-related issues.

Similar: https://github.com/ikvmnet/ikvm

> A Java Virtual Machine and Bytecode-to-IL Converter for .NET


Finally I can run Clojure in Common Lisp :D

Joke aside, that is an amazing project.


Does this have something to do with recursion? I guess what I’m thinking is bout is inception, I guess there already exist some lib called inception, wonder what it would do, a compiler? :D Java to cl compiler I guess :D

This will achieve perfect cult status if you use a Common Lisp that was implemented in JavaScript.

Nice work. I will have a read of the code over the weekend.

this is like the opposite of clojure right? lol amazing

More like the opposite of ABCL (Armed Bear CL.)

Speaking of Lisp, I'm developing a programming language that is similar to other Lisps, but I noticed that we can sprinkle a lot of macros in the core library to reduce the number of parentheses that we use in the language.

example: we could have a `case` that works as follows and adheres to Scheme/Lisp style (using parentheses to clearly specify blocks):

    (case name
        (is_string? (print name))
        (#t         (print "error - name must be a string"))
    )
OR we could also have a "convention" and treat test-conseq pairs implicitly, and save a few parentheses:

    (case name
        is_string?    (print name)
        #t            (print "error ...")
    )
what do you think about this? obviously we can implement this as a macro, but I'm wondering why this style hasn't caught on in the Lisp community.

Notice that I'm not saying we should use indentation—that part is just cosmetics. in the code block above, we simply parse case as an expression with a scrutinee followed by an even number of expressions.

Alternatively, one might use a "do" notation to avoid using (do/begin/prog ...) blocks and use a couple more parentheses:

    (for my_list i do
        (logic)
        (more logic)
        (yet more logic)
    )

again, we simply look for a "do" keyword (can even say it should be ":do") and run every expression after it sequentially.

A bunch of lisps do that. Clojure for instance or arc use conventions like that to reduce the number of parentheses.

See e.g. cond in clojure: https://clojuredocs.org/clojure.core/cond


> Clojure for instance or arc use conventions like that to reduce the number of parentheses

Clojure also reduces parentheses by introducing more characters for various data-structures (good and bad, many would say), like [] for vectors, which are for example used for arguments when defining functions

    (def add-five [a]
      (+ a 5))

In an old programming language project of mine, I wanted to use something similar to S-expressions for most of the grammar because they are very universal and flexible, but I didn’t like having too many parentheses, so I extended the syntax by allowing semicolons as an alternative.

So, in pseudocode:

  (print (+ 1 2))
  (exit 1)
becomes:

  print (1 + 2); // "+" is the 2nd place because it's a method call on "1"
  exit 1;
Both variants are valid; semicolons are just syntactic sugar to reduce the number of parentheses. In the AST, they represent the exact same expressions.

For function bodies, I also introduced an extension to make lambdas stand out more from the rest of the code by using curly braces:

  (0 to 10) loop ^(i) {
    print i; // Internally transformed to (print i)
  }

  // clarification:
  // 1. The "to" method of the 0 object creates a Range object, which has a "loop" method that accepts a lambda.
  // 2. The ^ symbol is just a shorthand for "lambda".
As you can see, this wasn't a Lisp dialect (and I'm no expert at Lisps), but I liked the idea of having a universal expression structure in the AST, however I eventually ended up extending it with two additional modifications that made it more readable (at least for me)

Your for example is basically Common Lisp's `loop`

    (loop for i in my-list
      do (logic)
         (more logic)
         (yet more logic))

if you think that there are too many parens in lisp then you should fix your editor.

the parens are just a textual representation of a tree. once you have a proper editor, then the eliminated parens actually become an obstacle.


Imagine if we had a Lisp with proper types, we could do something like...

  (typecase name
    (String (print name))
    (otherwise (print "error: name must be a string!")))
And if that Lisp had some kind of Object System, we may even go further and have stuff like...

  (defmethod namecall ((name String))
    (print name))

  (defmethod namecall ((name Number))
    (print "I am not a number, I am a free man!"))

In case anyone is missing the snark here, all of tmtvl's code above is in fact perfectly valid Common Lisp.

clojure's multi-arity addresses this to some extent but pattern matching like we know it in Haskell is very weak in Lisps...

In case the sarcasm was missed, the latter code was actual common lisp code to pattern match a generic function, and the former is nearly-there when it comes to common lisp code

Why "nearly"? Given how symbols are uppercased upon read in CL, 'string and 'String are actually the same symbol. I just checked: `(typecase "" (String 2) (otherwise 3))` returns 2, as expected. I'm less sure about the `defmethod`s, but I think they'll also work.

Plus, the capitalized type name reminded me of Coalton[1], which is an internal DSL for CL that implements ML-style type system. There, the dispatch on a type (conventionally named by a capitalized symbol) happens statically.

[1] https://github.com/coalton-lang/coalton


Because on the phone with one hand I couldn't check if I didn't forget some detail in typecase which I use way less often than defmethod :)

Fair! :) The HyperSpec is full of easily forgettable details that almost never matter, until they do :D There are even undefined and implementation-defined things there, which can trigger C++ PTSD on a bad day...

Wild!

Just install an editor plugin (parinfer I like now) to fix that easily. It's a pleasure. It's not hard to make your own really; it's, like you say, only cosmetic. You can format your code for reading in the same way with a trivial plugin; it is a really nice thing that you can very rapidly make something nicer tailored for you while still benefitting from the performance / debugging / etc in sbcl. For my dayjob, I work a lot with Typescript parsing and rewriting and, while the ts (and ts-morph and so on) are pretty nice, it's really a quite horrible experience vs what we have in lisp/scheme.

Actually, this reminds me very strongly of Rebol [0] and related languages (e.g. Red [1]). They work as you describe at the end: whenever a function is encountered, the interpreter inspects the following arguments, without any need for parentheses. Since these languages are homoiconic like Lisp, it’s possible to use functions as if they were macros (achieving it at runtime using a rather strange ‘binding’ system).

[0] https://en.wikipedia.org/wiki/Rebol

[1] https://www.red-lang.org/


It's just a bunch of macros to this, no ?

There are a lot more hairy/pressing issues given how old the CL spec is (the type-system esp. comes to mind).


You can do a lot with creating your dsl/tooling and something like Coalton really. More than enough imho.

pff. Now run a hardware Lisp machine under it..

https://en.wikipedia.org/wiki/Lisp_machine




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: