Hacker News new | past | comments | ask | show | jobs | submit login
Valtan: Common Lisp to JavaScript Compiler (github.com/cxxxr)
126 points by simonpure on April 3, 2020 | hide | past | favorite | 87 comments

Also potentially coming down the line is JACL (Javascript-Assisted Common Lisp), a web-app focused dialect of Common Lisp.


I'd hold off on getting too excited about this. Until it:

- Is fully featured and has a conditions and restarts system that works on JS, and has CLOS, multimethods, etc

- Has great interop with the existing JS ecosystem. No painful conversions to and from CL, works/is documented well with existing JS tooling like parcel, webpack, mocha, etc

- Is close to being fully featured and not just implementing <50% (usually <30%) of the features of the language - Is production ready -- no weird compiler bugs, I can use this with real business

- Has a powerful linting tool -- A linting tool that can blacklist/whitelist a subset of common lisp macros and expressions that can be used in a project. You would need something like this for a real company to prevent the team from shooting themselves in the foot. Could probably be done in CL, sblint comes to mind but it's not well documented (like most lisp libraries) so I'm not sure.

- Culture changes: specifically the community needs to document things much better and be more pro-social instead of living on an island. Backend workflow needs good docs. Ex: I think fukamachi's web frameworks supersede weblocks afaict? not sure, but he has like half a dozen or so web frameworks and it's not obvious which one to choose. Ningle? utopian? caveman? clack + woo? documentation in most CL libraries are really lacking and you need to dig into the source

- Run CL programs without opening a debugger. Modern day workflows have CI pipelines and automation out the wazoo. CL expects you to run the debugger manually. There's a way to run asdf projects like a script but it's not well-documented

- The debugger needs to be more user friendly. Don't get me wrong, the debugger is one of the coolest features of CL, but asking a plain old JS dev to try to understand those low level stack traces is a form of torture. It's roots come from lower level programming, but newer devs want to use it for web app dev (like this to-js compiler posted here). A nicer, easier to understand layer like ReasonML's BetterErrors would be needed or an update to CL itself

- A real package manager. Not quicklisp -- that's a curated package list where you need to talk with a single person who approves it to get your package published

I see a lot of Lisp stuff on HN : do a lot of people use it ?

You're using it right now :) https://en.wikipedia.org/wiki/Arc_(programming_language)

Lisp is probably more popular among those that read HN than the general developer public. It also depends on what you mean by "Lisp" (i.e Common Lisp or "the Lisp-derived languages")

We use it for the core of our AGI and our in-memory graph db in full production - very very fast. The web libraries for lisp are very good as well with Fukamachi libraries, such as [1] caveman and [2] woo (fastest web server), [3] dexador, [4] websockets, also [5] parallel processing, [6] rtg-math incredibly fast math library including quaternion support, etc.

[1] https://github.com/fukamachi/caveman [2] https://github.com/fukamachi/woo [3] https://github.com/fukamachi/dexador [4] https://github.com/fukamachi/websocket-driver [5] https://github.com/lmj/lparallel [6] https://github.com/cbaggers/rtg-math

Thanks for the links. I had not noticed dexador before. I have it on my schedule this morning to re-work my network access libraries for an Azure web service, and also access to Wikidata and DBPedia. I have been using old code of mine that needs reworking. I will definitely try dexador.

I highly recommend it, dexador is much much faster than drakma

Thanks, I experimented with dexador yesterday, it looks good. I ended up adding a caching layer for all web service calls and SPARQL requests in my app yesterday - this makes dev and testing much faster, but won’t help much in production.

Sounds interesting, especially the SPARQL. What does your app do?

Agreed, but there are a couple of edge cases that drakma handles taht dexador doesn't.

Last time I checked, woo doesn't support TLS. Has that changed?

We took the simple solution of running haproxy to handle ssl, and then using a woo server behind that. This is that setup: https://graphmetrix.com

Given present day approach of using reverse proxies for TLS termination, that's not much of a limitation.

That's the solution I use with mod_lisp running behind Apache too, but I'd like to get away from Apache. I'll try haproxy.

HAproxy is really, really good.

Personally I also use nginx as reverse proxy a lot, but that's more because there's a ready to use Ingress Controller for kubernetes that uses it internally.

It’s not super common anymore but it is used.

Some examples:

• Clojure which runs on the JVM enjoys considerable popularity.

• Common Lisp seems to be popular amongst Quantum Computing researchers.

• Racket enjoys popularity by way of the Pollen Typesetting Engine.

• Emacs Lisp is used for all Emacs scripting.

• AutoLisp is used to script code for AutoCAD

• HackerNews is written in ArcLisp.

• Square Enix employs a proprietary dialect as a scripting language.

Many GNU programs come with a builtin Scheme interpreter (gdb, for example).

You mean GNU Guile?

Yep [0]. However GIMP does not use Guile, it uses TinyScheme for it's Script-Fu [1].

[0] https://www.sourceware.org/gdb/onlinedocs/gdb.html#Guile

[1] https://en.wikipedia.org/wiki/TinyScheme

It’s not popular among quantum computing groups. We largely use python. It’s popular in a group inside rigetti.

The first seems like a nonserious thing and D-Wave is a crank company.

Aye, no TRUE quantum computing company (except for Rigetti) wid dae that!

I guess I assumed Stylewarning was representative, although I'm not surprised at the use of python, I probably would've expected that had I not seen Stylewarning's video.

To understand why LISP matters (in theory anyway) is to understand the concept of reification [1]. What you choose to reify (make explicit) in a particular language is a design decision. It's directly related to point 0 in Wadler's law [2] - semantics.

The design of LISP makes eval() and LISP's very syntax as first-class citizens which adheres to the code is data/data is code pragma. It makes LISP homoiconic [3]. It's an interesting and incredibly powerful property. Often too powerful for inexperienced programmers.

[1] https://en.wikipedia.org/wiki/Reification_(computer_science)

[2] https://wiki.haskell.org/Wadler's_Law

[3] https://en.wikipedia.org/wiki/Homoiconicity

Not really, but Paul Graham likes it, and narrowly speaking, he's not wrong about its unique power and flexibility. I think those traits meant a lot more in the late 90s and early 00s, when powerful and flexible programming environments and engineers skilled in their use were much thinner on the ground than today, but I like it too, especially in the form of Emacs.

I think it's more accurate to say a lot of people like it. Especially for personal projects. Double-especially for compiler/interpreter personal projects (one of its key features is being extremely simple to parse).

The R engine began life as a very simple Lisp interpreter. There is also an R package which gives the Lisp-style representation of an R expression. You can even use it as a R to Lisp Compiler.



Thanks. I wasn't aware of showTree.

Most R users don't realize everything in R can be written as a function, due to its Scheme roots. Example:

if (3 > 2) { print("hello") }

can be rewritten with backticks on the if:

`if`(3 > 2, print("hello"))

If the first argument is false, you need a third argument

`if`(3 < 2, print("hello"), print("go away"))

Essentially, the backticks replace some parens you'd use in Scheme, and then you can add some alternative syntactic sugar to write in Algol style.


If you look in the source for R (in C), you will note that all of the data structure names end with Sexp.

I use it every day in the form of Emacs, and I'm slowly learning it because I find it a fascinating language. (I'm a sysadmin/devops by trade/experience, moving into team lead territory, so I'm not a coder by any stretch of the imagination)

Lisp ends up in nearly every project I work on. For instance, I'm using a custom Lisp for defining CPU behavior right now: https://github.com/daeken/libmoonage/blob/master/Generator/a...

Clojure (not Common Lisp, but a lisp) is more prominent than many people think. I use it, and it's a primary language in a R&D area at the fortune 50 company where I work.

Common Lisp keeps me sane, most of my paid work is in more primitive and constrained languages.

It's great for prototyping and trying out new ideas.

Common Lisp also has a good story for application bundling and deployment. It has almost always been my prototyping language (1) but is great for other purposes.

(1) started in around 1986 when I had to write a neural network product in C++ (SAIC ANSim). I did all my prototyping in Coral Common Lisp, and converted to C++ once I had Art2, Hopfield, Boltzmann, backdrop, recurrent networks, etc. figured out.

Not really, but you should learn it anyways. I bought a bunch of lisp books and have been working through them. It is not really objective but I already feel like I'm a better programmer after learning lisp. It has also been a great history lesson in programming languages. I see pieces of lisp in nearly all other languages.

ITA Software, which became Google Flights, was written in Common Lisp. I had heard stories about Google trying to reimplement it in another language, but don't know if that's true or whether it succeeded. Can anyone comment?

Oh yeah! I'm not using Common Lisp but ClojureScript to build the entire frontend for https://bravostudio.app

Basically using Clojure(Script) because it's a way faster way of developing software and frontends in particular. What you see inside of the platform (the frontend), is all built by one person since the start of the project.

do you mean the landing page or also the bravo studio featured on the video? how long did it take? because to create such editors is like a good deal of work and planning.

No, landing page is made in webflow by not-the-frontend-team, talking about the Bravo Studio featured in the video (accessible here: https://projects.bravostudio.app). We're a young startup so been doing less planning and more doing, maybe to a fault, as what we lacked in planning would have to be covered up by spending more time working on it.

But yeah, Bravo Studios frontend had it's initial commit May 27th so coming up to be about a year soon. Bunch of work mixed in on a Figma Plugin that didn't go anywhere as well, so I haven't been 100% focused on the frontend all the time but think we've made good progress so far.

I use Common Lisp and Clojure(Script) for personal projects. Lisp can be a tough sell in a corporate setting, however, some large companies do use Lisp. To my knowledge both Walmart Labs and Cigna use ClojureScript for at least some of their products.

You can find a partial list of companies that use some clojure on clojure.org: https://clojure.org/community/companies

It's not complete or up to date.

Hacker News is built with it

I do, in ClojureScript. Here are two of my main projects:

* https://github.com/Cirru/calcit-editor * https://github.com/Respo/respo

There is uLisp for IoT Arduino, ESP8266 and MicroBit


This company does, https://www.siscog.pt/en/

More like Lisp stuff gets guaranteed upvotes and visibility. HN does a lot of Lisp service. This is Paul Graham's site after all.

Just mutual admiration society. It's a parentheses hell and good only for self flagellation. It does have a sense of elegance and can be fun but again, the parentheses hell would be enough of a turn off once you start writing long programs.

Use a regular "functional" language, like Haskell or scala and you get the best of all worlds

Now do WebAssembly.

Better than JavaScript.

No, that's just a Clojure inspired syntax for WebAssembly.

Yeah, that's true. Sorry, misunderstood the scope of the project.

But I guess it's closer to Common Lisp than what WASM's text format is, so might be helpful for some.

There are too many incomplete lisp-on-js solutions. No, CLJS doesn't count. It would be so awesome if one ever made it to actual usable maturity.

Is it too hard? Is there something that makes everyone give up? Or is lisp just too academic for most people to care?

1. Common Lisp is a fairly big specification for a lisp

2. People doing webdev in lisp have solutions that work "well enough" for them.

3. It's not obvious how to compile a lot of commonly used CL constructs into efficient javascript. Discarded multiple values, for example, are extremely efficient on bare-metal implementations, but will have measurable overhead when not inlined in any javascript implementation.

I'm curious why the same tricks can't be used on discarded multiple values?

If I recall correctly they typically involve manipulating the execution stack; that's not something you can do in JS obviously.

Fair. I was assuming you could do something at the evaluation layer that was somewhat equivalent. But it makes sense the bare metal ones are doing something at the machine code layer.

If you include Scheme under the heading of Lisp, BiwaScheme is pretty darned good.


I don't think I ever saw it mentioned in HN, but Hop[1] based on Bigloo[2] seems so much more than just a Scheme to Javascript. If I'm not mistaken, Hop (now hop.js) could even compete in a lot of use cases with Electron.

[1] http://hop.inria.fr/home/index.html [2] http://www-sop.inria.fr/mimosa/fp/Bigloo/

I do. Checking it out.

I find it somewhat arbitrary that you accept scheme but not clojure as a lisp.

I accept Clojure as a lisp. Didn't mean to start a holy war. I just don't personally want to deal with it and its ecosystem, which includes Java. I need your average web developer to be able to clone the repo, run yarn, and go.

I understand your point, CLJS is a superset so it's ideal to understand the substrate that it's based on, it's more complex than just JS from that point of view but usability wise:

  lein new re-frame myapp
  lein run

  git clone https://github.com/jacekschae/shadow-reagent.git && cd shadow-reagent
  yarn install
  yarn dev
It is very much fire and go in a way that a typical JavaScript developer might grok

Ah, I don't use CLJS, and I had thought it was self-hosting.

It could be done using WebAssembly. But AFAIK all existing Common Lisp implementations want to manipulate the stack in ways that WASM disallows.

There is schism if you are into Schemes, but it does rely on experimental support for tail calls and reference types.



Can you be more specific about the peculiar way in which CL manipulates the stack? Where to find more details about CL internals?

In CL, if you catch an exception, you have the option of trying to fix the problem and then letting the code continue from where it left off. This is only moderately difficult to implement in machine code, but somewhere between problematic and impossible on a high level platform that has already implemented exceptions by stack unwinding.

You mean call/cc type of manipulations?

Why doesn't CLJS count?

For the record, CLJS does count. And the tooling although painful is on par with the one found in JS (webpack). Maybe even a little less painful.

But one problem that ClojureScript avoids by virtue of being a 'hosted language' and not having a spec is having to implement its own numeric tower. Numbers can just be floats and move on.

CL mandates integers. Not just fixnums, like WebASM provides, but bignums. As well as rationals. That is a lot of code your run-time will have to provide.

iirc Whalesong (the Racket to JS implementation) bundled a numeric tower that implemented integers using strings).

Because I don't want Clojure. It's too opinionated. Also CLJS has (had? been a minute) painfully slow tooling.

> painfully slow tooling

On the topic of tooling, I found the tooling to be slow as well. However, a lot of my pain points were alleviated when I really embraced using the REPL for developing. Tools like figwheel make developing fairly painless although startup times are still slow.

Whenever I go back to using Common Lisp I'm always surprised how fast the startup time is by comparison.

I think his question is why is CLJS an incomplete lisp-on-js solution?

Not why you don't like or use it.

I completely disagree with that assessment. Disclaimer: I've been doing front-end for a long time. I've tried all sorts of ways to transpile/compile/deal with Javascript. I've tried Coffeescript, Typescript, Babel, Traceur, Fay, Haste, GHCJS, looked into Elm, Purescript and Reason, KotlinJS and ScalaJS. Clojurescript is one of the most stable and better choices for writing production-ready web-apps.

Clojure, currently is the most widely used Lisp dialect in the industry. Lots of people trying it and find it massively productive. Just because some lispers "don't like it", doesn't make it less valuable technological choice.

> Clojurescript is one of the most stable and better choices for writing production-ready web-apps.

Why? TypeScript, for example, is used pretty much everywhere these days; all the other languages are completely niche. I dislike the language (mutability everywhere, no standard lib), but it's dominating the frontend world ATM.

I've been using Scala.js in production apps for around 3 years now. Curious in your experience what advantages Clojurescript has over Scala.js. I ask because you often hear things like "best in class" when Clojure devs talk about the frontend development experience.

The REPL. Just like with any Lisp, true REPL makes a huge difference in the workflow. Anyone who never had honest and heartfelt experience with any Lisp or Smalltalk, would never understand how nice it is to be able to evaluate any expression and sub-expression without any kind of ceremony. Aficionados of non-homoiconic languages keep saying stuff like:

- "oh, we can probably do this and that and get the same experience...", or

- "give [their favorite language] five to ten years, it'll get the same..."

It's been over six decades. Non-homoiconic languages still don't have "true" REPLs.

"Clojure, currently is the most widely used Lisp dialect in the industry."

Citation needed.

I would bet money that this isn't true. There are a lot of places using proper Lisps that stay out of the public eye.

Facts do not cease to exist because they are ignored.

- Every PL ranking from RedMonk to TIOBE index indicates that what I'm saying is true;

- Clojure today has more conferences, books, podcasts, jobs and meetups. More than any of other language in its category of languages with strong FP emphasis. It is more popular than Haskell, Elm, Purescript, F#, Elixir, OCaml. Latest JVM survey indicates that it surpassed Scala and has become third most popular JVM language after Java and Kotlin;

- Clojurescript is the most popular alt-js choice (if you don't count Typescript as alt-js). Elm and ReasonML are far less popular;

- If you count by lines of code on Github and Gitlab, Clojure potentially may come second after Emacs Lisp (which is almost four times older than Clojure). Other Lisps wouldn't even be near their numbers.

- And please stop calling it an "improper Lisp." None of the authoritative Lisp scholars have ever indicated this even remotely to be true, in fact many have endorsed Clojure. Clojure can be viewed as a next evolutionary step in development of Lisp. Whether you like it or not, it is slowly spreading wide and beyond JVM (the platform it was initially intended to run on)

It's quite rare to see postings for CL jobs, yet plenty of companies are actively using and hiring for Clojure. It's certainly a far more popular choice for green field development nowadays. Sounds like you're the one who needs to provide citations here and not the other way around.

If you mean Common Lisp, that’s a Lisp-2 (actually Lisp-N), while JS is a -1. That means there’s no universal obviously “correct” way to do JS interop. It’s complicated...

Closure is a Lisp-1, as is Scheme, and (despite it being at least as “academic” as CL) there are lots of Scheme-in-JS implementations.

There is somehow a pretty obvious way to do C interop from Common Lisp, even though C has one namespace for functions and variables.

Maybe this is the one.

IMO, Babeljs could be leveraged to create one much faster.

My thoughts are along these lines, too. If it's not incrementally adoptable in webpack/etc projects, it won't see significant adoption.

The latter. Lisp is too academic for anyone to care to do it. Compilers need to be dogfooded, i.e., the people who work on it also write non-compiler programs with the compiler. TypeScript and Emscripten are mature because they are used to create real web applications. No one uses Lisp to write regular applications, much less web applications. Web applications in Lisp are unlikely to be its hidden niche after all these years, so very few people bother altogether.

> No one uses Lisp to write regular applications, much less web applications.

These are both false. This site is just one example. Here is another: https://wigflip.com/signbot/. And another: https://wigflip.com/roflbot/. (Base domain is https://wigflip.com/).

Grammarly is Lisp.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact