Hacker News new | past | comments | ask | show | jobs | submit login
Ask HN: 30y After 'On Lisp', PAIP etc., Is Lisp Still "Beating the Averages"?
113 points by dualogy 3 months ago | hide | past | favorite | 170 comments
Is it still at the "top end of the power spectrum", against which all others are merely different shades of Blub? And if so, _which_ Lisp and why?

Obviously this is asking for subjective opinions and experience among the crowd.

In one or another Arc (or Bel) intro essay, there's this notion that "a true hacker's-dream language" would be a new Lisp other than Common Lisp which is there accused of various suckages, and apparently also other than all the numerous Scheme variants (at least in the "no such corset of 'hygienic' macros, plz" sense). Perusing the various search results on 'CL vs Scheme / Racket' had me personally more convinced of CL for heavy-duty "real-world", non-"write-yourself-a-Foo" projects, though (wrt. perf / compilers / library ecosystem).

The last time I got hooked on pg's persuasive Lisp essays and got into it for a short while (cooking up an SSR templating engine somewhere in the PHP/ASP/Handlebars sense for Hunchentoot as a starter exercise) was in 2008. Real-world projects lured me away soon enough then and I didn't really miss it tbh. That short episode of first Lisping to me was just "kinda alright" rather than the-most-hackery-experience ever. Then ca. 2011 unleashed the novelties (for me) of WebGL, NodeJS, Golang and more, and those were enough "hackers delight" playgrounds for me for years to come.

But revisiting the essays (by now it might be nostalgia, like re-reading/viewing one's certain personal all-time / youth-time favourite novels, comics, movies once per decade or so, I guess =) I'm once again lured by the promise of "the language at the end of the power spectrum" against which every other one is just some shade of "Blub". Then again, I went into almost every language I ever got into from this sort of promising sense of power potential, including Haskell, C, and even Go (pre-generics) in its own ways of "power through simplicity, and then just code your code-gens for those rare meta-needs". (Zig, too, but though enjoyable, that kept changing itself up frequently enough back then to move me back to more settled grounds.)

You see the one sticky situation I keep ending up in is that I'm, in pg's words, always "painting myself into a corner" all too soon, hacking away iteratively and "dirtily" (in the make-it-work, then make-it-fast, then make-it-pretty/correct/maintainably-architected spirit — but I don't really get to the last part and only some of the 2nd if at all —we're talking green-field personal side projects here—, but anytime I start with the last part, it goes nowhere fast, nay slowly actually). I can paint myself into a corner in QBasic as well as in Haskell and anywhere in between (and lest I totally burn out my natural curiosity toward tech to learn to make things, I have to keep reminding myself that they probably aren't to be blamed as I'm the constant factor in all these experiences). And I keep picking up that "building stuff On Lisp" is maybe The One basis where that's either least possible / likely, or not a particularly impeding problem / hindrance due to "ever molding the tech basis / language itself around and toward the problem you're tackling".

So, what's your hot take on this question, your personal opinion, your experience report? In 2024, some ~30 years after On Lisp and some ~20 years after those essays, all the other languages have kept picking up nearly everything once-cutting-edge about Lisp, other than the "parensy pesky" that pg insists is Lisp's real power (convincingly so, at least from-a-distance, I might add), that whole melange of "sexpr (non)syntax / code-is-data / code-is-the-AST / read-time+compile-time+run-time macros in the same (non)language".

(And then, do we start purity-spiraling in a fever dream of a one-true-Lisp without the "suckages" of the existing and in-widespread-use ones? Probably not, eh?)




There are lots of interesting ideas to be found in the Lisp/Scheme family. A Lisp can be an excellent choice of primary dev language, especially for a dev shop which puts enough investment into training and tooling.

At the same time, the state of the art in programming languages has advanced since PG wrote those essays. Even many of the languages which he was presumably referring to as "Blub" are much better now than they were.

A bigger issue is: As one gets better and better at computer programming, at some point the specific implementation language starts to matter less and less. (It never completely ceases to matter; but, less and less.) In one of his essays, PG contrasts "applying Bayesian filtering" with "applying if statements", which describes the matter quite aptly. At some point, language constructs such as "if statements" really do melt away.

For this reason, I reject claims that using a certain programming language will make one "1000 times more productive" (or similar) out of hand. The big problems in software are not at the level of the language at all.

This may sound false to newer programmers who have just discovered a fantastic programming language which allows them to work much faster than they could before. To such ones: I've been in your shoes. I've tried many languages, perhaps even the one you are enamoured with now. The benefits don't scale indefinitely. At some point, as you apply your fantastic new language to bigger problems, you will find yourself wrestling with the same issues as you used to with your old language, perhaps in a different form.


> A bigger issue is: As one gets better and better at computer programming, at some point the specific implementation language starts to matter less and less.

There are things that you can do in more advanced languages which are simply not possible in the simpler languages. In that sense, e.g. Java is lower-level language than Scala. Easier to learn and probably faster too, but just much less expressive. I've seen expert Scala dev doing some magic in 100-200 lines, which literally saved us thousands of lines of code that would be a nightmare to maintain and which would be a neccessary evil in a lower-level language.


Sure. I've done similar things myself, though the "advanced languages" which I'm expert in do not include Scala. This is a good example of what I'm talking about in the last paragraph of the comment which you are responding to.


> The big problems in software are not at the level of the language at all.

IMHO this only partially true. Think e.g. of memory safety.


Memory safety is itself one of the "powers" that Lisp had that most other languages did not, back in the day.

But nowadays almost every popular modern language is memory safe. So it has become yet another thing that doesn't really differentiate Lisp anymore.


Eh, I've worked on a lot of really shitty C code bases. Yeah memory safety is a class of bug but that doesn't exist in other languages but every ecosystem has its share of footguns.

The real time sinks are caused by the same thing regardless of language; poor design.


Every language with a garbage collector has memory safety. It's not the issue.


It is the issue, when the industry nonetheless keeps on insisting on using languages that have neither garbage collection nor ownership semantics. Is your kernel, OS, userland memory-safe yet?


Using a programming language that you enjoy or you feel strong about is a much better choice, in that it will actually make you more productive simply because you’ll be programming more often.


> Using a programming language that you enjoy or you feel strong about is a much better choice

I enjoyed every single programming language I encountered and decided, or was compelled, to learn. Starting with BASIC, then Pascal, Assembler, C, VB, C#, F#, Java, Javascript, that followed by the plethora of "script" langs — Coffeescript, Typescript, Livescript, IcedCoffeescript, GorillaScript, etc. After that came Python and Haskell. Almost every book promised unbelievable riches and magic with this new (to me, at the time) language. Sooner or later, though, the long honeymoon would come to an end. At some point, while dealing with yet another clusterfuck of a project, I began to feel that I was simply not a good programmer and that I would never become one. Then I discovered Lisp. Somehow, inexplicably, things started making sense. Shit that confused me about Haskell and Prolog, I for whatever reason have started to grok, or at least I felt like it. For the things I build today, I use Clojure(script), Fennel, Elisp and CL. Even when I'm required to produce code in a different language, I still first prototype it in a Clojure. Not because I'm obsessed or blindly in love with one specific programming language, but because for the applications I'm building today, it makes sense — for me. I wouldn't argue that it would make sense for everyone, no. But for me, it does make sense. Today. Tomorrow that may change. Any given programming language is very unlikely to make you ten times more productive or accurate, yet it may provide you with even greater satisfaction. How? By allowing you to build something you love with it. Build something. Anything. And keep building more. And if you don't love it, rebuild it in a different language until you do.


The power of Lisp really boils down to the personal experience of having a connected REPL. If you can evaluate any expression and sub-expression from your editor without having to move it anywhere, retype it, restructure it, or set up a stage for the surrounding state, that is extremely liberating and empowering.

I personally don't understand why anyone who has had the first-hand experience of writing software this way would willingly give up on this fun. Yet, I also understand that personal impressions can be deceiving. Just because I love chocolate and every single person I have ever met also loves it, it doesn't mean that there aren't people who genuinely hate it.

Sure, tons of other programming languages have borrowed features from Lisps and continue to do so, but there exists an emergent category of programs that are assuredly more difficult to build using non-Lispy languages. Check out, for example, hyperfiddle/electric.

My biggest personal regret is that I discovered Lisp too late in my professional life. For a very long time I have had the same experience you're describing — starting projects with high promise but then hitting practical roadblocks. With the caveat that none of them were in Lisp languages.

Today, I don't even think about it — if I have to build something, I'll grab a Lisp dialect. I don't consider Lisps to be different languages; I can relatively easily switch between Clojure, Clojurescript, Clojure-Dart, Fennel, Common Lisp, Elisp, etc. I can't say the same thing even about Javascript and Typescript. Darn, even when I have to switch between different JS libraries, it feels like I have to deal with some mental overhead. That doesn't happen with Lisps.

At the end of the day, there are always trade-offs, whatever tools you choose and whichever language(s) you pick. Nothing is ever perfect. For me, if something isn't working, then maybe I need to find a Lisp better suited for the job. And usually, there are always options to choose from.


> The power of Lisp really boils down to the personal experience of having a connected REPL.

Yes, and it's a lot more than just typing in one liners (as the person below suggests). For instance, in standard C you can't redefine functions/procedures at runtime (e.g., to add debugging code, or even switch to an updated version in a running system without requiring a restart). Some of the more modern languages have that facility, but it's rarely as elegant as it is in Lisps. It's a consequence of functions/procedures being first class values, and is really hard to do (or do well) in any language where they aren't.


>Some of the more modern languages have that facility, but it's rarely as elegant as it is in Lisps. It's a consequence of functions/procedures being first class values

Can you explain how it is a consequence?


IMO, the OC is incorrect. First-class functions exist in many languages. As a matter of fact, most features that were first pioneered in Lisp now exist in many other programming languages. Languages have borrowed everything they could from Lisp, except its single most powerful feature — the homoiconicity. Homoiconicity remains less common outside of Lisp-like languages, though it is not entirely absent. Prolog, Rebol, and R are homoiconic programming languages without Lispy syntax.


> IMO, the OC is incorrect. First-class functions exist in many languages.

Note that I did say "Some of the more modern languages have that facility, but it's rarely as elegant as it is in Lisps."

Example: JavaScript is sometimes claimed to have first-class functions. It kinda does, but you can't redefine (e.g.) the "+" function as you can in Lisps. So it's basically half-assed. C has function pointers, which again is (sort of) like first-class functions, but also don't let you redefine "+" (they're also incredibly ugly to use). C++ has operator overloading, so you can (again, sort of) redefine "+", but unless things have changed since the last time I used it (which has admittedly been a long time, though not long enough), I believe that's only available at compile time, not run time.

And of course even in (e.g.) JavaScript when you redefine a function it doesn't automatically get compiled, as it does in many Lisps (maybe the JIT will eventually get around to it, maybe it won't).

And so on.

(I'm using "function" here as a shorthand for "function or procedure" because I don't want to type that a half-dozen times).


Ah... now I see what you meant. Indeed. + is an operator in langs like Javascript and C++ - a built-in, rigid syntax construct. You can't redefine it, you can't extend it, you can only use it for addition and string concatenation. You can't even extend it in JS to let's say, merge two objects together.

Lisp doesn’t even have "operators" — everything is a function. Gosh, the more you learn about Lisp, the more it feels like we went backwards. We tried to simplify programming, only to find ourselves buried under layers of complications. Just think about how many times we've reinvented state management for React alone: Redux, MobX, Recoil, Zustand, Jotai, Effector, XState, Overmind, Hookstate, Easy Peasy, Rematch. And that's not even a comprehensive list. What's funny is that some of these are "improved" versions of previous "improvements". Each time we try "improving" the previous "improvement", it seems we continually need to reinvent another layer of "improvements". Dafuk are we doing?


I started out writing machine code. With each "improvement" we lost flexibility and I didn't need any of it but they are all useful experiments and lots of things objectively improved. Imho software is for fooling around. We need it because we don't exactly know what we need or how to make it. When the needs are exactly defined we can bake thing into hardware. At that point all flexibility is gone.


The syntax of + and - is rigid in C++, but functions can be written to extend it. This is not new; you are making claims that were not true in the first ISO C++ in 1998, or its predecessors revisions going quite far before that.

When a is a class type (and more recently this is now supported for enumerations also) then the surface syntax a + b means the same thing as a.operator+(b): invoking the operator + member function of a with argument b.


> functions can be written to extend it.

Unless I'm mistaken (which I may well be), you can't do that at runtime, only at compile time.


I replied to this not knowing exactly where the goalposts are.

Indeed, in C++, our a + b expression is statically analyzed to be working with specific types; the declared types of a and b. When that code is compiled, it will work with no other types.

However, if we anticipate that kind of extension, we can have a be a reference to some numeric_base class where operator + is a virtual function.

With some help from a bit of platform specific coding, we can have dynamically loaded modules which can provide new classes derived from numeric_base that ca be passed to existing code in the program which uses + on numeric_base.


> Lisp doesn’t even have "operators" — everything is a function. Gosh, the more you learn about Lisp, the more it feels like we went backwards.

Don't learn wrong things. Lisp has operators and no, not everything is a function. LET for example is not a function. It is a built-in special operator in Common Lisp. Similar in other Lisp dialects. Scheme R7RS documents "let" as syntax: https://standards.scheme.org/corrected-r7rs/r7rs-Z-H-6.html#... with a "named let" as a variant: https://standards.scheme.org/corrected-r7rs/r7rs-Z-H-6.html#...

In Emacs Lisp: https://www.gnu.org/software/emacs/manual/html_node/eintr/le...

"The let expression is a special form in Lisp..."

Also: limiting the redefinition of standard operators is not backwards, it's useful, to prevent to crash a Lisp, since in Lisp function names are often late bound. -> changing a core function will have immediate effect on all code using that.

Macros are also not "function". Macros are operators which expand source code to new source code.

Lisp typically has these operator types:

  * functions, which are operators which get called with evaluated arguments
  * macros, which are operators, which get called with source code and which return new source code
  * special operators (QUOTE, LET, DEFUN, SETQ, IF, ...), which are typically hard-wired in the implementation with both syntax and semantics. Scheme has a set of those, too.
Additionally one my see fexprs, which are operators, which are called with source code. Then in some Lisps some functions lazy evaluate their arguments.

The glossary of Common Lisp explains "operator": https://www.lispworks.com/documentation/HyperSpec/Body/26_gl...

  operator n.
  1. a function, macro, or special operator.
  2. a symbol that names such a function, macro, or special operator.
  3. (in a function special form) the cadr of the function special form, which might be either an operator[2] or a lambda expression.
  4. (of a compound form) the car of the compound form, which might be either an operator[2] or a lambda expression, and which is never (setf symbol).
> You can't redefine it

In standard Common Lisp it is UNDEFINED what happens when one redefines a standard operator of Common Lisp. Typical implementations will signal an error (which often provides a restart). Typical implementations will also extend this to other protected packages, such that the user can't accidentally replace an operator and the Lisp system may crash.


The example of electric is really good. It's something that would require bonkers amounts of effort to implement in ts/js, but instead 15 contributors could build in clojurescript.

And I think the core thing that makes it possible is that the design decisions in lisp, optimize to provide expressive power to the user, because as a language designer I cannot perfectly plan what you will build.

Which in turn enables the users to build things that would require crazy effort in more traditional languages.

And yet, most programmers have an irrational fear of built-in metaprogramming because it "makes hard to understand code". But seemingly nobody is afraid of bad metaprogramming, like code generation. Which I would argue is more difficult to understand.

Seemingly, any sufficiently hard problem requires some new language constructs to be built to discuss it in a way that makes sense. A good example of this is react. It's a framework in JS, but when using react you aren't really writing normal js. You are writing react. This is magnified when using jsx, which few people will complain about using, as it makes the process of writing react code much more enjoyable.


> And yet, most programmers have an irrational fear of built-in metaprogramming because it "makes hard to understand code". But seemingly nobody is afraid of bad metaprogramming, like code generation. Which I would argue is more difficult to understand.

The difference here is that “bad meta programming” happens when the people writing thorny libraries and frameworks clamor enough for meta programming that the language implementers shoehorn in a subpar implementation. These people know how important it is to programmatically generate and modify code, because they had problems that just couldn’t be solved without that, and the other language users never see it (very few programmers in most non-Lisp ecosystems are curious about the code of libraries they use) and so don’t care whether or not it exists.

On the other hand, a language with pervasive meta programming is going to have cases where you could do a task without that feature, but can do that task much more easily with it. Meaning that of the programmers using that language, everyone in the subset of “programmers that like to minimize what they need to learn as much as possible” will be exposed to code using meta programming.

Most of these engineers will not have the prior experiences to understand that “ok, we need to learn meta programming either way because it’s essential to many important coding tasks and so has to be in the language; it’s not an increase in learning/comprehension difficulty to also have it in code that doesn’t strictly require it”. As such, their reaction is simply “why do I have to learn how to work around this extra thing?! Get rid of it, now!!!”.


I can't imagine REPL-based "development" is good for anything other than one-liners and quick-n-dirty one-off stuff.


REPL-based development doesn't require typing directly into the REPL. It's usually a text editor side-by-side with the REPL. Most LISP IDEs have a command to send forms over to the REPL to be evaluated. In this manner, you develop your code iteratively with the REPL almost as if having a conversation with the REPL. You try a form, and if it doesn't work, you change and iterate, line by line of your code. Through this "conversation", you've gone from a single form, to an entire function that works, than a module. You build from the bottom up. There is no, code-compile-test cycle, it's always instantaneous. With some REPLs (like Common Lisp), you can even break from an exception, update your code and resume the code from the point before the exception. It's much more than the REPL most people are used to with Python or Ruby.


Then you should upgrade your imagination ;-)

REPL-based development in a language that does not have large scopes of nested things is just so great. You can redefine everything on the fly. That is incomparable to restarting whole programs. Scala also has a repl but it not nearly as useful with idiomatic Scala code, because lot's of stuff is defined in nested scopes and by inheriting and extending trait. After a simple code change you have to recreate lot's of state. That does not happen in Clojure when you work primarily with builtin data types and lots of functions. This advantage is not only based on the REPL, but also on the way the syntax of is designed.


No way man. REPL is *more* useful the bigger, more massive the project.

I work on massive C# website that takes forever to warm up and get running. Stuck on old .net framework. A rebuild/rerun of the website after each little change is incredibly frustrating and slows down the feedback loop. Fortunately Microsoft implemented hot reload, and it has been a game changer. It still has all sorts of quirks and gotchas, but despite that I love it.

If there is one thing to take from common lisp, it's the live image development style. It's unfortunately hard to bolt on to languages not designed for it from the ground up.


If market conditions allow it, you stand to gain a huge improvement to QoL of dev loop by getting a position that is on a modern version.

But I don't know your situation, just sharing that .NET Framework based products are dreadful to work with in 2024 over .NET 6/8.


To add on to what others have said, a good repl lets you point it at at a minimum of a file and load its contents into active state (so if you want access to a bunch of functions you load that file based on the command of the language you are in). You can then do the one lines you are talking about, but you could also update the file if you find you want to tweak a function and usually highlight and ctrl+enter to re-evaluate that one function, updating its state in the live environment. This way instead of recompiling the entire system you recompile that one function and anything using it now points to the new version.

F# actually has this and while it probably is not as powerful as Slime with CL it can give an idea of the power you are missing (and F#'s has gotten some nice improvements in the script side of .fsx files such as being able to add nuget files with a line in the script in newer versions).


> I can't imagine REPL-based "development" is good for anything other than one-liners and quick-n-dirty one-off stuff.

Perhaps your imagination would benefit from some elevation. Try shrooms or something :) I think your understanding of what a REPL actually is might be somewhat skewed due to a shallow understanding of how it works with Lisp dialects. The experience of working with the REPL in Python, for example, is not equal to the experience in e.g., Clojure.

There are many different companies that leverage Lisp dialects to build their businesses. Companies like Cisco, Apple, Walmart, Nubank, Grammarly, Funding Circle, Pitch.com, Rate.com, and many others are not using it for "quick-and-dirty one-off stuff"


We use CL for many things and it’s definitely a big average beater for us: our clients and competitors (we have no competitors, not even small ones with less than 50 devs) think we are massive by how much we churn out. We are a tiny team beating the averages.


How long have your shop been using CL ?

I understand the value of mainstream languages at the group level (ability to distribute work through simple layers), but my brain doesn't like this way of working. I used to churn a lot on my own and I miss some of that (the speed and the ability to find creative / novel ways to solve problems).


We started with c# decades ago, but lisp (and prolog by the way) kept nagging, so about 8-9 years ago I started getting my senior devs in workshops ; after about 6 months some of them asked wtf we are doing with that c# stuff?! While that is unfair, the debugging experience, stability, flexibility etc are more painful for my taste.

Also, the biggest complaint by many; I have not found any cases where packages we were missing in lisp but are there in nupkg or npm actually sped up any work in the long run. Our work is very regulated (so we check, read and fork the stuff we use to make sure nothing bad lands in prod) and our integrations almost never have integration packages available and if they have them, they are out of date or very badly done because everyone just does it from scratch to make sure.

Maybe we are weird and that is fine, but, if there is one thing; we iterate faster than our competitors.


> While that is unfair, the debugging experience, stability, flexibility etc are more painful for my taste.

You mean c# debugging and stability are bad relative to your own debugging workflow (I can imagine how CL makes things way faster) ?

> lisp (and prolog by the way)

Talk about low odds, I don't know how many shops use both but that seems like a rare gem. btw you use swipl as your prolog implem ?

Do you do conferences discussing some of your work ? I'd be very interesting in following that :)


This is great to hear! What does your team use Lisp for?


We make software for streaming/realtime data analysis; our clients are, by first mover/network, mostly banks. We have been building for 20 years next to our ‘consultancy’ (no one else gets a peek; we are not investors like PG so I rather have the competitors slowly bleed Java and Python). We are not 100% cl yet (we started out with c# and Java), but all things that change a lot are.


Is there something in CL as opposed to a Scheme-impl that you would miss or would Scheme give you the same edge in your opinion?


Well, CL is very mature, with sbcl, abcl, ecl, clasp, clisp, lispworks, allegro etc all with different pros and cons, editors and libraries, besides the elegance of scheme, I would have little clue why one would use anything else.


Since you are focusing on CL, what editor or tools do you and your developers use? How do you onboard new devs for lisp roles?


It’s more true than ever that large projects resemble compilers. The existing llvm Frankensteins and code generators are evidence that people don’t know how easy this kind of thing is in lisp. As a concrete example look at auto diff.

The other advantages are less true because dynamic languages are prolific and powerful. But also you never really grok JavaScript or Python until you study Lisp. It’s common to see what’s essentially C or Java written in python syntax.

And let’s be fair nobody in the 90s understood C++ including pg.


LLVM IR and codegen tooling are both solutions to performance, not expressivity. The ergonomics of a language that will never go to production are irrelevant.


Although thats how llvm is serving the C family if you look around you will find a lot of projects and startups where that motivation is not well founded.

Common Lisp also lets you easily compile your language to assembly. To get that additional llvm benefit is at least 2 orders of magnitude of effort, but it’s probably not 2 orders of magnitude faster.


I chose Common Lisp for my web publishing system. I could not be happier.

Some benefits of Lisp that are less commonly reported:

Lisp let's you imagine a way of doing something, and then just implement that with nothing getting in your way. You can go from something half-assed to a whole-ass in no time. The type system lets you develop quickly and optimize when it's time. You can run interpreted, compiled, and even optimize the rendered machine code.

Image-based computing let's you skip the database for a lot of things as you can just read data into an image at startup and have, essentially, a fast in-memory DB.

Language-oriented programming allows you to extend Lisp with nouns and verbs related to your problem domain. So you can express the solution in terms domain experts already understand.

The hugely underappreciated condition system is far more than a way to do error handling, but can respond to events and do flow control.

Modernizing Common Lisp is something everyone does. We each curate our own distribution from Quicklisp and develop in that. More coordination there would be welcome, but we're not writing in your dad's Lisp.

A kitchen-sink, multi-paradigm language is ideal compared to a more opinionated system in which there are official paved paths to do things. You get to spend more time figuring out what needs to be done than figuring out the official way to do it. If there is a downside to that, it's that you have to take apart enough flashlights to figure out how things work and can't just NPM somebody's afternoon project over and hope it works.

When you have a perfect spec and you just need to implement it efficiently, think C. When you're not totally sure what you're doing, want to explore the problem space and not paint yourself into a corner, think Lisp. Actually, think Lisp anyway because specs are always wrong, so you're always exploring. Lisp makes for living systems that grow and change organically. C is for making complicated but dead automata.


> Lisp let's you imagine a way of doing something, and then just implement that with nothing getting in your way.

That's something that stuck with me from the SICP lectures: problem solving via wishful thinking. That's part of why I love Lisp.


Right on.

The way I think of it, CL is the rare language that I don't feel is fighting me every step of the way.


> other languages have kept picking up nearly everything once-cutting-edge about Lisp

Lisp makes metaprogramming easy via macros. It could be argued meta programming can be done in any modern language, but because lisp’s code is a data structure, metaprogramming in lisp is as easy as changing a value in an array, hash, or dictionary (whatever data structure you’re familiar with, it’s that easy), as opposed to regexing strings, escaping correctly, and other finicky and error-prone hijinks other (non homoiconic) languages necessitate.

The other thing lisp does well is its REPL. I’m not so familiar with it but AFAIK it’s like an interpreter and debugger combined into one, giving a very nice developer experience.


The REPL is why I like python a lot as well, although it is not as powerful as lisp where you can inspect and modify running code easily, it really is a key feature for prototyping new ideas. More languages should have repls


Python (or any non-Lispy language) REPLs are not truly REPLs in the way that Lisp understands the REPL. Every phase of Read-Eval-Print-Loop cycle has limitations:

Read: In Python, the input is read as strings of code, which are then parsed and compiled. In Lisp, the input is read as data structures (lists, symbols, etc.) due to its homoiconic nature (code is data). This allows for macros and syntax rules to be applied even before evaluation.

Eval - The evaluation phase involves interpreting the compiled code. Python’s evaluation is less interactive; changes in function definitions or classes often require the code to be re-run entirely, and the state can become inconsistent without careful management. Evaluation in Lisp happens directly on the read data structures. Functions, macros, and variables can be redefined interactively without restarting the REPL, maintaining a more fluid development process.

Print - The result of the evaluated Python code is printed out in a standard format. The printing is straightforward but lacks customization unless explicitly coded. In Lisp, results are printed in a way that preserves the structural representation of the data, facilitating easier inspection and manipulation in subsequent steps.

Loop - In Python, the main loop doesn’t offer much in terms of tooling or integration. Extending the REPL with custom behaviors or tools requires additional packages or integrations. The Lisp loop often includes advanced features like hooking into debugging tools, live editing, and runtime modifications, providing a richer development experience.

So, it really makes you ask: can you actually have a proper, "true" REPL in a non-Lisp language? So while it's fair to call them REPLs in a technical sense, they more like "interactive shells" and not REPLs. You really need a Lisp to have a proper REPL.


I'm a very long time Python developer, but also did a project in Clojure, a type of lisp, over the last year.

There is no comparison between the Clojure REPL and the Python REPL. Python's is just literally a shell that you connect to and can run commands in. The Clojure REPL is far closer to a Python Jupyter notebook (if you know about it) built into your editor - it's a long-running process to which you can send commands all the time. But because of the way lisp is structured, and because of the tooling, you can just write Python code in the file you would normally use, stand on one line, and hit a command to get it sent to the REPL and the output of it written in the editor.

It's... hard to explain how powerful this is, and how much better things could be if we had similar tooling in Python.

Note: I still much prefer Python for many reasons, but this one aspect is something that I also used to think (yeah, Python also has a REPL) and I realize now that I had no idea what power this really has in lisp-world.


> The Clojure REPL is far closer to a Python Jupyter notebook (if you know about it) built into your editor - it's a long-running process to which you can send commands all the time.

[...]

> It's... hard to explain how powerful this is, and how much better things could be if we had similar tooling in Python.

It would be kind of odd if we didn't, given what ecosystem Jupyter comes from. Sure, it's kind of surprising that the editor to integrations to do this haven't been around for a long time, but...

https://code.visualstudio.com/docs/python/jupyter-support-py


I haven't looked at this specific plugin in a while, so I might be wrong, but from a quick glance (and from vague memories), it doesn't seem like the same thing.

Look, I've been using Python professional for 16 years, and ever since starting to use Jupyter 7 years ago, have been a huge fan and proponent of Jupyter. But the Jupter model is different from the Clojure "true REPL" model.

The main difference is that in Jupyter, you're doing something that is outside the normal files you use to write code. Both by writing in cells, but also you're physically in notebook files which are not part of a "regular codebase". Sure, if you're doing standalone analysis work or something, that makes a lot of sense. But if you're, say, writing the BE of a webapp, then you'd have your normal Django or whatever .py files, and separate Jupyter files for one-off analysis, or for writing testing code, etc.

In Clojure, those can be unified in a way, both in the development process and in the future. Meaning, you'd start off with the equivalent of your Django .py file, you'd make a special "smart comment" which doesn't get executed normally, but which you can manually execute via an IDE command, you'd write some code there, and once it's working, you'd move it outside the comment to the "regular" part of the file. You've now essentially iteratively developed a new function that is just part of your normal codebase.

And the "comment" that you wrote all the testing code in? That comment is still part of your file. You can use it to document the way your code runs, but in an interactive manner - anybody chancing on this file can just execute commands out of that comment and immediately see what they do.

It's... hard to explain, but it's a really powerful model, that completely obliterates the line between "exploratory" code like writing in Jupyter, and the "production" code that ends up in your actual codebase.

If you've never seen it, I highly recommend you look for a good video example of Clojure REPL-driven development.


> The main difference is that in Jupyter, you're doing something that is outside the normal files you use to write code. Both by writing in cells, but also you're physically in notebook files which are not part of a "regular codebase".

That's not the case here (it works in normal .py files.)

> In Clojure, those can be unified in a way, both in the development process and in the future. Meaning, you'd start off with the equivalent of your Django .py file, you'd make a special "smart comment" which doesn't get executed normally, but which you can manually execute via an IDE command, you'd write some code there, and once it's working, you'd move it outside the comment to the "regular" part of the file. You've now essentially iteratively developed a new function that is just part of your normal codebase.

Yes, that exactly will work in the VSCode Interactive Python environment and is what distinguishes from normal Jupyter notebooks even though it relies on Jupyter under the hood (=the code can't actually be in a comment if you want to use the integration to send it to the interactive environment, because Python only has single line comments so sending it to the interactive environment would literally send a comment, but it can be in guarded-as-unreachable code -- it could also be in a triple-quoted string, which can work like a comment, but then you'll lose syntax highlighting, completion, etc., while writing it.)


That sounds pretty great, I'll definitely check it out.

One thing I'm skeptical about is whether the way Clojure itself is written, with lisp-style S-expressions that are identical in look, is an inherent advantage for this kind of work over the more standard way Python works. I'm curious to find out.


I try to tell my friends this and they accuse me of "gatekeeping" REPLs and that I'm an old man yelling at the cloud.


It's actually not that hard to explain, yet without experiencing it firsthand, it is hard to see what this REPL fuzz is about.

So, let's take a minimal, simple example:

   if True:
       print("Hello, Python!")
and the equivalent code in Clojure:

   (when true (println "Hello, Clojure!"))
In Clojure, you can send any expression fragment like a single symbol, list or even incomplete code and it will be evaluated according to its context. So, you can sent `true`, you can send `println` and you can even send `(when true println)` or `(when true (println))`, or even `(when true ,,,, (println))` (you can replace commas with line breaks if you want) and it will still be evaluated.

In Python, you can send `True`, and you can send `print`, but you can't send `if True: print` (even with corrected indentation), because Reader would have hard time making sense of that.

In Python, you need to adhere strictly to its syntax, and partial expressions that don't form a complete statement won't be evaluated meaningfully. In contrast, Clojure's REPL can work seamlessly with code fragments because the underlying representation of code is just data structures that are always valid within the REPL context.

What makes it mind-blowingly cool is that you can send these expressions to a remote REPL without any transformation, serialization, or encoding, because it's not just code — it is data. In fact, some teams do just that. They'll keep an open socket on a running app, possibly performing somewhere in the cloud, so they can interact with it in real time. Putting aside the notion that this might not be the greatest idea, let's agree that this is a cool feature. The famous example of that is when something broke on Deep Space 1 (DS1) spacecraft in 1998, the team was able to fix it via REPL, see https://www.youtube.com/watch?v=_gZK0tW8EhQ - "The Remote Agent Experiment: Debugging Code from 60 Million Miles Away"

Sure, yes, Python is great, but when it comes to REPL stuff, you simply need a Lisp. It really is an amazing feeling when you can, for example, poke into any HTML element on a page dynamically, directly from your editor. Sure, you can do that with devtools in the browser, but it's not the same. You still have to switch to the browser; you can't type statements that are too long; and it's just not very convenient.

Very nice (now years old) example is when Bruce Hauman showcased developing a "flappy-bird" clone - he would tweak the params, or straight change the code in his editor and the bird in the browser would fly differently, it's literally like writing code while playing a video game. Or Sam Aaron, writing music with Overtone is another great example. Another, more practical example is when you're exploring API response results, if you have REPL - you can send a single request and then dig through the data - group it, slice it, dice it, sort it, etc., something seemingly simple like this would require a bit more effort with any other language. I have my editor connected to a Clojure REPL instance at all times; I never know when I might need it.


I run an algorithmic trading application that I wrote from scratch. I'm fluent in C#, python, lisp, and I speak but am not fluent in C/C++, js, Fortran, go, and a few others.

I chose common lisp. It's very expressive which means my development velocity was very high with minimal boilerplate. I love being able to build functions up from the inside out in the REPL, and the DX of doing this in emacs is awesome because of the mature support built into the editor.

The library ecosystem in lisp is not stellar, and I think this is lisp's biggest shortcoming. With that said, there's still robust libraries for threading, networking, JSON fuckery, and (importantly for my use case) statistical analysis.

Lisp is great and the SBCL compiler produces fast code.


Half of the reason Lisp never caught on is that other languages have become better and added many of the Lisp features. First-class functions especially are is the killer feature you can't live without anymore once you tasted it.

The other thing that pg keeps hammering on about is Lisp Macros which I am not well versed enough to properly say how good of a feature they are. The alternatives most other languages have are much less powerful, but at the same time it seems that macros are very... abstract-y?

Modern software design thinking says that abstractions should be avoided when possible because they make the code harder to grasp for people not familiar with the abstraction on top of making the code too rigid and hard to modify once new requirements come in.

The alternatives to Lisp Macros like codegen, method annotations, Rust Macros. Are all much less powerful and usually harder to create, but maybe that is a good thing.


Our 30kloc Julia codebase only has one key macro, but it's a really important one. It's ~70 lines of code that's saved us ~5,000 loc, and more importantly let researchers do a lot of stuff that would need normally need more programming expertise. We might be able to live without it, but it would have cost us a significant amount of time maintaining and debugging, and a much slower research feedback loop.


Other languages with less powerful macro systems often create a DSL (domain specific language) for this kind of problem. With Javascript it can even be mixed in with normal code through the use of tagged template literals. For example:

  const query = sql`SELECT * FROM User WHERE name=${name}`
It is so widely used in the JS ecosystem that editors often have plugins to add proper syntax highlighting to some tagged template literals and some frameworks pre-compute some tagged template literals at build/bundler-time.

It is definitely much harder to do it compared to Lisp-like macros though. But maybe the barrier to entry to create such powerful abstractions is a good thing. In my previous post I didn't say you should never use powerful abstractions, you should just not overuse them.


We did in fact start by creating a data structure that we parsed!

The main problem we were trying to solve is essentially that we had a lot of code like:

f(x) = sma(ewm(x))

g(x) = slope(ewm(x))

And we wanted to avoid recalculating ewm(x), while also not holding memory for it for longer than necessary. Our production use cases would have tens of thousands of intermediates, so the potential saving from getting this right is huge.

Naturally a DAG is a great fit for this type of problem. We started by creating an explicit DAG, where everything like ewm(x), sma(ewm(x)) would be a node in the DAG. We could then evaluate the DAG efficiently using multithreading, dropping nodes when everything that depended on them had been calculated, etc.

And I would usually stop there, but there were two main challenges. One was that the syntax was pretty clunky, and we wanted researchers (who weren't software engineers) to be able to use it. The other was that we wanted _any_ reference to something like ewm(x) to map to the node, even if they were used in very different places – and it was quite difficult to map these dependencies without explicitly tracking.

It's hard to quickly summarize our macro, but basically what it let us do was write the short syntax and track dependencies using the underlying Julia expressions. We mapped those expressions to nodes in the DAG, and that let us calculate the dependencies without having to track them explicitly. I don't think this would have been possible without some amount of homoiconicity and macros.


Can you elaborate? I'm using Julia macros to help make my research software easier to extend for others, so would be interested in your experiences.


See my reply here: https://news.ycombinator.com/item?id=40587847 Happy to answer any other questions!


> The alternatives to Lisp Macros like codegen, method annotations, Rust Macros. Are all much less powerful and usually harder to create, but maybe that is a good thing.

I kinda agree that macros can lead to a lot of (bad) abstractions if you overuse them, but when you need a macro a lot of times the alternatives a language provides can somehow feel more magical if you try to get them to do macro-y things (decorators/attributes... languages always seem to mix those two terms) or just cumbersome and in my opinion kinda an ugly hack to paper over the fact that your language is lacking in some way (codegen). I'd much rather read a simple (lisp) macro then understand how your ugly codegen system works.

I dont have enough knowledge to comment on rust macros though.


> First-class functions especially are is the killer feature you can't live without anymore once you tasted it.

Wouldn't it depend on application domain what's a killer feature & what's not?

"Include if lightweight & very useful" seems appropiate. But what's considered "lightweight" varies depending on who you're asking / where you're looking, doesn't it?


I disagree, first-class functions and closures are just so very useful to the point that even languages that have added it after initial release still feel gimped today because a lot of the older libraries (and probably most of the stdlib) don't use them extensively.

Like passing a function to any function that iterates over a data structure (like array sorting), or registering an event handler. Sure you can usually achieve the same thing without first-class functions but it feels so much less concise.

I just love first-class functions so much that the main reason I dislike python is that it doesn't include proper anonymous functions.


The whole "top of the power curve, all other languages are Blub" never was true.

Consider Lisp and Haskell. When a Lisp user looks at Haskell, they know they're looking down. No macros? How can you get anything done in Haskell? But when a Haskell user looks at Lisp, they also know they're looking down. No decent type system? How can you get anything done in Lisp?

You have two languages, both looking down at each other, and both knowing why they're looking down. Something is wrong.

I claim that what's wrong is the idea that languages can be ranked on a single-dimensional axis labeled "power". To those who think it can, I ask, power for what? For writing programs? Which programs? And don't say "general programs". I've never written a general program in my life. I've written a bunch of specific ones, though.

What I actually care about is power for writing the specific program I'm trying to write. (Why would I care about power for writing the programs that I'm not trying to write?)

So the really useful question is, what makes it hard to write the program that I'm trying to write? (I don't mean what problem the program is trying to solve. I mean, what things make it hard to write a program that solves that problem.) This may be a variety of things - team skills, algorithms, memory management, thread management, and so on. You can think of those things as a vector in a multidimensional space. Then, think of languages as a tree in that multidimensional space. Pick the language that is the branch that reaches the furthest in the direction of the vector that is the things that make it hard to write the program. (Oh, and libraries. The availability of libraries can make a huge difference.)

Or, to state it without the metaphor: Pick the language that makes it easiest to write the program you're trying to write.

For example, say you're Google, and you're writing programs that have to have large numbers of threads, and have lots of dynamic memory, and will be long-lived and worked on by a large number of people who have to be brought up to speed on the existing code. Go is a language that addresses that set of problems. Neither Lisp nor Haskell would work as well. They may be "more powerful" in some sense, but they're not more powerful for that program.


The whole point of Lisp is to allow you, the programmer, to make the language that makes it easiest to write the program you're trying to write. Said language will be syntactic abstractions in Lisp primarily built out of Lisp macros, but there are additional means available to integrate your language idea into Lisp outside of macros.

So, really, Lisp is in exact agreement with your position, with one proviso: Pick the language that makes it easiest to write the program you're trying to write—and express that language in Lisp.

Case in point: Lisp does have a powerful type system that rivals Haskell. Full type inference, multi-parameter type classes, parametric polymorphism. It's called Coalton[1], and it's... just another Lisp library.

Whether or not you can be bothered to spend the effort to align Lisp to your domain is another question, but the mere fact it's both possible and reasonable to do is a testament to what some people mean by statements like "Lisp is powerful".

[1] https://github.com/coalton-lang/coalton


This idea was popularized by Graham who wrote (in the very article this submission is about) that 20-25% of the Viaweb source code was macros, and made careless hyperbole like: "Programs that write programs? When would you ever want to do that? Not very often, if you think in Cobol. All the time, if you think in Lisp."

The idea has become oversold, and has caused some harm by giving rise to the circulation of reactionary "Lisp curse" type misinformation.

Lisps are useful without the application programmer writing a single macro.

Most Lisp coding style guides include a few words to discourage the writing of new macros.

Macros, as such, help Lisp dialects themselves grow. That in turn helps programmers need to write fewer macros of their own.

Macros are great for managing language growth and versioning.

One example is that, within certain limits, it's easier to make a Lisp program work with an old version of its language implementation in which certain newer macros are missing, than to make programs in other kinds of languages work with older language implementations in which syntax is missing.

Programmers using Lisp but not writing macros are certainly not missing the point of Lisp, let alone the entire point. You do not think about programs writing programs "all the time" when coding in Lisp; it is hyperbole.


I disagree with you on many points.

I do not claim that not using macros means that one cannot use Lisp usefully or profitably. It's a wonderful language even without macros, and I'm sure I'm preaching to the choir by listing all the reasons why.

However, I do think adamantly that the "point" of Lisp really is syntactic abstraction. Why have S-expressions, code-as-data, symbolic data types, etc. otherwise? All of that is there to enable syntactic abstraction.

If out-of-the-box Lisp is sufficient to express solutions to your problems reasonably, then I would wholeheartedly agree to not write macros. But if it doesn't? Well, Lisp is ready to offer a platform to make it happen.

Lisp macros have been given a bad rap because of the plethora of almost useless examples of them. In any beginner book about Lisp, much ink will be spilt in little shorthands, like aif, with-gensyms, etc. Better examples would be actual, real life DSLs people use, like LOOP, lexical analyzers, web server routing, parallelization primitives, static typing, infix notation, etc. Things where gobs of functions and classes don't really capture the best way to denote the solution to a problem.

Learning about macros, then being advised to never use them, to me, is miscommunication at best, a grave misunderstanding at worst.


I don't think there's a strict "blub" hierarchy. Lisp excels on several axes of flexibility and keeping your OODA loop[1] small. The APL and ML families each have some cool things that Lisp doesn't do as well (though a subset of those cool things have been implemented in Lisp as libraries; see earlier point about flexibility). Once my kids are a bit older, I'd love to try out Pony because it seems to have some cool things. Of course the Blub paradox means that as a Lisp programmer, I won't always know when Lisp is Blub.

1: https://en.wikipedia.org/wiki/OODA_loop To a certain extent this is tooling, but CL, at least (I can't speak for other members of the Lisp family) was designed with a peculiar combination of dynamism and efficiency in mind that nobody else does. It's in many ways more dynamic than Python and yet easier to write an efficient compiler for, which is quite the feat.


I think these are (some of?) the mentioned ML and APL inspired libraries: https://github.com/coalton-lang/coalton https://github.com/phantomics/april


In my experience (Clojure and Clojurescript), LISP is a powerful tool for quick prototyping and experimentation as an individual hacker. The REPL-driven flow can get you far very fast. Add a couple more experienced hackers to the mix and you can build something great fast.

The downside is that it's a difficult tool to become fluid in and so it's difficult to find experienced hackers, so scaling the team is going to be slower than with other, more popular languages.


I see your point, but in my experience you can ramp up some eg: Java developer real quick; and in a couple of months they can quickly start to become more productive than with Java.

There are alot of caveats here, and it highly depends on the teams, projects, maturity, quality of tools, and so on, but there are not alot of weird concepts in clojure that makes it that hard to understand (as a counterpoint to eg: rust borrow checking rules, or C++ templating, and so on).

The hard part with clojure is having just enough discipline to keep things in check.


> scaling the team is going to be slower

On the other hand, the ROI from hiring a couple of developers with Clojure/CL experience is almost always guaranteed to be higher than hiring a bunch of "regular" programmers. Niche languages like Clojure usually attract more experienced, tired, "seen some shit" developers who have likely worked with multiple languages before. Their broad experience typically leads to higher productivity and better-quality code.


I am trying to implement my pipe dream of Blockchain and Lisp coercion, no any other PL fits my needs because Lisp is the only language where I can update the compiler every 6 minutes or so, no compilation of compiler is required.

> And if so, _which_ Lisp and why?

Are there any other production-ready Lisps except of CL?


We've been using our own lisp dialect, ellipse (https://github.com/luthersystems/elps) for several years in exactly that domain and it's been our secret weapon.


Clojure, ClojureScript, and LFE are all production-ready, primarily because they have straightforward integration with existing VM's/ecosystems (JVM, JS, and BEAM respectively).

Sadly the .Net Clojure seems to be unmaintained, and there aren't obvious leading choices for the LLVM or Python worlds (Hy exists, is usable and interoperable, but doesn't have a clear value prop, maybe macros). C world has Racket I suppose. WASM has an S-Expression representation, but that alone does not a lisp make.


Maybe also Janet? https://janet-lang.org/


Clojure?


I have not used Clojure, but it’s the singular s-expr language that addresses all of the critics.

Whatever X Clojure can not directly answer, the ever expanding universe of solutions based on the JVM can.

If the OP wants to stop painting themselves into corners, write more code. All systems offer up lots of paint and lots of corners. Only experience in those systems can guide you out of them.

I’m a CL fan, so I’d turn to CL if a CL implementation will work for you. But if you’re looking for an s-expr with no excuses, it’s Clojure on the JVM. Though pretty sure GraalVM can turn even Clojure into native code if that’s a goal.


I have a ridiculous amount of projects on production using CLJS on node, with occasional use of npm modules. I would consider this prod ready.


Scheme, Racket, Clojure.


I still wish I could work in Lisp, but have also become very happy with Julia.

Julia is not as elegant as Lisp and doesn't offer as many powerful language constructs. But it is a very cleanly designed and well thought-out language that is fun to work in. I especially love its metaprogramming abilities (heavily drawn from Lisp), which include "real" macros.

In the end, the reason I use Julia over Lisp is the community and the ecosystem. As an ecological modeller, Julia is something my colleagues have at least heard of and that I can convince a few to use (instead of R and Netlogo). I fear that I could never convince anybody to use or contribute to a software written CL. Also, the Julia standard library is just very well put together and there are more packages available for the things that I need to do, compared to CL.

I still love CL as a language and sometimes toy with the idea of going back to it, but for the moment, practicality beats purity.


Thanks to all for your varied opinion sharings and experience reports, I value and enjoyed them greatly! Might be pre-biased but I'll admit the more-"pro"-Lisp posts especially really moved the needle here for me, as they speak from a vantage that I'm eager to yet reach. So CL, here I come (and will try my utmost to do so in VSCodium rather than Emacs.. 'Alive' extension looks promising) — guess it's never too late to adopt a 65-years-old.. language =)


When it comes to the experience of Pharo/smalltalk, there are quite a few things modern languages don’t do. Live objects are a different beast.

A few years ago I had a choice to learn lisp or smalltalk. It was a coin flip but I went for smalltalk


As long as most Lisps and Schemes rely so heavily on Emacs as part of their tooling, it will never reclaim its status from its heydays. Emacs is just too clunky in 2024. Just IMHO of course. Also Common Lisp, while a great lang with fantastic implementations such as SBCL, really do need a better or standard packet manager, Quicklisp is a great effort but still in beta, and saying "well it is technically still beta, but basically ready or good for use" is simply not cutting it. Quicklisp also still, as far as I know, does not do https and the communities opinion on that seems rather "oh well" on it. I guess where I'm going at is that Lisp's tooling has some serious limitations or gaps compared to most modern languages.


> Emacs

For plugins for Vim, Atom/Pulsar, VSCode, Jetbrains, Sublime, Jupyter notebooks and more see https://lispcookbook.github.io/cl-cookbook/editor-support.ht... they have very good to good and improving support.

> Quicklisp

see https://github.com/rudolfochrist/ql-https

a new package manager: https://github.com/ocicl/ocicl

and also https://www.clpm.dev or Nix, Guix.


> Emacs is just too clunky in 2024

Believe me, you couldn't be more mistaken. Modern Emacs offers virtually unlimited ways to provide a smooth, graceful experience for so many tasks, sometimes in completely unexpected areas.

I just watched someone show me a way to integrate YouTube. It allows you to subscribe to channels and playlists, automatically fetch new videos, and watch them directly within Emacs. One feature that blew my mind is the ability to follow video transcriptions alongside the video, karaoke-style, providing an enhanced, distraction-free experience in a more interactive and engaging manner. You can not only search through the content of videos but also jump to any point. I have never seen a better way of consuming useful information from videos.


That's why I said IMHO. I tried Emacs in 2023 and 2024 and to me it is still way too clunky, gets in my way all the time and has , what I consider, bad defaults and to me very bad ergonomics. It literally negatively impacts my hands. And performance was nothing special either for me. It is however very powerful, I agree with that.


I don't know what to tell you. I myself never got into Emacs on the first try. It took me about two years of constant debate and deliberation. At that time, I was using IntelliJ and had been for seven years. I knew it extremely well and was very happy with it. Yet, seeing what’s possible with Emacs continuously fueled my curiosity. I was annoyed by my own inability to finally understand what makes it so great. Little by little, I started finding features and packages that were awesome, but I still kept stumbling on other roadblocks that ruined the whole experience. At some point, instead of blindly copy-pasting Elisp snippets, I tried to dissect and understand them and had a few aha moments. Finally, one day, I decided that instead of reaching for the tools I knew well already, for every problem at hand, I would try to figure out how to get it done with Emacs. It was very slow at first, but after a couple of weeks, I felt more productive than I ever was with other tools. A few months later, I received a notification about having to extend my IntelliJ license, and I realized that I didn’t need it anymore.

Emacs is extremely empowering, although it comes at a cost, even though technically it is free. For some people, including myself, that time is the best lifetime investment. I have never regretted the time and effort I spent learning it. Even with the plethora of features that constantly get added to VSCode, I have zero FOMO. I do occasionally fire it up just to make sure I don't get stuck in my bubble. Once you understand how advising functions work in Emacs and realize that you can redefine any behavior of any function, whether third-party or built-in, it gives you incredible authority over absolutely any textual data you have to deal with on your computer. Check this out: I can perform a search that looks for things on Google, DuckDuckGo, Wikipedia, my browser history, and other sources (this is customizable). Then, I can select any given url and send it to an LLM to extract a summary. After that, I can create a note with a deadline and schedule to review it at a specific date and time, or add this stuff to my Anki cards collection. All that without ever leaving Emacs, and with just a few keystrokes.

If you ever feel curious about it again, maybe try Doom Emacs. Who knows? Maybe this time, it will be different.


Again I agree that Emacs is powerful and can be an excellent tool. Never disputed that. But it still, again, to me is clunky. I dislike its defaults, its interface, the ergonomics. Also I have no use for many of its offerings. Could I configure it to something I find tolerable, I could but to me the return is simply not worth it. I simply would use emacs as (sourcecode) editor and for that other offerings provide me with what I want and need with only a fraction of the cost and investment needed and my hands thank me to boot. I tried Doom Emacs (and Prelude and Spacemacs) as well, it just made realize I could just use neovim if I wanted vim bindings. I tried Emacs and gave it an honest go multiple time and simply did not like it or like it enough to invest further into it. The alternatives for me simply do better job for what is important to me. And just like you I have zero Fomo with my decision.


Ah, yeah, the pace at which Neovim is developing actually makes me more envious than the VSCode world. One can use Fennel to configure it and probably get the REPL running, getting very close to how Emacs operates with Elisp. To be honest, if I didn't already know Emacs and had to choose today between Neovim and Emacs, I'm not completely sure which way I would go. I've been using Vim for a long time, and the advancements in recent years are pretty awesome. My Neovim config is minimal because I know Emacs too well at this point to try scratching the itch and build some sophisticated config, but maybe I should give it a try at some point.


I don't think neovim is as strong as emacs when it comes to extensibility. It also has no self documenting like emacs. If one is committed to the philosophy of using emacs for everything or almost everything neovim would probably leave one wanting. I do agree however it has come a long way. It is good there so many strong options these days. And I will cede one thing about emacs and vim (neovim). They most likely still be around after whatever next big editor replaces vscode.

If you do try neovim again maybe check out (in case you may have not heard of it) https://github.com/Olical/conjure It was (is) pretty nice last time I used it.


Yes, I know about Conjure as I am a relatively frequent user of Fennel. One of my projects was, at one point, the third or fourth most starred Fennel project on GitHub. Thank you for reminding me about it; I may decide to borrow some ideas from it, as I'm finding more and more cases where Lua is used. The latest revelation is that mpv can be extended with Lua, and I just found out about it. I jokingly call Fennel "the best formatter for Lua code; it makes your Lua look much nicer..." :)

Thank you once again. I tip my hat to you, my fellow vimmer, as another die-hard vimmer who, quite amusingly, happens to be on Emacs.


Thank you, too for the friendly discussion.


Because nobody else mentioned it, elisp is here to stay and a very useful tool to have around if you use Emacs. Not the one-true-Lisp for sure, but a practical way to make your everyday life more enjoyable and efficient and enough to satisfy the occasional lisp craving.


Lisp is cool, but also not everyone reasons in the way lisp operates. For some, lisp extremely "ain't it".

I've never seen lisp in production, but if I had I'm sure I'd be like, ok, we use lisp here.

Lisp is just a tool, and some people with find it ergonomic and powerful, and others will find it awkward and cumbersome. Use the tools that make you happy and productive and don't worry too much about trying to find the best in an ambiguous space.


> find it awkward and cumbersome

Without knowing what (if anything!) they don't find awkward and cumbersome, that is not very useful.

Data is very biased in this space, because the Lisp family receives a lot of drive by visits from language nomads. A good chunk of those people find programming awkward and cumbersome. They are having a hard time, and blame the tools.

Lisp is not going to fix someone who can't learn C++ or Java (or whatever) to save their life, but blogging about their disappointing experience will generate flak for Lisp.

I'm saying that if you don't find it easy to edit Lisp, you can't be "normal". You must have dyslexia, ADHD or whatever. Show me how well you can work with C, Rust, Makefiles, Shell scripts, CSS, JavaScript, Kotlin, assembly language, and whatever else before I believe that you're having a problem with Lisp and nothing else.

Lisp is not a refuge for those who can't hack it. I can see how some people might make that false interpretation, by a false association of tenuously related ideas. A language or language family that is under-used is not automatically something that is suitable for or used by the inept or the misfits. Programming language fringe and cognitive fringe are totally different categories.


Lisp is more like an idea (or set of ideas) rather than a concrete tool for specific tasks, and it's an absolutely wonderful, eye-opening idea. Unfortunately, as often happens, good ideas groundlessly get ignored. Concepts like formal methods, reactive programming, and immutable data structures also largely go unnoticed, despite being immensely useful for certain types of problems. Unlike those, the basic ideas behind Lisp are far simpler to grasp and probably have far more tangible benefits. The only practical inherent drawback of spending time learning and practicing Lisp today that I can think of is the limited job opportunities. Beyond that, there are only benefits. The common pushback against Lisp usually comes from misunderstanding it. Many programmers often get tired of using the same language after learning it cover to cover, as every language has certain cons and deficiencies. In my experience, with Lisp, it's the opposite. Having been part of multiple teams and guided many programmers on their Lisp journey, I've seen it many times - if someone doesn't like a Lisp dialect (e.g., Clojure) after using it for a day or two, they tend to start liking it more later. However, some people lack patience and quit without ever seeing what actually makes it so cool, deciding that Lisp is just a fad, awkward, and cumbersome. And that's really sad because, really, it is not. Just because someone dislikes, e.g., math, it doesn't become a fad.


Alternatively, people who try it and don't like it understand themselves sufficiently to say, "this isn't for me."

Nothing is objectively cool, it's super great that you enjoy it. It's also fine that other people don't.


I'm not trying to get people to like it; I'm trying to dispel the myths people believe. Just because I don't like running and exercise, it doesn't mean I should tell everyone there's something wrong with that. I've seen too often comments like: "Lisp is archaic and outdated," "Lisp is hard to read," "Lisp doesn't have a powerful type system," "Lisp code is not easy to refactor," "Nobody uses Lisp in production," etc. None of this is true.


To be fair some of these are subjective. People of course should never sell their opinions as objective or facts. Which btw, I don't think the person you responded to did.


> To be fair some of these are subjective

Okay, let's try to go through each of them more or less objectively, shall we?

- "Lisp is archaic and outdated"

While Lisp was introduced in the late 1950s, it has evolved considerably. Modern dialects like Common Lisp and Clojure offer extensive libraries and features that compete with contemporary languages. New Lisp dialects are constantly being added to the family. Clojure-Dart and Jank are more recent examples.

- "Lisp is hard to read"

Lisp’s syntax, based on simple and consistent S-expressions, can be highly readable once you familiarize yourself with it. The concise and uniform structure allows for straightforward parsing and manipulation of code. Try taking any JSON and convert it to EDN, or any HTML/JSX to Hiccup, and visually compare it - the latter would look far more readable. One doesn't even need to know any Clojure at all, to see that this is true.

- "Lisp doesn't have a powerful type system"

While Lisp is generally dynamically typed, Common Lisp supports powerful optional type declarations and checks. Dialects like Typed Racket offer strong, static typing. There's a new extension for CL called Coalton, and Clojure has Spec and Malli. You can do mind-blowingly awesome things with them, e.g., building a data structure for a ledger where each transaction is ensured to be in the correct order and shape — something that would require significantly higher effort to build in any other programming language.

- "Lisp code is not easy to refactor"

The homoiconic nature of Lisp, where code and data share the same structure, makes it particularly amenable to metaprogramming and automated refactoring. Tools like SLIME for CL and Cursive for Clojure can do an amazingly good job helping you refactor.

- "Nobody uses Lisp in production"

While not as mainstream as some other languages, Lisp is actively used in various domains. Companies like Grammarly use Common Lisp, and Clojure is widely adopted in the industry, particularly for data processing and back-end services. Apple built their payment system, Walmart their billing, and Cisco their entire cybersecurity platform. There are tons of examples like that.

There are many seemingly great ideas in our industry that initially received hype but later faced significant criticism — goto statements, the waterfall model, object-oriented databases, CORBA, SOAP, XML everywhere, and so on. Lisp, though, despite all the skepticism, keeps coming back again and again, and just refuses to die.

So, I stand by my words: Lisp (as an idea) is really awesome. Every programmer should gain at least some basic understanding of what it offers, because there's no harm in it — only benefits. Again, I'm not fighting anyone disliking any concrete tool, but hating the idea? Why? The only explanation I have for that - shallow understanding of what that idea is about.


You invented all those quotes and then argued against them. I said none of them. It's probably not your intent but it feels like you are coming after me angrily with these long text responses.


I'm not coming "after you." I'm sorry if you feel attacked, and I apologize if you think I'm angry. We're in a public forum, and I'm simply expressing my opinions (for everyone to see and read). They are not specifically aimed at you or in response to what you've said. This particular comment wasn't even a response to your comment; it just happened to be in the same thread.


> Lisp is more like an idea (or set of ideas) rather than a concrete tool

Indeed, there is no language that is just "Lisp" any more. There once was, of course.

There is a legend that he requested that no new language be called Lisp (and nothing else) so that there is no confusion.

SBCL, Racket, Gauche Scheme, TXR ... are examples of concrete tools for specific tasks.


> all the other languages have kept picking up nearly everything once-cutting-edge about Lisp

not all of them have CL's set of features: excellent long-live REPL, image-based development, compiling to binaries, the condition system, the interactive debugger and excellent developper tools (resumable stack frames, never re-run a long operation from zero [1]), the whole language at compile-time, stable syntax, extensible syntax, stable ecosystem, fast implementations, numeric tower, first-class interop to Java (LispWorks, ABCL), commercial support, mobile platforms delivery (LispWorks)…

New CL implementations get created (CLASP, CL on C++ with LLVM or NPT, in C) and CL compilers and libraries evolve too [2]. SBCL is now callable as a shared library, it got a SIMD module, rumors say a bear party after ELS 2024 escalated and that they are adding coroutines soon. ECL is targetting WASM, all other major implementations had new releases.

New libraries unlock new possibilities (Coalton: Haskell-like typing on top of CL), companies still pick CL today (CL is all the rage in quantum computing?),

the community creates more resources and more tools such as, last but not least, new editor plugins o/ https://lispcookbook.github.io/cl-cookbook/editor-support.ht... (VSCode, Pulsar, Jetbrains, Sublime, Jupyter… Lem of course etc)

[1]: https://www.youtube.com/watch?v=jBBS4FeY7XM

[2]: https://lisp-journey.gitlab.io/blog/these-years-in-common-li...


I have always loved Lisp but practically its not easy to hire for or find a job for. In the “normal” realm I have always found ruby to be the closest cousin. There are huge differences but the malleability of ruby is my “hackers dream” and I love using it. Most people only think of it with regards to Rails, but I use it for everything and its a delight.


> I have always loved Lisp but practically its not easy to hire for or find a job for.

No one ever got paid for using Emacs and writing Elisp (except for that single Kickstarter campaign for Magit). You can get paid to write in different languages while still hacking on Lisp dialects for your own satisfaction.

Need to configure a Lua-compatible window manager? Fennel is a great choice. Trying to scrape pages from a website? You can start a Clojurescript REPL and interactively identify elements on the page, "click" on them, "fill out the forms", and finally build a concurrent, async pipeline to sift through hundreds of pages and their content. Need to explore an API? Instead of dealing with JSON directly, you can use Babashka and, interactively, examine the results — slice, dice, group, sort, and filter through them in a way that would be extremely difficult with tools like awk, jq, and others.

Everyone may have their own distinct idea of what defines a "hacker's dream," and often developers gravitate towards tools they find comfortable, often falling into the "everything is a nail" trap. The idea of Lisp, however, with some practice, feels more like a factory for making your own hammers rather than a concrete hammer on its own.


Greenspan's tenth law is that every language ends up with a half baked Common Lisp implementation. This is apparently true, with the stipulo that these implementations end up fully baked over time. This is true since functional programming languages are commonly research languages which mean industry languages catch up.


I'd say Lisp still has a decided advantage at greenfields programming of new stuff. Especially if the syntax & internal API requirements aren't completely clear yet, then the programming language with minimum commitment to syntax has an advantage. People can experiment more cheaply and in my experience refactoring lisp code is a lot easier than more verbose or inflexible languages. Migrating from lisp to other languages also seems to be a relatively easy path to follow too (I laugh, but I'm serious - this is a useful property when prototyping).

But that isn't very powerful compared to a problem domain expert working in a language they like. If someone doesn't like lisp then they shouldn't use it and they'll probably do well.


Quite honestly, Lisp is a fairly ideal dynamic language and Ada the preferred static language for me but unfortunately the choice of a language hinges mostly on the tooling and availability of third party libraries, and that's where both are lacking. After a decade of using Racket, I'm currently using Go for development of my projects and am quite happy with it. Of course, it's a trade-off. I've always had plans to write larger projects in CL, but even with Quicklisp it still suffers from this strange syndrome where every third party library is somewhat of a hack and lacks documentation.

The ideal Lisp for me would be one with an integrated IDE that comes with "batteries included", yet is smaller and more elegant than Racket. I don't care whether cross-compiled or interpreted, it should allow me to write native GUI applications with all the bells and whistles and one-click deploy them on all major platforms, including mobile and browsers. Basically, something like what REALbasic used to be when it was still affordable shareware, but for more platforms and with Lisp as the language.


Why Ada the preferred static language?


It is extremely long-term maintainable and readable (self-documenting code), supports development in large teams, and once it compiles, the program tends to be correct. Packages work forever, which ironically irritates newcomers when they think packages are out of date. In my experience, Ada enforces clarity of thinking and YAGNI more than any other language.


I have used Common Lisp for many years at about the same time as you, as well as some other Lisps. And I also had to go away, because in the real world I was not always the one who chose the language.

About the "top end of the power spectrum", after discovering the array languages (of which for me APL is the one that clicks), I have come to believe that they are at the top end, higher than the lisps. I was disappointed when I found out that those languages existed before I was born, that I studied CS and thought I knew about all the programming paradigms, but I had still missed those.


I love lisp and On Lisp, SICP and Let over Lambda all taught me so much. But, I like high volume programming, at least a few K calls per second per core, and that means working in big orgs with other people which means code that takes a lot of focus and innate talent is not the right choice for code. Homogeneity of syntax but also of concepts and big vision are more useful than super smart code. Even when everything is going well, one needs to be able to skim code rather than think deeply about each symbol.

Caveat I have never written Lisp professionally, tho I do use function generating functions when I can. E.g. an API with 20 methods that all do the same basic thing with little variations. I will write one big function generating function that takes parameters for all the variations and then just assign the dynamic output to the expected API name (in Python). I have a go test suite where it runs the AST over a directory and extracts all the functions with a certain name and generate a list of tests then all my components run the dame test suite but calling different implementations for service calls. Or test generators where you specify parameters like expected output and input and implementation details and get an array of callable tests in nodeJS. So my code isn't trivial for outsiders to read but easier than a very DSL macrofied lisp would be.


PG is coming from the "flip this app to yahoo" perspective, not the "i will own this shit forever and have to hire a dozen outsiders to maintain it" one. People in those spaces standardized on imperative structure, classes, stronger typing, etc for good reasons. HETERO-iconicity, boilerplate, structure... helps the new hires find their bearings in the codebase. Being able to catch a type or signature mismatch at compile-time instead of run-time is another HUGE advantage when extending or re-factoring (i.e. typescript vs javascript). Aaaron Schwartz was no blub-brain, but quickly realized that pg's kool-aid was the wrong flavor for his party, so pivoted from lisp to python in the early reddit days. I occasionally come back to lisp code i wrote years ago, and it takes about twice as long to regain my bearings there than in any other language--even assembly!


> and it takes about twice as long to regain my bearings there than in any other language

Is that CL, Clojure, a std Scheme or something? I really don’t have any issues jumping in either of these 3 even after a decade or more, but maybe if something obscure is used? And I wonder about the difference in naming and commenting as well then.


A lot of it boils down to types--the difference between a labeled and an unlabeled cadaver. One has a little tag next to the vein with "blood, to heart" written on it, the other is just an empty tube. What was in the tube? Which way did it flow? I'll have to probe it with the REPL to find out. While with rust, or go, or even c, i have a type instead of another list, i can browse to its definition in a click--a lookup instead of an investigation.

The cultists are all crying, "you suck at naming! you suck at comments!" And maybe I do! But those 12 people who will have to maintain it are going to suck even more. That's where a more "blub-like" language shines. It is enforcing a standard. And the cultists cry again, "i'm so smart i don't need blub to hold my hands!" Ok, if you say so... but where is your Minecraft (blub)? where is your Excel (blub)? where is your Facebook or your Google (blub)? maybe we can just accept that lisp is nice in some places, but not so nice for others?


I'm curious if these problems are more about flexibility.

Because I find with any well designed and flexible program in a statically typed language, you end up with the same "tube of something" as you have in your dynamic language examples.

Take any large generic framework in rust, the types are extremely generic, dispatching on more generic traits. It's hardly a tag that says: "blood, to heart" unless it is highly coupled and working in one area.


> generic framework

Yet at some point, the template is instantiated--in the text!--and you can fill-in-the-blank by reading instead of having to do a runtime probe. For large systems that have a lot of preconditions, the "runtime probe" approach can be a thick fat time-waster (i.e. caressing the program into a state where something is even in the tube).


But do you really think the runtime probe is less effective in an environment where you have a good way of interacting with the program at runtime? Or is it just bad tools tainting the experience?

Because I find that in a good dynamic environment, it's easier to figure out how to use something complex when compared to trying to uncover how and why the types line up.

The absolute extreme being something like smalltalk, where you just guess how it should be used and then fix it in the debugger until it works.


> But do you really think the runtime probe is less effective in an environment where you have a good way of interacting with the program at runtime?

No matter the tool, it will be very difficult to try out all code paths to figure out what can be in that tube. The difficulty scales with the size / age of the application.

Take your typical 20 years old enterprise application. Nobody from the original team is around. Whole generations of programmers worked on the project, each with their own favorite patterns / code style / level of diligence. There's a massive amount of features, half of which you don't even know about. Test coverage is spotty at best. You see some common function triggered from hundreds of places, it makes a huge difference in your ability to reason about it if you have types on the data being processed or not.


So you mean to say; it takes 2x as long to get into not statically typed lang code? Asm? I mean sure it’s typed but…

And we are pro static types, it’s just easier to add them after I am done experimenting. Not upfront. And we do via various (backward compat) custom CL constructs and provers.


types are just an example. structure (modules/classes/keywords/constructs) is also a big win for organization and assisted code navigation. even with dynamic types, i find it far easier to grep a python codebase than a lisp one. is it code? is it data? python syntax makes it fairly clear.


Fair enough; can’t say I have this experience but that’s fine; I am not religious. Although I would say, unless you are doing a lot of things with macros (hint; you shouldn’t and where you do, it should be clearly indicated; Ruby suffers from this as well when reading code imho) otherwise it seems very strange the code/data barrier isn’t immediately clear. But maybe I don’t hang as much on syntax as others do (we our own k3 to lisp to CUDA for some things and I don’t find it different/more difficult to read).


> HETERO-iconicity, boilerplate, structure... helps the new hires find their bearings in the codebase.

But only in theory. The shit that I keep seeing of incomprehensible spaghetti code in C and C++ disproves that theory on a daily basis.


Yup. I've seen programmers whose first language was Clojure. Later, when they tried to get into Java or Typescript, their first reaction was almost always, "WTF is all this shit? How do people even look at this stuff all day long? How do you make sense here, why everything feels so needlessly tangled?"


Languages enforcing a questionable structure or boatloads of boilerplate are one thing, but it's also that writing comprehensible code is a skill in and of itself. If you don't have that (yet), no language on earth is going to save you.


True. Writing beautiful code that is easy to reason about is a skill. Just like writing prose, it comes with experience. One can try multiple programming languages and still suck at it; It is not language-dependent art. I've seen horrible code in languages I know well, and I've seen beautiful code in PLs I have no idea how to write idiomatic structures in.


How did python help, I always just assumed it was chosen to allow the hiring of more programmers, not that it provides any technical benefit above lisp.


The man is dead, yet his words live:

http://www.aaronsw.com/weblog/rewritingreddit

"The others knew Lisp (they wrote their whole site in it) and they knew Python (they rewrote their whole site in it) and yet they decided liked Python better for this project. The Python version had less code that ran faster and was far easier to read and maintain."


Spez seems to contradict some of the claims made in that article (https://web.archive.org/web/20160803061607/http://www.reddit...).

for example he dosent make any claim about the python implementation being "far easier to read and maintain" (even by proxy by claiming that CL was hard to read/maintain), and honestly starts out by absolutely singing CL's praises

his reasoning seems to boil more down to the ecosystem surrounding Common Lisps libraries being poorer than pythons:

> If Lisp is so great, why did we stop using it? One of the biggest issues was the lack of widely used and tested libraries. Sure, there is a CL library for basically any task, but there is rarely more than one, and often the libraries are not widely used or well documented. Since we're building a site largely by standing on the shoulders of others, this made things a little tougher. There just aren't as many shoulders on which to stand.

[etc...]

> So why Python?

We were already familiar with Python. It's fast, development in Python is fast, and the code is clear. In most cases, the Lisp code translated very easily into Python. Lots of people have written web applications in Python, and there's plenty of code from which to learn. It's been fun so far, so we'll see where it takes us.


Something was bugging me about this article the first time I read it, so I made a mind map.

Turns out, not once does the author actually give a reason why Python was better than Lisp for Reddit. The only references to that connection at all were claims that some “X” (all among the set that are oft-refuted by Lisp programmers) wasn’t the reason!

It reads more like a PR justification (“oh, there’s not much difference really, we even asked unspecified engineers who agreed with the change, and our code is working better and better!”) than an actual explanation.


"less code that ran faster" seems fairly clear-cut to me. Why would he lie about that?


As I understand his claims, “less code that ran faster” was after switching from existing web frameworks to web.py, a from-scratch web framework he wrote himself to ‘do the right thing, simply’ (as all the best frameworks do).

The decision to write web.py instead of web.lisp was not elaborated on, however.

As mentioned above, the closest point I see to him discussing that decision is saying that ‘Python uses objects to make frameworks somewhat like the syntactic constructs Lisp allows you to make’, ie ‘you can do the same things as Lisp less easily in Python, so that’s not an advantage of Lisp’. Which is both incoherent as a refutation of the referenced reason to keep Lisp, and not a reason for the change to Python.


Read it again:

"THE PYTHON VERSION had less code that ran faster and was far easier to read and maintain."


That's actually really direct. It appears as though they had python rewrite had more familiarity with the programmers so they had an easier time writing it.

I come from the opposite side and find python syntax and style stifling.


Instead of an ordinary blub like python, they should have doubled-down, went full-retard, and used PHP. For web work in those days, it probably would have been even faster... even easier to read and maintain.


Common Lisp's string support certainly feels like it comes from a world where allocating memory is expensive, and something the programmer should be aware of, and have the option to reuse buffers instead of allocating additional memory...


> Common Lisp's string support certainly feels like it comes from a world where allocating memory is expensive, and something the programmer should be aware of, and have the option to reuse buffers instead of allocating additional memory...

We are still in that world. I do game programming, and am convinced that if a game programmer isn't aware of their memory allocations, then their game will drop frames. This is true today, even at 60hz (and things are moving towards 144hz, 240hz, and beyond). Reusing buffers is essential.

I think that being aware of when and where your memory comes from is the foundation of performance. The discipline of handling memory carefully tends to naturally lead toward making more performant decisions on everything else.


Yeah, I won't argue that there aren't some domains where you have soft real-time requirements, and need the language to allow you to do things like that. Reddit's app server isn't that, though.

Separately, my gut feeling is that in 2024, programming language runtime technology is advanced enough that manually reusing buffers isn't necessarily a good use of hours-spent-optimizing. To pick on Python in particular, it _does_ actually reuse buffers when they only have a single owner; more languages might get this in the future via the Perceus refcounting work?


Based comment and it's sad that common sense gets pushed to the bottom of the thread. You are dead on.


If you only learned Common Lisp from skimming PG's On Lisp, it's forgivable that you never really learned it. PG doesn't like OOP, but like, approximately all the big Lisp systems I've seen make use of it, and it helps with understanding quite a bit. Same thing with using more namespaces (defpackage) instead of throwing everything into cl-user. One reason I prefer CL to Clojure or Scheme is indeed the compile-time warnings about things like type mismatches. It's a trivial problem (it didn't really bother me when I did lots of Python) but it's nice to not worry about it. Similarly, I can do a lot of code navigation with vim, I don't need to learn emacs though it can do the same navigation, as can vs code or whatever. If I'm coming back to a piece of code, or investigating something new for the first time, it's rather easy to get my bearings, more so than a C program. I hacked my PDF viewer (atril) recently to show estimated time left while auto-scrolling a doc, it took me longer than I bet it would have if it was written in Lisp.

Typing being optional is not a problem. Like, if I'm looking at some function, and for some reason it's not clear what type of argument it's expecting, with one command I can pull up a cross-reference of other code that calls that function, and jump to it, and jump to the definition of the argument. If it's an object, I can jump to the definition of the class. This isn't really any more burdensome than jumping to the class in the first step, especially because in static langs I'll occasionally need to look at uses in context via cross-references anyway.

I don't get your dismissal of runtime probes. In large static systems, runtime probing is necessary for me to orient myself, the types don't really help. "How does the execution thread even get here?" is easily answered by sticking a break point there and running the thing, then you can inspect the stack. Common Lisp also has a very nifty "trace" function built-in, with a keyboard stroke I now get output every time the function is called along with what its arguments were and what its return values were. I can do this for any function, even built-ins, no instrumentation setup necessary or modifying the code or stopping/restarting the program. The full interactivity also means I can just jump to the definition even in some library-of-a-library code and if it pleases me rewrite it to add some prints or other info or just fix a bug and move that redefinition to my project. It's much easier for me to investigate a large Lisp system than a large Java system even though I've probably had more experience with the latter. Even grepping stuff is rather simple because of naming conventions.


Not a dismissal of runtime probes, but a statement that, sometimes, they are not the best tool for the job. I like the REPL, but I'd rather not have to use it just to find out what X is.


That’s not a matter of the language; and I don’t mean that in the “No True Scotsman” way either.

Lisp definitely allows you to write code that is very easy to come back to and understand. I’ve written code like that, and I’ve seen code like that in eg certain sections of the SBCL internals, among many other projects.

However, there’s a reason we don’t write all our code in Python or Assembly. The fundamental semantic restrictions of one and the simple functionality of the other are both traded off against horrible scaling relative to a technical project’s complexity and scope.

If you write a project with complex stages/components in those kinds of languages (or mimic their styles in Lisp code, for that matter), either you have to split those components into parts that don’t make sense separately (making the code base significantly more complicated than eg an equivalent Lisp codebase), or you have to write extremely long and convoluted programs to accomplish conceptually simple functions (making the code base significantly more complicated than eg an equivalent Lisp codebase).

Similarly, if you use the Python or Assembly coding paradigms for large projects, you quickly get lost in a sea of individually simple-seeming code segments, unable to grok the connections between said segments just because of how many things need to be kept track of.

In contrast, for Lisp languages:

A) The default style used by Lisp programmers is highly functional, using many higher-order functions, using macros that solve their problem-domains without explicitly exposing the code doing the solving (objects also fall into this general role, btw, which is apropos given the Common Lisp Object System was defined purely with macros), and clearly delineating code / systems with mutating effects.

While this style is admittedly harder to grok than Python/Assembly for simple cases, it has almost no reduction in reading-complexity as code scale/complexity increases. Complexity gets naturally encapsulated and automated by functions/macros/objects (as opposed to either a restricted subset of those or simply the developer’s own head, which are the 2 approaches most other languages offer for complexity management) (algebraic type systems are a very cool exception to the prior note, but a pervasive object system (like CLOS) paired with macros allows you to make a type system (like Coalton) without feature/ergonomics loss).

B) Interactive development and ergonomic DSL creation are objective wins for complex or large programs; the former significantly reduces iteration time for both cases, while the latter makes it easy to just write code once that addresses the complexity/scale of the domain and then never deal with that complexity again (or at least not with the full complexity, even if you don’t understand the domain well enough to obviate the irrelevant complexity entirely in your DSL).

Sure, hot-reloading in a few other languages gives us most of the former (although things like CL conditions, debugging REPL-layers, access to object contents & stack-frame inputs (only addressed by some debuggers, and only in a restricted way), and some other features still lack equivalents) and other languages generally use objects alone to address (most instances, though not all, of) the latter (rather than Lisp’s mix of macros/objects/higher-order-functions) (we’re ignoring the text-templating macros in languages like C++ because everyone agrees they’re more trouble than they’re worth). But Lisp is still by far the most featureful and ergonomic offering of both of these facilities.


That's only because True Lisp hasn't been tried yet </s>


Julia is very lispy (and was originally designed on femtolisp), which gives it a lot of nice features including macros. Julia's macros aren't quite as flexible as Common Lisp, but they're way beyond e.g. Python.

We have used these macros to save a lot of time and money (multiple developer salaries). One example is by implementing a DSL for numeric code that allowed researchers to write in Python-like syntax, but would get transformed into a DAG for longer-running jobs that could be evaluated much more efficiently + in parallel.

We could not make something nearly as performant in Python – we spent hundreds of developer hours trying, because we already had a lot of working code and were trying to make it faster/cheaper to run. Nobody wants to rewrite a large, working codebase. Yes, we tried numba, pypy, custom C extensions etc. – in the end a Julia prototype written in one week significantly outperformed all of them. Three years later we still use Julia for all our numeric code.


Although I love LISP and it will always have a special place in my heart, there is power in the assistance the compiler for languages with powerful type systems like Rust.

Writing in LISP can benefit from, I believe, a kind of psychological stance of wabi-sabi (non-perfectionism), for (barring exotic LISPs with sophisticated type systems), it will always be more challenging to refactor and get things just right than with languages like Rust.

Given the pressure of writing code for firms that can demand we are perfect or pay the price [being on call, etc.] (ie working for perfectionistic firms with perfectionistic customers — it’s a recursive problem) also can push us towards languages with powerful compilers.

Then again, maybe we need to invert things — use the languages we love that are not “perfect” but are so easy to love; for perhaps that love can just as recursively emanate outwards as would-be perfectionism…


Seen this yet? https://github.com/coalton-lang/coalton


Lisp's type system is fine.

    (defstruct adult (name 'error :type string) (age 'error :type (integer 18 200)))

    * (make-adult :name "Bob" :age 3)

    debugger invoked on a TYPE-ERROR @5529ACEE in thread
    #<THREAD tid=426288 "main thread" RUNNING {1003F000A3}>:
      The value
        3
      is not of type
        (INTEGER 18 200)
      when setting slot AGE of structure ADULT


    * (make-adult :age 18)

    debugger invoked on a TYPE-ERROR @5529ACF2 in thread
    #<THREAD tid=426288 "main thread" RUNNING {1003F000A3}>:
      The value
        ERROR
      is not of type
        STRING
      when setting slot NAME of structure ADULT


Note that this behavior is completely implementation specific and totally undefined in the Common Lisp standard.


What is standard CL way to enforce these type conditions (apart from declaim)? Some spec library similar to the one Clojure has?


`declaim` is just as implementation-dependant as the :type specifiers for defstruct, thus by lispm's rules, there are no 'standard' ways to apply type conditions.

But then, the standard /is/ useless.


The standard leaves it to the implementations/implementors to decide what to do. An implementation can check all these type constraints. It may not do it, for example, if code gets compiled with the opimization quality SAFETY of 0 (and possibly SPEED = 3) or similar. It may not do it all.

There are implementations which check these type constraints. So it is best to check the manual or ask the maintainers. IIRC there were implementations which added it on user requests.

But don't think all "Common Lisp" implementations will check those constraints, for example ABCL does not. Some don't and in others it can be turned off.


What makes you think that Lisps don't have powerful type systems as well? For many years significant research has been conducted on type systems specifically in Racket. In Clojure, you can do interesting things, such as defining complex data and function specifications like "this function should always return prime numbers", which generally speaking is difficult to achieve in many other languages.


specs are not types. Your Clojure compiler will have a hard time statically checking specs at compile time.


Yes, Clojure.specs are technically not types, they cannot enforce constraints broadly at compile-time. Yet they still can enforce finer-grained constraints at runtime. From the practical point of view, specs are really great tool for building robust software and have other properties like simplifying property-based testing through automatic generation of test cases.

Take for example a trivial example when we need to validate user data with the following constraints:

- Age: Must be between 18 and 120.

- Name: Must be a non-empty string.

- Email: Must follow a valid email format.

- Credit Card Number: Must pass the Luhn algorithm check.

Something like that is pretty straightforward to achieve in Clojure.spec within 20 lines of code. It's close to impossible to get with any advanced static type system, because type systems can't deal well with custom constraints; they often can't offer detailed, human-readable error messages for the cases like that; they often can't describe complex and nested data structures, or trying to do so would be cumbersome and less expressive.

So, they are not types. Okay. Cisco built their cybersecurity platform using Clojure. The market capitalization is about $180B. I imagine this would be a few dozen billions more if they had chosen to use an "actual type system", amiright?

----

edit: After thinking about it a bit more, I have to admit that I spoke too harshly. It's not "close to impossible" You probably can create a type that checks for these constraints, but you would likely need to use a dependent type system or refined types, perhaps something like LiquidHaskell., and it still would be very tricky, especially with the credit card part. In Clojure.spec - it's very trivial.


There is nothing which would require me to use Clojure to check those things.

For example... one can easily express all that in Common Lisp:

https://www.lispworks.com/documentation/HyperSpec/Body/04_b....

    CL-USER 9 > (defun non-empty-string-p (string) (and (stringp string) (plusp (length string))))
    NON-EMPTY-STRING-P

    CL-USER 10 > (deftype name-type () `(and string (satisfies non-empty-string-p)))
    NAME-TYPE

    CL-USER 11 > (deftype age-type () `(integer 18 120))
    AGE-TYPE

    CL-USER 12 > (defun foo (age name)
                   (declare (type age-type  age)
                            (type name-type name))
                   (print (list age name)))
    FOO
also, avoiding SATISFIES:

    CL-USER 4 > (deftype name-type () `(and string (not (string 0))))
    NAME-TYPE

    CL-USER 5 > (typep "hello" 'name-type)
    T

    CL-USER 6 > (typep "" 'name-type)
    NIL
It then has assertions integrated with the condition system, which makes this easily checkable at runtime.

SBCL has this extended with type declarations as assertions: http://sbcl.org/manual/index.html#Handling-of-Types

Eiffel had promoted Design by Contract for a long time.

> Cisco built their cybersecurity platform using Clojure.

Cisco uses a lot of software. Your Clojure marketing attempts are getting silly.


> So, what's your hot take

Unless you already have a rare Lisp job, or you're retired, spend your time on things more employable.

> Perusing the various search results on 'CL vs Scheme / Racket' had me personally more convinced of CL for heavy-duty "real-world",

CL has more wins on this, but Scheme has, on at least one occasion, been used for large, important, rock-solid stuff, and super-productively. It comes down to the individual programmer(s).

> all the other languages have kept picking up nearly everything once-cutting-edge about Lisp

Much of the relative value of Lisps nowadays is the communities. They're generally made up of people who would be doing this even if software weren't a well-paying job. And there's relatively little BS to wade through (like explodes in all directions around popular employable platforms), since most BS follows the money.


> It comes down to the individual programmer(s).

I think you hit the nail here. People credit lisp because it is, or was, the language of the super high end geeky dev. But it's the high end dev self-selecting into lisp, not the language itself, that is the real advantage IMO.

Find a really good developer and let them use whatever they like. I've known the proverbial 10x (or even more!) dev in a number of languages, even "blub" ones like C#. A motivated, highly competent developer with an eye on the vision and skin in the game is the real "secret weapon" no matter which tool she happens to choose.


> Find a really good developer and let them use whatever they like.

Unfortunately, it's rare that a company can identify a really good developer.

Also, trusting a developer to choose the programming language for a project might not be a good idea right now, due to a pervasive industry culture of resume-driven-development and job-hopping. (Unless "really good developer" implies conscientious professionalism and alignment with business needs, and you're able to discern that.)

> A motivated, highly competent developer with an eye on the vision and skin in the game is the real "secret weapon"

Agreed. Though founders tend to think they deserve 20x+ the equity of even first hire engineers. To the point that even your typical early hire is probably best off freshening up their resume keywords (via resume-driven-development), and job-hopping as soon as a better compensation package comes along (maybe after vesting cliff). The founder already signaled that they think the developer is a commodity, not a secret weapon of the company. That's not motivation for the hire to be aligned with company success, much less beyond.


Yup, but when you have to integrate with things you can't control, the advantages are fewer, especially compared to something purpose-built for that thing and the trade-offs can easily turn away from Lisp. e.g. you're going to have a much easier time making a game in plain Godot than you would trying to mix CL with Godot's native API, or if you're crazy doing an entire 3D engine + game in CL. Even in that last approach, integrating with the graphics stack on modern systems is going to be nuts. Going further into OS-facing stuff like sound, rendering text, handling multiple languages, handling inputs from mice/keyboards/controllers, dealing with multiple monitors, microphones... it's a good thing Common Lisp has a decent FFI (it's even possible, though more difficult, to integrate with pure C++ stuff using SBCL and not having to switch to Clasp, but it's nice that Clasp exists in the same way it's nice ABCL exists to more easily integrate with Java stuff) because you will be using native libraries for some things, and it will suck to some degree because those are written in the world and mindset of dead, static programs. But the pure Lisp side will be great, and you'd be using those libraries or something like them anyway if you switched Lisp out.

An underrated Hickey talk is his Effective Programs one: https://github.com/matthiasn/talk-transcripts/blob/master/Hi... It's another entry in my personal (and useless time-wasting) fascination with wondering what could have been if CL won in the 90s. No need for Java to come around and "get C++ programmers half-way to CL", with excruciatingly slow followups year after year that finally by Java 8 and after got us something (when also using additional things like a giant IDE with good refactoring tools and simple code generation (no manually writing getters/setters or managing a giant list of imports), Spring, JRebel, and sometimes AOP frameworks) actually not too unpleasing compared to a CL experience. But most things were always there in the CL experience in the 90s, especially if you could shell out for the commercial systems, it really is just Java playing catch-up. And then you get in that talk and elsewhere a story about how non-technical decision making didn't even give Lisp systems a chance in production, so as a last resort Lisp was put on top of the JVM which was accepted. Clojure is a great achievement, and ABCL came too late, but it's kind of a shame either has to exist. Still, the thousands upon thousands of man-years have been spent, and because of that Lisp's advantages are fewer.

However the Lisp world itself hasn't been static since the 90s either, and so the counter-advantages of vastly larger ecosystems have grown fewer as well. For example, it took until as late as 2010, but finally quicklisp was made and the problem of libraries and finding/getting them was dramatically lessened. For comparison, Perl's CPAN is from the 90s, Python's PyPI is from 2003, Maven 2004... Lisp was late. But the last 10 years of Lisp seem to be going a lot better than the 10 before, and while there are still problems, they don't really look like uniquely Lisp problems, and supposedly unique ones like supposedly not being able to do infix arithmetic are addressed with slow education to the contrary (while I'm here might as well link for the nth time, the solution available since 1993: https://github.com/quil-lang/cmu-infix). It was such education years ago from a HN comment that made me give CL another look and then never look back at Clojure/Scheme/etc. (though Java is still my preferred breadwinner should I find myself suddenly needing more income). (The comment was simple, just showcasing that yes in fact CL has typing support, the compiler is available at runtime, and with SBCL the compiler can report not just about wrong types / typos but dead code elimination and failures to apply optimizations. Cool stuff. Made me realize I never knew CL at all. Even CLOS, the OOP system, had some pleasant surprises when I finally got around to learning it and seeing it's far more than what other langs offer.) Meanwhile the advantages are still there.


I see a lot of myself in you, both proclivities, journey and the question.

I don't think lisps strengths can be matched by piecemeal feature adoption. The GC, the REPL, the ability to reload code in a running process, the macro system, the minimal syntax - it _all_ comes together and it shapes your development process.

Case Study: My finest work was written in Clojure, I'm pretty sure I wouldn't have a clue how to achieve the same in Go, Rust or whatever else the broad programmer base is typically interested in.

The single most challenging project I've ever worked in, domain-wise, had so much churn in the spec because we laid the track as we went. We changed data storage products (Postgres -> Datomic), we frequently re-wrote the UI (back when ReactJS was popular and Redux etc were being proposed). We re-architected mid-way to use event sourcing as we needed to be able to recreate the system state at historical points in time. And so on. A lot of dysfunction as well. But Clojure was probably the biggest reason we managed to limp along. I can't imagine having to r-earchitect so furiously while pleasing a borrow-checker which tends to make refactoring much more extensive.

Recommendation: If you want to give it an honest shake, but you're afraid (as I read it) of painting yourself into a corner, consider Clojure. Clojure is a fringe language, like all Lisps, but rides on top of two massive ecosystems: the JVM (Clojure) and Node (ClojureJS). Additionally, if Emacs/(Neo)vim is not your thing, there's an IntelliJ plugin, Cursive, and it all works _beautifully_.

Additionally, the community, and its designer, have many interesting takes on software design which I found very educational.

Where I think Lisp shines ?

* When things are _complex_ or projects drift, revise and change often. The ability to be functional, to build custom DSLs, border-line custom interpreters with way less effort pays off.

* If I had to teach programming to _complete_ newcomers, gun to my head

* When valuing _genuinely_ different perspectives. Picking up Lisp offers more than yet another C-like language

* When you have limited mental capacity for more stuff, i.e. you have a family life and a complex, demanding job already - maybe pick something that won't drag you through borrow-checker nonsense or category theory.

Observation, why programming in the large tend to suck: Why normal languages kind of suck. --

Essentially, enterprise code tends to require a lot of lines, those lines can have bugs. People want to reuse code (write less code, really), so they start writing abstractions, leveraging the many different (flawed) tools their language provides. These are typically a _lot_ harder than Lisps function composition and macros (say, Rust macros, Python metaclasses+inheritance), this means complexity goes through the roof. Over time requirements shift and abstractions become less suited, now you're fighting on two fronts. I would say you either need Lisps ability to write terse code (macros being the unique feature), or you need a _good_ code generation tool. And we seem to have neither in most projects.

Conversely: where is Lisp less useful ? --

The well-paved roads where a popular language has a set of finely developed libraries for solving your issues. But once your domain itself, your business logic, gets complex, there won't be a NPM, PyPI, or crates.io to save you.


Based on the software people actually use and the languages they are written in, probably not.


When you take the subway in London or Lisboa, you're traveling thanks to Common Lisp.

When you buy a plane ticket on Kayak or Orbitz, you're traveling thanks to Common Lisp.

fun fact: there is a Common Lisp NFS Server inside the Boeings 747 and 777.

https://github.com/azzamsa/awesome-lisp-companies/ (nothing official)

a new one: when people in Norway drive under tunnels, they do it safely thanks to Common Lisp.


stylewarning on reddit https://www.reddit.com/r/lisp/comments/1d8jmuo/ask_hn_30y_af...

---

[Disclaimer: Below is mostly unsubstantiated, unscientific, personal, and perhaps even somewhat incoherent opinion.]

This might sound jaded, but the software industry and its incentives are far more perverse than they were 30, even 10 years ago. By and large, writing new, efficient, and/or interesting programs is not valued culturally among software engineers as a trade group. The number of software engineers "on the market" is at record highs, and the vast majority of them speedran any% a college degree or bootcamp and are looking for their entry-level six-figure Senior Software Engineer role straight away. The mere suggestion of taking a step back to look at language offerings, like Lisp and the paradigms it enables, is not helpful toward that pursuit. And even if someone does take that step back to take in what has been available for 7 decades, the average employer, too, won't care whatsoever. Lisp might provide a plethora of benefits, but the incentive structure of modern day industrial software engineering has all but eliminated the realization of those benefits in practice.

Maybe there's some comparison to the coal/oil industry vs nuclear to be made. If you can go out in a limb and agree that nuclear is better in almost all ways, why do we continue to depend on and bolster the coal and oil industry? Why is the market for coal/oil bigger? Why do we keep hearing news articles about this and that nuclear plant getting shuttered? At this point, it's almost nothing to do with the actual intrinsic pros and cons of each energy source, but rather the culture, politics, and infrastructure in place to support each one.

Lisp was born out of an idea that we can think about programs symbolically, and the Lisp of today carries that ethos in many ways. Even ignoring general debate about functional vs OO vs whatever, the idea that programs ought to first and foremost express ideas declaratively—and that we should have macros to let the programmer carry that torch into their domain of expertise—just hasn't caught on. In contrast, mechanical, line-by-line thinking still dominates every popular general purpose language (and I argue, will continue to).

If your goal is to write a program that does something new and reliably, then Lisp is still ahead in my opinion. It's still, to me, the most efficient programmer's programming language (similar to how jazz might be called the musician's music) that is accessible and supported today. It's still an excellent choice for building many kinds of products.

If you want a job in which you'll move slop around the internet superhighway, you will find one faster by writing Python.


Js is the blurb now.


lisp, json, xml, unicode, ai love spiral


Lisp is not for everyone and based on what you wrote, it seems to me that it's clearly not for you. So I would suggest you stop wasting your time with a language that's obviously not a good fit.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: