Only tangentially related, but for anyone interested in the idea of a simple, quick Python-like scripting Lisp, there are two Clojure-style languages to look at:
1) Hy (https://hylang.org/, compiles to Python bytecode, usually slower than Python but compatible with all Python libraries)
2) Janet (https://janet-lang.org/, very light Lua-style embeddable VM ~1 Mb, roughly twice as fast as Python for similar ops, very easy C interop)
since we're at it, why not a batteries-included Common Lisp: https://github.com/ciel-lang/ciel it comes as a binary that starts fast and that includes libraries for mundane tasks.
Ciel is exactly what I wanted out of a "scripting language" that runs atop Common Lisp. Really glad that you mentioned it. The built-in SLY/SLIME support makes me instantly want to ditch Babashka for this
This looks so cool. For CL newbies wanting to try and grok the value of a SLIME and REPL driven approach to development, are there any good beginner articles?
an actually difficult question! Different persons will absorb different things from articles, and will enjoy different projects as a first encounter. Pointers:
I'm guessing what might really be happening is that functional code can be slow in Python. And Hy tends to encourage functional code where vanilla Python mostly discourages it.
That might look like a whim, but I always found Lisp to be a rebutal language due to it’s mandatory enclosing brackets.
As M-expressions avoid this pitfall, I wonder if any actually language either implemented the homoiconicity and conceptual elegance of Lisp without requiring these encompassing parentheses.
It does, though I would also point out that decorators also cover a lot of the same practical use cases. And is probably the preferable solution if your goal is to benefit Python code in general. Hy macros can only be used from Hy code, so it's not really an "improve my Python codebase" play so much as it is a "I want a lisp that can leverage the existing Python ecosystem" play. Kind of like Clojure vis-a-vis Java.
Yep. I also got really excited thinking this is homoiconic python I might be able to make my coworkers use (without having them buy into lisp). But no, it's just lisp.
You may want to take a look at Rhombus, that is a somewhat-python-like language with macros that is written in Racket.
> Rhombus is an experimental, general-purpose programming language with conventional expression syntax that is built on Racket and that is macro-extensible in the same way as Racket.
Looks like it. I was curious to witness this homoiconic Python but I'm not sure it's even possible. For that to be true, Python syntax would have to be made out of tuples, lists or dictionaries, and the interpreter would have to evaluate those directly.
I thought the point of homoiconicity was essentially runtime reflection and evaluation of programs constructed at runtime, i.e. code is a first class datatype in the language.
I don't think these properties imply much about the underlying implementation of the language.
Another functional language that can be concisely implemented [1] in Python is Binary Lambda Calculus, with a large part of the code dealing with BLC's pure I/O model. Instead of using an association list for variable lookup, it uses de-Bruijn indices to index the environment array. The same page shows implementations in 9 other languages, with BLC's self interpreter being the most concise at 232 bits (29 bytes), which includes a parser and tokenizer.
MIT's fundamentals of programming course requires all its students to write a Lisp interpreter in Python. It's a throwback to when the class actually was in Lisp.
I find it funny and also ironic that a lot of modern languages re-discover Lisp's amazing features decades later. The other day I was infuriated to see my Python program halt after 9 hours of API calling to my own home server. The API calls were similar (calling an LLM with a pre-defined prompt template, constrained using grammars). I wanted to save the state of the program and exit it before running the remaining iterations separately. So I needed a way to modify the current running Python code and inspect the variables. I couldn't figure out a way to "dig into" the running Python process and save the state and/or fix it. So I lost 9 hours worth of work.
The fact that CL had this feature built into the language decades ago just blows my mind. Not to mention numerous other features (like the most powerful macro system there is), etc.
> I find it funny and also ironic that a lot of modern languages re-discover Lisp's amazing features decades later.
This is never surprising as Lispy languages are some of the most flexible and expressive languages. But that's the problem, it is too expressive. The Lisps always reminded me of mixed media (visual) art, where the freedom of expression from the mixed media sounds good on the face of it but in the end it generally produces sub-par works compared to more traditional single media art. It turns out that the restrictions of the medium are just as important as the expressiveness.
I've seen shitty code running in 'prod' at every employment I've had as a developer. The language isn't that important, the people and the organisation are.
It's not important from an external standpoint, looking at code quality. Like in all the arts, most of it will be bad. That is not the point. The point is from the developers perspective as a medium. To put it another way, you can produce bad art in any medium but some mediums seem more conducive to creating good art.
Maybe. You probably don't want to base your web development studio on colorForth for quite a few reasons but I'm not so sure artistic reasons are relatively important.
You seem to think that organisational factors, like management, profitability, market influences and whatnot are less important than the conduciveness of the tooling to create good art. Why is this?
IMO they are on different dimensions, orthogonal if you will. The medium will often be dictated by outside demands, but when talking about mediums themselves and the different aspects of that medium can take place without taking those into account. Just like you could discuss the differences between oil paints and acrylics without taking into account any specific work... of course if you flip that, talking acrylic or oil for a specific work, then it would matter.
Language is very important. Bad code in python is at least still python. Bad code in lisp can be an inpenetrable eldritch abomination where the only option is to rewrite from scratch. I believe this is actually the primary reason google build go. You can't write good code. But you also can't write really bad code.
Why do you believe it impossible for Python to be impenetrable?
A Common Lisp will likely be much more introspectable and debuggable than your average CPython.
I think Google designed Golang to be a small language for implementing network services that will tie newcomers to programming hard to that specific language and then also Google. Hence the syntactic quirks and the err spectre. That it enables development of CLI applications was likely an unexpected bonus.
@cess11 on Golang, no, rather Ken Thompson and colleagues designed a language stripped off things they find as an unnecessary cruft and they could finally get back to the fun they had before Java and OO.
Yeah that's ok for you to think like that, at the same time it's fun for the creators and for a number of users, including me. Btw there so many languages with sum types (and almost none that fixed data races) that everybody can choose.
The point is, I was aiming to refute one cummenter's claim that Go was created to lure specific people into Google as devs. Highly unlikely, Go's adoption in Google wasn't bad but also wasn't exactly a bed of roses.
It's not impossible. But rather much less likely. And even less likely to do by accident. Lisp by itself basically encourages programming deviating from standard practices. That the entire reason it's so powerful. It takes a real genius in addition to strong restraint to keep writing good code in that environment.
He means he once read an article written by a web developer with no Lisp development experience called "The Lisp Curse", and that's his entire Lisp development experience.
Right, yeah, seems likely. Such comparisons are usually pretty weird, putting non-hygienic macros against ordinary programming in some other language, rather than macros against the metaprogramming used there.
Experience with macros and their footguns arguably prepared me pretty well for using JAXB and reflection professionally.
Lisp Macros are encouraged. In most other language you can't even write syntactical extensions. Such things are, generally, horrendous for readability. Like I said you can wield them effectively. But the average developer won't be able to.
Google Lisp style guide: Use macros when appropriate, which is often. Define macros when appropriate, which is seldom.
Heinrich Taube's "Lisp Style Tips for the Beginner": Beware of macros. They are a very important feature of Lisp but a poorly implemented macro can cause bugs that are very difficult for a beginner to solve. Avoid writing macros until you understand how the Lisp reader and evaluator work. Never write a macro to make things "more efficient".
Guy Steele and Kent Pitman, "Tutorial on Good Lisp Programming Style" [1993]: Decide if a macro is really necessary. [...] Don't use a macro where a function would suffice.*
Carnegie-Mellon Lisp FAQ: Never use a macro instead of a function for efficiency reasons. [...] Don't define a macro where a function definition will work just as well -- remember, you can FUNCALL or MAPCAR a function but not a macro.
I've never seen a book, tutorial or style guide document saying that Lisp coders should be writing lots of macros, and constantly looking for any excuse to write another macro or some such advice.
> In most other language you can't even write syntactical extensions.
C? C++? Rust? Erlang? Scala? Julia? R?
> Such things are, generally, horrendous for readability. Like I said you can wield them effectively. But the average developer won't be able to.
Macros actually are widely used to improve readability. In Lisp that's one of its main purposes. It leads to more compact, declarative, domain specific code. Large programs usually benefit the most. Programs can be much short and more readable.
Take for example in Common Lisp the Common Lisp Object System. It has three layers: an object-layer at the bottom, then a functional layer and on the top is the macro layer. The macro layer is what the typical developer uses, which is compact and which shields the developer from the details of the lower layers.
The Common Lisp Object System blends seamlessly into the rest of the language and the usual basics can be easily learned.
If you disagree that Lisp has a tendency to become unreadable then maybe the typical developer might have a "neurodevelopmental deficit" compared to you. bindings don't change the language that's executing. That's the entire point why Macros can quickly make code unreadable.
Macros do not change the language that is executing (as in self-modifying code).
Self-modifying code is possible to express by other means in traditional Lisps that have a quote operator.
In Common Lisp, it is undefined behavior to modify a quoted part of the program.
E.g. this wold be undefined:
(defun counter () (inc (car '(1))))
The function is referring to a piece of its own syntax (1) which initially holds the integer 1 and incrementing that. This may have the expected effect under some circumstances. Or it could have the expected effect, plus some unexpected effect, or not have the expected effect at all. It may signal an error, also.
I happen to agree with your underlying point, but expressing it in the form of a personal attack is the worst thing you could do here. It destroys everything that we're trying for on HN, and also reinforces the worst clichés about Lisp on a social level.
Do you think this is a good faith way to communicate with other people? No where in my comments did I attack anyone or accuse people of lying. Yet you suddenly come out of the gate swinging. Why?
"I believe this is actually the primary reason google build go." Well the actual quiet part is that they built Go to reduce costs of paying those highly trained software engineers and replace them with moderately trained monkeys.
Actually it was because some engenieers take pride in creating complex solutions for problems that a moderately trained monkey could solve in a managed language.
I don't know how that would have helped. Most languages (including Python, AFAIK) unwind the stack when an exception is thrown, all the way up to the nearest catch. Whatever you wanted to do at the place of throwage is no longer possible because that stack doesn't exist other than as an incomplete printout.
In contrast, Common Lisps condition system keeps all stack between throw and catch so you can resume from anywhere in between, possibly with altered code or local variables.
Basically, Common Lisps exception stack traces are interactive by default.
For that to happen, you must have implemented what that SO answer says in your code. In contrast, CL allows you to modify the current running code and apply fixes.
If there is an exception a matching error handler will be selected and called in the context of the error. A default handler could be a debugger, a break loop (which is a Lisp REPL), something which asks the user waht to do or it could be a handler provided by the software, which could do anything, including looking for restarts and using one.
For example the variable a is unbound and we try to add 3.
CL-USER 37 > (+ 3 a)
CL shows an UNBOUND-VARIABLE error and provides Restarts. Restart 3 is a USE-VALUE restart.
Error: The variable A is unbound.
1 (continue) Try evaluating A again.
2 Return the value of :A instead.
3 Specify a value to use this time instead of evaluating A.
4 Specify a value to set A to.
5 (abort) Return to top loop level 0.
Type :b for backtrace or :c <option number> to proceed.
Type :bug-form "<subject>" for a bug report template or :? for other options.
We are now in a break loop, a REPL one level deeper, in the context of the error. Let's see what the condition (-> exception) is:
we call one restart (listed above, number 3) interactively, it uses the new value and continues to compute the expression. It asks for a value. I enter 5 -> 5 + 3 = 8
CL-USER 39 : 1 > :c 3
Enter a form to be evaluated: 5
8
We can do it also programmatically. HANDLER-BIND establishes a handler for UNBOUND-VARIABLE. It invokes the restart USE-VALUE with 4 -> 3 + 4 = 7
In the CL condition system they have separated error detection (exception thrown) from handler selection (catching exception) and handler implementation (what are called "restarts".)
So basically instead of registering one handler (catch block) that is responsible both for determining the proper course of action and implementing it, the condition system allows you to register multiple "restarting points" which the handler can select from when deciding how to handle the error.
One of the restarts often used in debugging is restarting any of the functions in the call chain, i.e. just trying the same thing again. But just as common is varying some parameter or local variable and then restarting the function.
The process hasn’t crashed after an exception. It still might even exit with a success error code, if you really want it to.
An exception is raised when you attempt to parse HTML as JSON. The process doesn’t end. You catch the exception, see what happened, and respond with a 400.
Ironically that stack unwinding is exactly why I like a straight old sigsegv.
Just dump the core and analyse the frozen state. One thing that annoyed me in go is that even when it core dumps on panic and after I worked on the backtrace analyser for gdb, most of the core dumps were useless as everything of worth was already unwound and lost.
Hidden in plain sight is the fact is that structured exception handling in 32-bit Windows (the linked-list-on-the-stack version[1,2], not saying anything about the gigantic-tables “zero-cost” version) is a resumable exception system a la Common Lisp conditions—inevitably, seeing as it subsumes SIGSEGV coredumps!
> Most languages (including Python, AFAIK) unwind the stack when an exception is thrown, all the way up to the nearest catch.
You might have to use a few tricks/hacks, but I'm pretty sure you can set up python in a way to breakpoint at an exception, including all local state at the throw site. Otherwise pdb would not work.
Can the program be recovered once the exception is thrown and you start debugging? Or is the keeping of the stack the last allowed state before exiting?
Does anyone know of any lisps that have a type system that tames some of lisp's tendency to get unreadable as programs evolve? Or not a type system but some other aspect of it to tame it?
All the metaprogramming is really cool, but sometimes it reads like the gnarliest abstract haskell you've ever seen, but without even the type signatures to guide you. Type systems and linters are the best tools I know for automatically taming code, but I can't really imagine how you would really keep lisp projects' tendencies in check without hamstringing all the reasons you'd reach for lisp.
A Lisp with a type system? Sounds like Common Lisp to me. SBCL (I'd say as close to a de facto standard as it gets) even has pretty good compile time type checking. E.g.
(defun foo (a b)
(declare (type String a b))
(+ a b))
; in: DEFUN FOO
; (+ A B)
;
; caught WARNING:
; Derived type of COMMON-LISP-USER::A is
; (VALUES STRING &OPTIONAL),
; conflicting with its asserted type
; NUMBER.
; See also:
; The SBCL Manual, Node "Handling of Types"
;
; compilation unit finished
; caught 1 WARNING condition
The type checking is the main thing that drove me to switch from Scheme to CL in the first place, and I stuck around for the nice little things like restarts and continuable asserts.
An expression oriented Python would simply not be Python. That's fine but "expressions as second-class syntax" is pretty fundamental and a deliberate choice to make developers who try to write paren soup have a bad time. Hence why lambda: and := are deliberately clunky to discourage their use and you can't write anonymous functions.
Python says if you want to be clever be clever with iterators not callables.
The idea is you can build up an AST with a DSL and then either generate Java code that gets fed to Javac or execute the code with a tree-walking interpreter. One thing I discovered was that there is quote and eval so an interpreter has an extended type system over Java because it is possible to construct an Expression<T> (always immutable) and then eval it.
The plan was to use ferocity0 as a library to support the stub generator, then use the generated stubs to write ferocity1 in ferocity0 and possibly a ferocity2 in ferocity1. The idea is that the code would be awkward to write without templating because you have to repeat a lot of things for 8 primitive types so I want to write as much of ferocity as I can in ferocity. (My analysis is that some recent features that involve type inference such as switch expressions would run into problems w/ type erasure that I don't know how to solve but it ought to be possible to implement a "complete" set of operations and apply syntactic sugar of various types on the ferocity side.)
I also made a functional programming library which is greatly simplified compared to streams which would really benefit from code generation, for instance
could be extended with higher-arity methods than it has but I am not writing them by hand so if I ever finish ferocity I plan to write a new version of pidove.
I did a couple of projects where I'd pick up a broken Scala system and fix it, it was usually "screw around for a week in Scala and it (1) still has race conditions and doesn't give the same answer every time, (2) has totally busted error handling (they seemed to think saying the word 'monad' meant you didn't have to think about error handling) and (3) used at most 2 CPUs out of 8."
Rewriting it in Java with ExecutorService would solve all those problems in like... 20 minutes.
But yeah, I think a lot of people would think Ferocity combines all the awful things about Java (verbosity) and Common Lisp (the "write once never edit" problem of metaprogramming)
That's funny, because I've been taking servers full of large blocking thread pools, rewriting them to be non-blocking, idiomatic Scala, Cats Effect 3 and get +50% throughout.
Blocked threads are seriously bad for performance. Yes, Java 21 project Loom will help with that (finally), if people get with the program.
There aren't any race conditions, because our classes and data model are fully immutable.
The problem with our legacy servers was that the original devs didn't know what monads were, and decided not to use them. Error handling is now great, by using...
I ended up using tuples instead of lists for cosmetic purposes :-) My Fakelisp then looks like this:
from fakelisp import *
# And now you can start mixing Python and LISP
X = (BEGIN
(SET (F) (LAMBDA (X)
(IF (EQ (X) (1))
(1)
(MUL (X) (F (SUB (X) (1)))))))
(LIST (F (4)) (42)))
# Back to Python any time
print "x: ", str(X)
1) Hy (https://hylang.org/, compiles to Python bytecode, usually slower than Python but compatible with all Python libraries)
2) Janet (https://janet-lang.org/, very light Lua-style embeddable VM ~1 Mb, roughly twice as fast as Python for similar ops, very easy C interop)