I think you might be missing the main "aha!" of clojure REPLs vs REPLs in non-homoiconic languages: you can very easily execute small parts of the program you're editing without re-typing the code.
In emacs, for instance, you often use `eval-last-sexp`, by default bound to C-x C-e. This lets you move your cursor to a particular point in the file, often deep in a function, and get the results of just the form(s) you're pointing at.
This is a superpower! It lets you test small pieces of your code without writing any test harnesses or scaffolding. Try it with an editor that embeds the repl and has these commands, and you'll never want to develop any other way.
It does cause you to want to structure your code in 'repl-friendly' ways, so that you can, say, restart your main server or reset your state without restarting the whole process.
Homoiconicity isn't required for that, all you need is to be able to map between source and AST enough to identify expression boundaries.
Plenty of development environments for non-homoiconic languages provide the ability to evaluate a selected expression (not just the last one) in a linked REPL on demand.
I love the Lisp family, but I don't know why its advocates often sound like they haven't seen a dev environment for a non-Lisp language since the early 1980s when pointing to “unique” advantages of Lisps.
Homoiconicity is an ill-defined concept. If you take every subjective feature from it, all it implies is that you are able to cut your code into well defined AST structures (what all languages have, but for some it's easier than for others).
So, yeah, anything people do with Lisp could be done for any other language, it only requires more complex tooling. How much more complexity depending on the language, and it varies from "barely perceptibly more" into "that's completely not practical".
To me, "only" and "complex" shouldn't be used in the same sentence. Big part of the job of a developer is to reduce complexity, and we barely do it (depending of the system). What is "barely perceptible" now can be "completely not practical" 2 years down the road.
Maybe it's not required. However, I can confidently say Clojure in emacs feels like an entirely different development experience than my daily Python work in PyCharm/VSCode and C# in VS. I recommend trying it out if you haven't already.
MAL. AKA Make A Lisp. Instructions to write a tail recursion lisp interpreter in a weekend in almost any language. It teaches almost anything you want to know about lisp, plus some practical compiler/interpreter knowledge.
There is a significant, qualitative difference between using a language that designed for REPL use and one that isn’t.
Those boundaries you talk of are the crux of the issue. A highly dynamic, completely expression based language is going to enable a much different experience. Homoiconicity also plays an important role here, because you can ispect and parse code within the language, with the same functions and algorithms as everything else.
I am not familiar with how Julia and Elixir work, but form my experience Python’s REPL is not the same at all as REPL in Clojure or other lisps.
In Clojure, you literally built your program in memory while you’re writing the source code file, updating the memory representation of your running program part-by-part without ever stopping it.
This experience is absolutely magical, and it’s very hard to go back to edit-compile-run loop in other languages after that. It is completely different from “my language has a REPL and I can execute parts of code in it”.
No, Julia and Elixir are the same as Python in that regard (Python is more annoying because of the indentation problem). I was just talking about interactive evaluation of expressions selected in the editor. Although I’ve messed with Clojure a bit, I admit I don’t fully appreciate what you’re describing. I’m a little afraid to find out, frankly, because it sounds seductive.
Yes, that restart is painful exactly because it interrupts the "built your program in memory while you’re writing the source code file" OP talks about.
my inkling is that the more special forms the more difficult it is to repl a language. if everything is an expression not only can you eval parts of it, but that is enough to support quite a bit of extensibility sans macro.
a comment here some time ago explained nicely what most of these "unique" advantages of homoicinicity are actually about; it was essentially another step on the same spectrum of statement-languages and expression-languages.
in particular this does not mean that it is better but the same advantages of being able to put anything into anything that you find in expression languages like rust in lispy languages are also syntactically in editing source code.
since I started using shrink/grow selection in vscode I have much increased interest for simple syntax aware editing, something simple would be moving blocks of code token by token
The homoiconicity of Lisp is not what gives it this superpower. It is rather Lisp's dynamic nature that actually allows you to keep running your project while also developing it that is crucial - something that Clojure running on the JVM can actually only approximate.
For example, in a full CL, you can modify a class that is currently being used, and objects of that class will adjust accordingly (for most modifications). This is not possible for the JVM - to modify a class, you would have to unload it, which can only happen if the Class object is garbage collected, which requires all instances of the class to be GCd, and the ClassLoader that loaded that class as well - only then could you add a field with a default value to the class.
Does it? Asking for a value when encountering an unbound variable is a default restart
$ sbcl
This is SBCL 2.1.1.52.HEAD.321-f8a57bcca, an
implementation of ANSI Common Lisp.
More information about SBCL is available at
<http://www.sbcl.org/>.
SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses. See the CREDITS and COPYING files in the
distribution for more information.
\* (\* x x)
debugger invoked on a UNBOUND-VARIABLE in thread
#<THREAD "main thread" RUNNING {1001860103}>:
The variable X is unbound.
Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.
restarts (invokable by number or by possibly-abbreviated name):
0: [CONTINUE ] Retry using X.
1: [USE-VALUE ] Use specified value.
2: [STORE-VALUE] Set specified value and use it.
3: [ABORT ] Exit debugger, returning to top level.
(SB-INT:SIMPLE-EVAL-IN-LEXENV X #<NULL-LEXENV>)
I suspect you are right.
Rebinding an inner function inside another function that contains variables probably will not work.
But you could still alter and rebind the entire thing.
And that is still flexible; you can still change a running program on the fly with only that ability.
And you can always reference objects at the top level.
Smalltalk famously let you code inside the debugger.
You can run code with methods that don't exist yet and add them in the debugger when it notices they are missing.
You can use a REPL powered debugger. You can interact with the debugger to a higher degree than with most mainstream languages. It’s not quite as powerful as CL though, from what I’ve read.
Or you pull out expressions and test them in isolation with given (assumed) inputs.
What if you don't know what you want it to do yet?
Maybe you have a new API or library you're working with and the documentation is lacking.
Maybe you're exploring some dataset trying to figure out what kind of information can be gleaned from it.
Maybe it takes 3 minutes to get it into the state where it failed, so instead of starting over every change you just want to modify the one function over and over.
Doing it in a repl like jupyter has you run the actual code many times as you're writing it so ensure things don't break before you ever save the file. Of course thats not an excuse to skip tests, but if I was going to pick one, I would choose running the code as I write it, instead of testing it after the fact.
A. Then don't submit the exploratory test in your next PR.
B. Then memoize the expensive function to disk. Bonus: Can keep multiple versions on disk, thus can tweak the expensive function while having fast reloads with guaranteed correct version.
So that you can run your unit test from some unknown state that may not even be reachable from the current text of the code, rather than running it from the start in a reproducible way? Why would that be something you want?
> In emacs, for instance, you often use `eval-last-sexp`, by default bound to C-x C-e. This lets you move your cursor to a particular point in the file, often deep in a function, and get the results of just the form(s) you're pointing at.
but that depends on how the environment is looking at the time.
during development, in scheme, i just have a function (reset) that reloads all the files of an app. sometimes i create a thread and poll the filesystem and reload everything that changed in the last second. that way i never have to C-x anything. if i need to eval a subexp i just jump on the repl and do it.
Python can hot-reload an application from scratch, too. The idea here is to not reload everything from scratch, but to redefine functions incrementally/iteratively as you work.
if i change anything in filters.scm then i (load "filters.scm") instead of just sending a snippet to the interpreter to eval. i have this happen when i save the file. same thing, except i never have to worry about sending code or a mis-match between editor and repl. it also forces you to be more organized with code so that you don't override global state. i just code, save, code, save. you can have hooks run things after every save etc ...
Well, a UNIX REPL (read sh, bash, fish, etc) is pretty much "homoiconic", since everything is just a text stream. Of course the value of ARGV[0] will be some executable, written and possibly compiled, but that's no different than any other language which needs to eventually make it to hardware instructions.
You can do this in IDEA's IDEs too, just place a breakpoint and then you can "evaluate expression" in that context to call functions, read values, etc.
Kotlin and IntelliJ IDEA really are a joy to work with in my opinion, they're very much my primary tools at the moment. I'd never really been exposed to functional programming features until I got into Kotlin and now I'm learning Scheme of all things, it's been a bit of a gateway drug for me.
The thing that bugs me about the IDEs is how slow they are. I'm super productive in them, but I really don't feel like they need to be so slow. I'm not even talking about the indexing which I get.
In fact they weren't always unresponsive, seems Jetbrain's products have regressed in the last year, but mostly Webstorm. I use WS on OSX and Win10, and on both I get all kinds of slowdowns - different projects too. Tried disabling features, plugins, different GC, increased heaps. I may have to fire up J. Mission control at this point.
IDEA and Clion on Win10 are mostly fine.
It would just be cool to have all of these features, but the frontend be a responsive CLI.
This post could be summarized as "write as much of your project's tooling as you can in the project's main programming language and the project's main programming language should be Clojure" and I agree wholeheartedly.
I think their point is bigger than that. Historically one of the major points against using Clojure to write your tooling was the slow startup times which are just painful from the CLI. It looks like the clj-exec idea linked to in the article is the secret sauce that makes moving to writing your tooling in Clojure a workable idea, since now you have a unified calling convention for both calling tooling from the REPL during development but also activating your tooling from the CLI in some CI or build pipeline where the massive Clojure startup times don't matter.
This article comes at a perfect time, we're just starting a new Clojure project and were looking into how to automate tooling since we'd been burned by Clojure startup times before. Looks like clj-exec means we can now unify our work on Clojure.
Also check out https://babashka.org: it offers Clojure scripting with very fast startup time. It also has a task runner (similar to make, just, etc.) that can be used to store long invocations (like clj-exec tends to have).
Hopefully there will be some way to just "reload" your whole `deps.edn` though
"If you get an error, the execution stops by default and you get a stack trace."
I'd say the other missing piece of REPL development is that while you get a stack trace, you don't get a program state like you do with GDB or ELisp. Maybe I'm "holding it wrong" but this causes a lot of friction and lost time. I'd be curious how others approach this. And that all being said, the CLI doesn't relaly offer a better alternative here.
For entirely replacing the CLI I think that since Clojure is a general purpose language there ends up being a tad more boiler plate than you'd like.. It's not at the point where you're gunna just run `clj`, load in some library with `add-lib` and start messing around b/c things are just a tad too clunky.
For instance if you wanna read in a CSV file (I had to look this up)
Uhh.. so you're prolly gunna want to wrap that up in a helper function. I personally end up making a dummy "project" where I keep a bunch of helper functions and then doing my REPL "scripting" and messing around in that. It feels a bit wrong.. but at least to me it looks like a solvable limitation. Given a nice set of helper libraries you could probably get to a point where a bare `clj` REPL would be as ergonomic as a more explicitely interactive language like R/MATLAB/etc.
CIDER ships with a perfectly adequate debugger if you want it. Might require a bit more manual labour than some environments but you’re never stuck just staring at a stack trace if that’s what’s bothering you.
I think using external dev dependencies and rich dev/user.clj files is pretty common on Clojure projects. Your REPL isn’t just somewhere to interact with the current codebase directly, it’s a framework for building that software and managing its environment more generally.
I need to play with the debugger again.. Does it give you the state along with a crash stack? I remember that unfortunately the debugger couldn't be turned on globally. You need to instrument particular functions. And this turned out to be a major inconvenience in my usecase (a GUI app)
Yeah you’re right, that’s what I mean by manual labour, there’s no ability last I checked to break on any exceptions. But in any case of a reproducible issue you’re not stuck, is all I mean.
> Maybe I'm "holding it wrong" but this causes a lot of friction and lost time.
It is a completely non-problem for functional code. It is a big problem for imperative code.
You can't just write all of your code in a functional style, but depending on what you are doing the limit gets larger or smaller. So it's normal that this will be a showstopper for some people, and irrelevant to others.
I don't really follow.. How is it a non problem in functional code? For instance you have some recursive idempotent function that blows at some point. Wouldn't you wanna see the state at which things broke?
Just because things are functional doesn't mean you always knows the inputs at all times
> Don’t forget the set -euo pipefail at the beginning of your script. What does -euo means? I don’t know, I copy-pasted it and man set said there is no manual entry for set.
Use "man bash" to find the list of built-in commands. Scroll way down, or search for "SHELL BUILTIN COMMANDS".
The "e" command-line switch to the set command tells the script to exit immediately if there is a non-zero return value. The "u" switch tells the shell to treat unset variables as errors when performing parameter expansion. The "o" switch enables the following option (in this case "pipefail"). "pipefail" tells the script to return the value of the rightmost command that returned with a non-zero value in a pipeline. This is all paraphrased, the details are in the man page.
Off-topic, but Bash's manpage is awful and a great demonstration of the limitation of manpages.
Not that the content is bad (it's great!), but as OP (and you) demonstrated, finding relevant information is close to impossible in the humongous document.
In Bash's defense, its manpage is a concession to people's habits, and its documentation really rests in its Info page. Except that doesn't seem to be popular either (I'll admit to being still inexperienced with its UI, after decades of sparse usage)...
I always forget about the info command, even though it's mentioned in most man pages. So I typed "info bash", and went looking for the "set" command with my almost null knowledge of the info command. It took me a while, but I eventually found it's description under indexes/built-in index/set. It was surprisingly more difficult to find than just looking in the man page, but did take me much less time since I didn't need to skim the whole document to find it. Typing "info info" was actually really helpful, though. Info has a lot more power than I knew about. Typing "i set" while in info brought me right to the set command description. Guess I should learn the tools a bit better.
Contrary to typical GNU practice, the bash info manual states that it is intended as a “ brief introduction” and that the man page is the definitive reference on shell behavior:
Perl (and Git and TCL) seem to have found a middle ground, where they broke up their manpages into different ones that refer to each other (via "SEE ALSO" section at the bottom). But it's sad that, in 2021, we still have such a poor terminal documentation system, and the improvement invented 40 years ago (Info) gained no steam...
"man set" can't provide that information trivially without knowing your shell; dash/bash/ksh/zsh will all be subtly different.
However, if you are running bash, "help set" is fine.
FWIW, I really don't like pipefail as a default (e.g. piping to "head" causes random pipefails) and -e also has some confusing semantics. Also either failglob or nullglob are more important than either and -C is useful for some scripts as well.
[edit]
Simple example of confusing "set -e" semantics:
The following does not print "hi" and shows returns error status, as expected:
(set -e; echo hi); echo $?
But put it in an if statement, and it's suddenly success, and does print hi:
if (set -e; false; echo hi); then echo hello; fi
The same problem applies to functions. The below function will remove all files in the current directory if it's called from a conditional, but not otherwise!
foo() {
set -e
cd /some_directory # set -e means we exit if this fails
rm -rf *
}
I think just defining a die() function and using it after any command that must succeed is more verbose, but less error prone:
> I think just defining a die() function and using it after any command that must succeed is more verbose, but less error prone:
Yep. It's the style I developed 12 years ago, when working at Bazaarvoice, when I was lead of devops team. I created the whole library for bash, to use this pattern consistently. See https://github.com/vlisivka/bash-modules#error-handling
Considering its ecosystem, what do most here use Clojure for or what do you think is its sweet spot? I used it in a project with pg/ring/reitit and while some things were nicer it was not better enough to make me switch from pg/nodejs/express for new projects. Used emacs with inf-clojure (found cider too bloated and buggy, was tempted to port some features of cider to a fork of inf-clojure) and macros were helpful in a couple of places, but other than that, you can write JS almost as how most use Clojure. Also found JS to be faster unless you got out of your way to write Javaish Clojure, which reminds that is was a time sink having to deal with Java libs(it's really not as "seamless" as most advertise) because of no Clojure equivalents.
Nowadays I use Clojure as my primary general purpose programming language.
As far as sweet spots go, due to pervasive immutability, I think it can be a good choice anytime you're dealing with concurrency.
If you're dealing with relatively straightforward web applications, outside certain specialized scenarios (and certain really bad choices) I don't think language choice matters.
Clojure's sweet spot is very much the "situated program" Hickey likes to talk about. Clojure is a great fit for line of business apps where maps really are the right choice for modeling your domain.
We recently evaluated Clojure vs node for an upcoming project and went with Clojure because while most of the work really is just querying a db and writing JSON, we also do a fair amount of heavy reporting and pdf generation which will bring node to a crawl relatively speaking. Rather than have to take on the operational complexity of microservices for the slower parts of our app we just went with a Clojure monolith. The draw of a single language to work with for both client and server was very tempting, but ultimately the JVM won out on the server.
I'd be amiss if I didn't also say the whole Deno situation has us worried about the long-term implications of spinning up a brand new node project, where the JVM seems far more reliable logistically.
As far as ecosystem, Clojure itself is smaller, but the whole JVM ecosystem is huge and you have access to that.
We're not at a point where we've figured out how we're doing report generation yet. Internally we do not have extensive experience with either Clojure or Node, but we have heard issues from colleagues with Node and running into performance issues by trying to do too much on the main thread. Since our load demands are not that high, and we're migrating from Rails so we already expect a very nice performance bump, rejection of Node is more a choice of avoiding potential downfalls rather than a fully informed decision based on experience with Node.
Some Clojure frameworks and libs do provide REPL support for things that we would otherwise expect to be CLIs, such as running migrations, (re-)starting services and so on.
The startup time criticism is valid, but in context of development you typically don’t restart it except you pull in deps (very rare) or something terrible happens (rare).
Then, there is also borkdude/babashka which is a Clojure powered scripting tool with fast startup times due to GraalVM.
> I use add-lib branch of tools.deps.alpha that allows me to add dependencies dynamically at the REPL and then start using them immediately, just like in the shell.
You could do this with (battle-tested) pomegranate [1] ages ago, which is used by leiningen as the default resolver.
In emacs, for instance, you often use `eval-last-sexp`, by default bound to C-x C-e. This lets you move your cursor to a particular point in the file, often deep in a function, and get the results of just the form(s) you're pointing at.
This is a superpower! It lets you test small pieces of your code without writing any test harnesses or scaffolding. Try it with an editor that embeds the repl and has these commands, and you'll never want to develop any other way.
It does cause you to want to structure your code in 'repl-friendly' ways, so that you can, say, restart your main server or reset your state without restarting the whole process.