Hacker News new | past | comments | ask | show | jobs | submit login
A History of Clojure [pdf] (clojure.org)
627 points by puredanger 32 days ago | hide | past | favorite | 374 comments

Prior to Clojure I feel that I didn't really know how to do things well.

In the context of the domain I have the most experience with, web applications, I'd summarize the bulk of programming as defining data, data transformations and shoving data through APIs. Once you get the hang of it, Clojure makes it trivial to transform datastructures from A to B. This has enormously expanded the kinds of problems I can solve.

When Rich says "It was impossible to avoid the sinking feeling that I had been “doing it wrong” by using C++/Java/C# my whole career."

I feel somehow that this is the case for the majority of people but they don't realize it yet because their experience is the most popular trendy language or framework. I've seen many examples of libraries in different languages having enormous amounts of commits and hundreds issues for problems that are trivial to solve in Clojure.

I was in the same boat, constantly grabbing for frameworks and if one wasn't available that made my task simple, would struggle trying to bend the frameworks or language I was using to come up with a solution.

I'm not a language geek and I don't write code outside my work for fun. I want to spend the least amount of time possible in front of the computer and sleep knowing my web applications won't topple over. Clojure has fit me so well that I don't think I would have accomplished what I have in other languages.

Could you elaborate on a problem that illustrates this property of Clojure? This sounds awesome, but I have a hard time understanding what you're getting at without knowledge of Clojure.

Three examples of this:

1. core.async: So the guys who built golang did it because they had this cool idea for creating coroutines using channels. However, since it required very low-level functionality to be part of the core of the programming language, they thought they'd have to design a brand new language to implement their idea. However, shortly after golang was released, Rich and some other clojure folks implemented the same feature in clojure as a totally ordinary external library, proving that the core of clojure was general enough to support this. And it wasn't just a gimmick: I use core.async every day and think it is better than golang's implementation.

2. The Expression Problem: One of the core challenges in language design is designing it so that (simplifying a bit) you can transparently add support for new function methods to objects designed by a third party. Clojure makes this easy https://www.infoq.com/presentations/Clojure-Expression-Probl...

3. Lots of languages have attempted to allow you to write a single library that you can use both on the client and the server, having your code transpiled to different languages in each case. However, this type of feature is rarely used in production, because there are usually lots of headaches involved, with many limitations. However, for clojure developers it is perfectly normal (and usually expected) that all code is written in cljc files & so that it can be used on both the client (transpiled to javascript) and the server (transpiled to java). It is easy to do this, even for cooperatively multithreaded code.

I use core.async a lot too and really like it, however, I do feel that it suffers from being a macro instead of part of the language, for example, you cannot nest core.async functions inside other functions as they must be directly visible by the go macro, which cannot see jnside function calls. I’ve also had errors where the stack trace did not reference my code files at all, because it happened inside some core.async setup (IIRC it was a variable that was meant to be a channel but was nil, inside a sub or mix or something, ie I connected two core.async things together, it created go blocks internally, the exception happened inside that and therefore never referenced any if my source files, since the exceotion happened in an asynchronous context after my code ran).This was extremely painful to figure out.

Neither of these issues can be solved as long as core.async is implemented as a macro. However, it is extremely cool how far it was able to be taken without changing the language!

> I use core.async a lot too and really like it, however, I do feel that it suffers from being a macro instead of part of the language,

Agreed. The core abstraction of a go block is essentially a user-space thread. Because macros are limited to locally analyzing/rewriting code (and limitations of the JVM), go blocks are super limited in scope. As you point out, one call to a function in a go block, and you can't switch out of that go block during the function's execution. And functions calls are very common in Clojure code and can easily be hidden behind macros that obscure the details of the control flow.

There are 'core.async'y ways around all this, but the net effect is that 'core.async' imposes a very distinct style of writing (and one that tends to be contagious). Python's 'async/await' is contagious also, but because it's more tightly integrated into the runtime/compiler, it doesn't feel nearly as restrictive. (at least to me).

I don't think it makes logical sense for there to be a "core.async function". Instead, you have each function return a channel with it's own go macro, then have the parent function consume data from the channels.

Sure, that’s a solution, but its necessary only because of a limitation that only exists because its a macro that needs to look into the code to rewrite it. It also adds extra (cognitive) overhead that such functions are always themselves asynchronous and the return value are channels - they can never just return a raw value - which limits what you can do with them or how you can call them or pass them elsewhere.

I mean, yes, usually it isn’t a problem at all, but it IS a limitation of core.async that wouldn’t need to exist if it had tighter integration into the language.

> and think it is better than golang's implementation.

How is core.async better than the golang implementation. In golang, all network I/O you do is automatically asynchronous, so your goroutines won't block a thread on network I/O. That is simply something core.async can't do.

I don't know if it does, but why couldn't it? The Java has non blocking IO.

It's not just a matter of non-blocking IO. core.async uses heavy-weight threads. Go-lang uses light-weight go-routines.

The JVM also would need to incorporate a lightweight process scheduler that can wait on I/O and schedule j-routines when I/O is blocked.

`core.async` does M:N scheduling where many go-blocks are scheduled across a smaller number of heavy-weight threads. `core.async` also has channels and select. It pretty much works the same as in go.

However, since the default in JVM is heavy threads and blocking behaviour, extra care needs to be taken to avoid blocking a thread that a go-block is executing on (as that would also block many other go-blocks).

So it's definitely easier in Go, since you don't need to juggle these two contexts.

JVM has NIO, which provides non-blocking IO. You can use this with core.aysnc's channels to resume a go-block. You have all the pieces you need to write a go-scale clojure service.

Sorry, I completely disagree. I don't consider mere M:N thread work distribution as anywhere close to what the go scheduler offers.

This article mentions some of the fundamental problems with core.async: http://danboykis.com/posts/things-i-wish-i-knew-about-core-a...

Having to pay ridiculously careful attention to what code blocks across libraries and function boundaries is something that golang completely takes away the need for - because it has a true cooperating scheduler built into the runtime at the boundary of every syscall.

Decision making is in hands of the Go runtime. You can write in standard dumb sync fashion at 3am in the night within a go-routine making all the Go library calls you wish to make without needing to scan code with a microscope to see what blocks and what doesn't.


core.async is a half-baked error prone solution. People will make mistakes with it. Until the JVM has native support for fibers, continuations and a compile mechanism to classify all legacy blocking calls as unsafe, core.async will continue to be hobbled.

M:N thread distribution is exactly what the go _scheduler_ offers. It's a work stealing scheduler like Java's ForkJoinPool.

The benefit lies entirely in that there's no other way to do concurrency. Which is great, as it's less error prone. But if you were to introduce native threads in Go you'd have the exact same problems as you'd have with core.async. So it's not like core.async (or async/await in other languages for that matter) is half-baked or badly implemented, but it won't automatically remove a part of the Java runtime that has existed since the beginning.

In a similar vain, Rust's actix and Java's Akka are not half-baked or bad implementations of the actor system, but it is more error-prone to use compared to languages built entirely around the actor paradigm, like Erlang and Pony.

So no, core.async doesn't magically give you non-blocking concurrency, but it does enable it. And that is a much better option than re-writing your entire codebase in Go.

> core.async is a half-baked error prone solution. People will make mistakes with it. Until the JVM has native support for fibers, continuations and a compile mechanism to classify all legacy blocking calls as unsafe, core.async will continue to be hobbled.

With Loom it won't be necessary to add a compile mechanism to classify legacy blocking calls, as there won't be any. This is why Loom is taking so long, the entire runtime is being re-written to not block if IO is performed in a virtual thread, but work as before if a full thread is being used.

And when you can simply create virtual threads instead of native ones, there's really not much value in core.async anymore (well, ClojureScript will still benefit) as Clojure already has great primitives for working with threads.

I don't know anything about the JVM and my experience with JVM-based languages is incredibly limited, but isn't that the essence of the Loom Project?



Yes, however, the Loom project is still some years away and the Java standard library will need to be extensively revamped to support this. I see this easily a decade in the future.

They have been rewamping the Java standard library slowly and silently over the last couple of years in anticipation for Loom. With Java 14 they re-wrote the tcp socket class. In Java 15 they're re-implementing the udp socket class. These re-implementations are being done to support loom seamlessly in a future update.

They also collaborated with IntelliJ to find bugs related to debugging Java apps, to make sure Loom doesn't break anything there.

Loom is well underway. I wouldn't be surprised if we saw Loom in Java 19 or 20. So in 2-3 years.

Hell, I just used loom last night on some pet projects. You can use it today if you dare.

The next LTS version of Java is planneed to to be Java 21 (fall 2021, numbers a coincidence) and my presumption is they are trying to get it into that.

> So the guys who built golang did it because they had this cool idea for creating coroutines using channels.

Communicating sequential processes is an "old" idea. See: https://en.wikipedia.org/wiki/Communicating_sequential_proce...

That is true, but my impression is that the original syntax was cumbersome and that golang made major advancements on that front (though I'm no expert and haven't studied the original papers)

I'm no expert myself and wasn't trying to call you out or anything. (I've only read that paper once or twice before a Papers We Love event I attended.) I think it's a common misconception, though, and thought people might find the history interesting.

This stack overflow answer has a very fascinating explanation of the history of the evolution of CSP from Hoare's original paper, to the work done on it with Occam up onto Go, I recommend reading it if you are interested in this stuff: https://stackoverflow.com/a/32696464/172272

2. requires static typing in my personal opinion

cf. http://lambda-the-ultimate.org/node/4136#comment-62959

Spec is pretty close to being a library for static typing; Clojure core being dynamically typed isn't really a blocker.

If you really want static typing Clojure can probably do it. The community support isn't going to be as solid as for something like Haskell. But I'm pretty sure spec offers stronger control of the useful parts of a type system than something like, eg, C. Except the control over RAM; that isn't a Clojure competence.

> So, the expression problem is the problem of solving another, unnamed problem, while satisfying the constraints of static typing? It seems a not very useful term then, and implies that every problem will need two names - the name for the actual problem and the name for the problem of solving that problem while satisfying type system constraints. Bleh. -- Rich Hickey

Solving the problem doesn't require static typing.

tl;dr on the above: The Expression Problem was originally expressly formulated with maintaining a Haskellian level of static type checking as one of its core requirements.

Saying you've solved it, when your solution involves on dynamic typing, or even casts in an otherwise static language, is arguably akin to kicking the ball across the center field line and then claiming you've scored a goal.

(See: http://homepages.inf.ed.ac.uk/wadler/papers/expression/expre...)

Actually, clojure compiles down to java classes, which are statically typed. Existing java types can be extended with clojure protocols. The link below talks about this, specifically about the expression problem.

See https://www.ibm.com/developerworks/library/j-clojure-protoco...

However, I learnt something. I didn't know that the definition of the expression problem required static typing. I prefer to think of it as adding static types to multiple dispatch.

You missed a detail: or even casts in an otherwise static language.

Clojure does compile down to Java classes, but Java is only a partially statically typed language. It also permits run-time casting, which, in a language that even tries to be strongly typed, means run-time (to wit: dynamic) type checking. And Clojure relies heavily on that.

That's why I invoked Haskell. It's an example of a language that is truly statically typed, in that it doesn't (generally) permit any run-time type conversions. ALL type checks must be done statically.

Then, the conclusion can be that dynamic languages don't suffer from the expression problem.

Pretty much. Not entirely unlike how one can't implement the Y combinator in a statically typed language, but it's trivial to do so in a dynamic language.

“I know of no widely-used language that solves The Expression Problem while satisfying the constraints of independent compilation and static typing” - kind of implies that the problem is different from whether or not the language is statically typed, doesn’t it?

I don’t think you should be writing off Clojure’s multimethods too quickly - they are quite nice, and I haven’t seen anything like them in other dynamic languages like Python.

You've cherry-picked that sentence, though. Given the entire context of the article, I would argue that one should interpret that as a case of unclear writing, and not something that is intended to directly contradict the definition given in the second sentence: "The goal is to define a datatype by cases, where one can add new cases to the datatype and new functions over the datatype, without recompiling existing code, and while retaining static type safety (e.g., no casts)." (emphasis mine)

I’m not much of a clojure fan myself, but I think the observation is that by relaxing that last constraint of static typing (and given the other appropriate tools), that while you can’t solve The Expression Problem itself, it’s a bit easier to solve the real world problem that happened to manifest itself as the expression problem in your code.

By the way, I was also enticed by multi-methods in this area, but I ultimately don’t think multiple dispatch is necessary.

Yes, absolutely, but doing so would be moving the goalpost right out of the stadium. The expression problem isn't supposed to describe a challenge for people implementing line-of-business applications. It's a problem for programming language researchers.

>core.async: So the guys who built golang did it because they had this cool idea for creating coroutines using channels. However, since it required very low-level functionality to be part of the core of the programming language, they thought they'd have to design a brand new language to implement their idea. However, shortly after golang was released, Rich and some other clojure folks implemented the same feature in clojure as a totally ordinary external library, proving that the core of clojure was general enough to support this. And it wasn't just a gimmick: I use core.async every day and think it is better than golang's implementation.

It is very limited -- it was done by a single university student as his MS thesis -- but luaproc had added this functionality to Lua before Go even existed:


Other M.S. theses from PUC have extended the library (in particular, adding support for sending tables) but it is otherwise unmaintained:

https://www.maxwell.vrac.puc-rio.br/30267/30267.PDF https://github.com/lmillanfdez/luaproc-master https://www.maxwell.vrac.puc-rio.br/35424/35424.PDF https://github.com/fernando-ala/luaproc---messaging-tables-a...

Go was inspired by several systems that have come before it. You can write coroutine code with channels in C.


It’s available for unix systems as well in the Plan9Port, and that predates Go by a bit.

I’m not sure what any of this has to do with Clojure though other than to say “look we have Go-like concurrency without starting over”. It’s been shown that you don’t have to start over to add features to other languages as well.

See also libmill http://libmill.org/

Go was not solely created to have goroutines and channels, but that was definitely one of the features desired.

Yes! I must point out a little factual error: JVM Clojure is not transpiled to Java, it is compiled on-demand to JVM bytecode.

core.async in Clojure is not equivalent to Go's asynchronous support. Even worse, core.async is the simplest & least-performant way to implement asynchronicity: It's just using an internal thread pool, meaning that go blocks in Clojure are not at all like goroutines in Go since they are state-machines produced by a macro that can block the entire thread when doing blocking calls. So, one has to manually "park" them by using core.async channels.

Summarizing: core.async is macro sugar on top of java.util.concurrent.

Go has actual lightweight processes that are scheduled on top of OS threads. Big difference in both design and runtime performance. Erlang too has m:n threading and offers a simpler and safer programming model (each Erlang process has its own heap).

Re "core.async is macro sugar on top of java.util.concurrent" - yes, but also no. Java does not (yet) have a way to spin light-weight processes that can park without consuming a thread OR a way to select/alt over multiple parked ops, so being able to do that in Clojure is actually novel (and enabled by having macros that can rewrite code as a state machine).

There have been a few projects that did this in the past on Java using runtime bytecode modification but none of them are widely used afaik.

This will change once Project Loom lands in Java and provides m:n fibers.

Yes, yes in the future at some undefined point java will be better.

Meanwhile go works very well today. And you don't need to be saddled with the JVM and the very bloated java ecosystem.

>very bloated java ecosystem

Why waste time reinventing what java has already solved?

Life is too short to waste on Java? Yes, some of us really find Java that bad.

Plenty of other languages have solved the same problems in much better ways (as demonstrated by this very discussion re: asynchronicity). I'll pick Go over Java, Erlang over Go and wouldn't pick Java (incl languages running on top of) for anything today.

I usually find immutability to be more important than this particular detail. Because of this my preference is usually Elixir/Erlang, followed closely by Clojure (for the potential interop with a huge ecosystem), with languages like Go and Java fairly close to eachother but pretty far down list.

I would love it if Clojure on BEAM were more mainstream.

Why not?

The java ecosystem has solved so many problems already.

Don’t you want to just focus on the business problems?

The business problem still can include things like "concurrency", which is just -painful- in Java, especially if I/O bound.

I much prefer languages that make easy stuff harder, but hard stuff easier, than languages like Java that make easy stuff easier, and hard stuff harder.

Java doesnt solve business problems.

It solves enterprise ones.

Or at least claims to.

The Java ecosystem has solved many business problems, for me and plenty of other people I personally know, both in small startups and in large companies. Just because you are personally prejudiced against Java for some reason, doesn't mean that its not a good ecosystem, because it is. I don't like Java the language (and certainly not the crazy frameworks), although it has improved, but the ecosystem, especially from Clojure, is fantastic.

Enterprises do business last time I checked.

I don't solve business problems, I solve my own problems and try to have fun along the way.

You are confusing Java with the JVM.

No I'm not, there is no escaping Java when you work with Clojure, since you'll be using Java libraries and working with Java APIs all the time.

Depends on what you're doing.

Regarding Java, sure. But in the context of Clojure, it's here right now as well, no need to wait for anything.

I also don't really understand the "bloat" comment. What do you refer too? Are you referring to the fact the JVM supports too many features?

Sure, sometimes a simple, crude runtime will do, but if you need really good performance and low-overhead deep observability, you might want to be saddled with something that's more state-of-the-art.

Go's concurrency primitives are a less expressive version of concurrentML, which imo has been the nicest way of writing concurrent (but for a long time not parallel) applications the last 30 years.

Guile scheme has a parallel version of it that is implemented as a library: https://github.com/wingo/fibers/wiki/Manual

Comparing concurrent systems is hard. I would suggest implementing selecting with negative acknowledgement (nack). If I can do that nicely I know I will probably like the system.

I am confused. If I remember correctly, go's runtime uses a thread pool as well. It's not like go somehow invented a runtime that does not rely on OS threads. So in the end of the day, what difference does it make?

If you ask me, java.util.concurrent is a much better and battle tested concurrent infrastructure than the simplistic implementation in go, because java.util.concurrent is actually written by the world's foremost concurrency expert, Doug Lee. I will take that over someone else's hand rolling concurrent runtime any day. God knows how many bugs there are in your home grown concurrency code.

It makes a huge difference. You can't use blocking calls in core.async, since they'll block the entire thread. There is no switching when you do a blocking call, the thread your code runs on will be out of commission until the blocking call returns. So this means that blocking APIs are out of the question and this translates to you not being able to write linear, straightforward code. You have to use non-blocking APIs and callbacks, and rely on core.async channels to force switching between blocks (core.async doesn't have lightweight processes). This is a total nightmare to debug and forces you to bend your programming logic to support this non-optimal programming model.

Constrast with Go goroutines or Erlang processes where you write linear, synchronous code that executes as part of lightweight processes. The built-in scheduler will automatically schedule these processes on real OS threads, and switch between them when they block (either through channels or blocking calls). OS thread utilization is far more effective, debugging is a piece of cake and the code that you write is easy to read and understand.

I'm bypassing your "world's foremost concurrency expert" comments. I suggest you use the concurrency offered by Go and Erlang by spending a few hours to do tasks you've done with core.async. You will learn something and you will see how Clojure falls far short of the optimal.

Hum, what you're saying seems to contradict a lot of what I've read. We're talking about IO here. So something has to block unless we're using non-blocking IO. I think what you mean is that Go will detect the block, and will yield your execution, and manage the IO in a non-blocking way automatically for you?

Under the hood, go still relies on non-blocking IO, epoll and all. And for disk and other serial IO I think it actually allocates real blocking OS threads as well.

I agree, the automatic management is convenient, but I fail to see the difference from core.async and the direct use of async IO or blocking OS threads when that's not available like for disk IO.

Go scheduler is sophisticated and it indeed intercepts all blocking system calls.

Essentially Go runtime will run a different goroutine, if current goroutine is blocked on: blocking syscall (for example opening a file), network input, channel operations, primitives in the sync package.


I would always prefer the runtime to do this job instead of having to scan code under a microscope across functions, libraries and module boundaries to see what blocks and what doesn't.


If I wanted complete control, I would use C++

I don't disagree about the convenience of having the Go runtime handle this automatically for you. I do wish core.async could as well.

That said, all your prior comments seem misleading, at least they confused me because it sounded like you were implying something that didn't feel accurate. Core.async has lightweight processes. The core.async Go macro can yield to other Go blocks and it will be really cheap to do so. You can have hundreds of thousands of them concurrently parked and cooperatively switching between each other, etc.

In effect, it means that you can achieve the same scale of concurrency in Clojure as you can in Go.

Yes, it will be trickier in Clojure, because you have to manage the async IO on your own, and you have to delegate an OS thread on your own when blocking IO is needed. In ClojureScript it won't be as much an issue since all IO is async, but still not as nice as auto-yielding by the runtime under the hood as Go does.

core.async doesn't have lightweight processes, for any sane definition of these terms. A code transformation spat out by a macro, subject to all the serious drawbacks I've already mentioned, is not a lightweight process.

But looking at the number of downvotes I've received, stating the obvious no less, I think I'll stop here. I will however take the completely nonsensical, misleading and downright erroneous responses illustrated in this thread as signaling me to steer well clear of Clojure and its community. Let's not forget that this thread started with someone claiming core.async is better than Go's goroutines.

> But given the number of downvotes I've received here

If I were to guess, I'd say it's because your comments feel like they come from a place of bad faith.

I'll give you the benefit of the doubt here.

> core.async doesn't have lightweight processes, for any sane definition of these terms

We can argue semantics if you want, but that leads to nothing constructive.

I understand very well that the underlying implementation is different. Core.async is a very clever code rewriting machinery. Go actually has a user level thread implementation, with each Go block its own stack, and the Go runtime schedules those over real OS threads. I know all that.

The implementation doesn't matter for what I'm saying though. At the end of the day, it means you can pause and resume multiple logical code blocks and cooperatively alternate between which one is given another chunk of CPU time. All while keeping memory footprint low, and with very fast context switches.

Thus you can achieve a similar scale of concurrency, and even though they achieve this with a different implementation, Go and Clojure both achieve the same end result.

While I totally acknowledge as well that the implementation Go uses allows for a more ergonomic programmer experience.

Here is a very simple example that I think does a great job of illustrating how Clojure was designed to deal with "data" in a straight-forward and concise way:

  (defn csv-data->maps [csv-data]
    (map zipmap
      (repeat (first csv-data))
      (rest csv-data)))
This is a function that, when given a contents of a CSV file (list of sequences), converts it to a list of maps, where the keys are the column headers.

I'm sure there are clever ways of accomplishing this in other languages, but the default way a Java/C# dev would probably approach this is to create a class that represents the CSV columns, and imperatively iterate over the contents.

With Clojure, the above function is idiomatic, and 4 lines long.

When it comes to "information processing" systems, which is what a lot of us work on, having a language with a primary purpose to provide powerful and concise tools to process that information is.. I don't know, liberating? (maybe not the best choice in words).

(this example was a bit of an eye-opener for me, coming from a .Net background, as I was writing some simple Clojure code that needed to read in some data and was struck at how simple this solution is)

Here it is in Erlang, which is another language which forces you not to think in loops:

  csv_to_maps([Head | Lines]) ->
      lists:map(fun(L) -> maps:from_list(lists:zip(Head), L)) end, Lines).
Another language which works differently is Rebol:

  csv-to-maps: func [csv-data] [
      map-each line next csv-data [map zipmap first csv-data line]
Whats interesting about above is that `first csv-data` will be re-evaluating on each `map-each` sequence. Of course you can assign to variable beforehand. Alternatively because code-is-data and vice-versa you can use COMPOSE it inline:

  csv-to-maps: func [csv-data] [
      map-each line next csv-data compose [map zipmap (first csv-data) line]

NB. Rebol doesn't come with ZIPMAP so here's a definition for above to work:

    zipmap: func [keys vals] [
        collect [
            forall keys [
                keep reduce [keys/1 vals/(index? keys)]

Opps... a superfluous parenthesis sneaked into Erlang code there!!

  csv_data_to_maps([Head | Lines]) ->
    lists:map(fun(L) -> maps:from_list(lists:zip(Head, L)) end, Lines).

If you really wanted to (and you understand Clojure's destructuring and lambda syntax) you could even do it in two lines:

  (defn csv-data->maps [[head & lines]]
    (map #(zipmap head %) lines))

i prefer perl

It may look terse, but its not code golf.

Any Clojure developer would understand what this does (its actually pretty straight forward), but to an outsider, it definitely looks strange.

Part of using Clojure is becoming familiar with the language, its syntax, and its idioms.

Rich even has a section in "Simple made easy" where he talks about this very thing; its on us to become familiar with the tools we are using.

Do you have any recommendations of online resources or books that can get a web programmer (experience in PHP / C# ) up to speed with clojure ecosystem ?

Hi fellow PHP'er and in a long time gone C#er here

I would highly recommend learning the spirit of Clojure first:



There are a few classes of tech that are uniquely Clojure:

Data driven DSLs:

- https://github.com/noprompt/garden

- https://github.com/weavejester/hiccup

- https://github.com/seancorfield/honeysql

Hyper normalised relational databases:

- https://www.datomic.com/

- https://opencrux.com/

- https://github.com/replikativ/datahike

Advanced SPA tech (hyper normalised data driven):

  - http://fulcro.fulcrologic.com/

  - https://wilkerlucio.github.io/pathom/v2
Once you understand the spirit and rationale for Clojure it becomes apparent why other communities don't have this kind of tech yet

Once it gets down to practical things I recommend using clj-Kondo with type hints, Cursive for Intellji, make sure you learn how to hot code inject new code into your running program using your editor shortcuts, and TDD in Clojure is also excellent and immediate: https://cursive-ide.com/userguide/testing.html

Also look out for GraalVM and Babashka we're using it to compile fast native binaries out of Clojure

While this is a pretty complete list of some of the most interesting things out there, it might be a little overwhelming to a newcomer.

I'd suggest starting with what everyone seems to start with:

- https://www.braveclojure.com/clojure-for-the-brave-and-true/

Its a pretty fun read, and does a good job of covering the language (plus its free, which is probably why most people start there).

You could also take a look at Programming Clojure (written by some of the people behind the language, Alex Miller and Stuart Halloway), which I think is a better resource, but it does cost money.

It can be tempting to want to start in the deep end and try to create something like a web app from scratch as your first attempt at using the language, but there isn't really a Django or Rails for Clojure and you can easily get lost in the weeds (as I did).

My approach to learning Clojure was to start with a simple setup (for me its VS Code with the Calva extension, and deps.edn for managing the project files) and forcing myself to use Clojure for any small utility scripts that I might have otherwise done in bash or JavaScript.

This allowed me to get a feel for the language and how to work with the REPL without having to also digest a lot of information on how a specific library or framework works.

thanks, i'll take a deeper look at Clojure. I need to have a way to get my java fix and scala isn't doing it.

You can look at Kotlin as well. Or ABCL.

If you actually supplied a Perl version maybe you could provide a convincing argument.

Here's a more or less equivalent implementation in Common Lisp, using alist-hash-table from alexandria:

    (defun csv-to-maps (csv)
      (loop for row in (rest csv) collecting
            (mapcar #'cons (first csv) row) :test #'equal)))
For CSVs with only a few columns (maybe less than 10?), I'd probably go with alists instead of a hash table to save memory, and it'd look like this, using curry from Alexandria:

    (defun csv-to-alists (csv)
           (map 'list (curry #'mapcar #'cons (first csv)) (rest csv)))

If you read Rich's paper, there's a section where it says:

> maps were built behind several abstractions. Thus, maps can start with a representation optimized for small maps and switch to a representation optimized for large collections if and when they ‘grow’ beyond a certain size.

Which is pretty cool, because basically the Clojure version of your code abstracts over alist-hash-table and alist, and the most optimal type of concrete map will automatically be created based on the size of the map you are asking to create.

There's actually an implementation of the same HAMT data structure that Clojure uses available in QuickLisp and on GitHub: https://github.com/danshapero/cl-hamt/

Technically, since the hash-table implementation isn't specified in the CL standard an implementation could use a HAMT for the built-in hash tables, but I don't think any do.

The construction of a Map is abstract in Clojure. The concrete type of Map you get is not something you specify. This isn't related to the HAMT per say.

So what I mean is Clojure will use an array backed map when you ask it to construct a small map, and it will give you a HAMT backed map when the map is large.

This is true as well as you add elements to a map, Clojure might automatically promote the map from an array map to a HAMT as its size grows.

I'm a little late to the party, but I don't think I would hire an experienced C# developer that didn't immediately know this could be a one liner:

  var csv = ImmutableList<ImmutableList<string>>.Empty;
  var map = csv.ToImmutableDictionary(y => y.First(), y => y.Skip(1).ToImmutableList());
I would hope they would then tell me that this will throw an exception when there is a repeat header in the csv data. I'm not sure the functional and safe version is that much cleaner than a for loop:

  var map = csv
    .Where(x => !string.IsNullOrWhiteSpace(x.FirstOrDefault()))
    .GroupBy(x => x.First())
    .ToImmutableDictionary(x => x.Key, x => x.First().Skip(1).ToImmutableList());
Edit: But I would hire a developer that knew how to do the equivalent in Clojure without hesitation.

"The default way a Java/C# dev would probably approach this is to create a class"

No, not for a long time. The idiomatic C# way would be to use LINQ which actually predates Clojure a tiny bit. Java (since version 8 in 2014) could use Streams to similar effects. Either way the solution would be as short as your example.

Assuming that you are mapping the results to a Dictionary, sure it would be similar.

My point was more that C#/Java developers (myself included at one point in time) would typically reach for classes to store the results, rather than just a Dictionary<String, String>.

It’s not that other languages can’t to this, it just that this approach doesn’t always jive with their conventions like it does with Clojure.

Right, the more idiomatic approach in java would be

- define the class

- let the framework fully handle serializing/deserializing it from your file to the neat in-memory object

That’s great if your dealing exclusively with CSV files and a fixed set of data types, but the point is that this approach in Clojure is universal regardless of where the data is coming from, or going to.

Everything comes in as a map, gets manipulated with the powerful set of core Clojure core functions, and spit back out as a different shaped map.

It’s a very flexible approach to dealing with data, and that is one of the main appeals of Clojure.

Yup, Java in its current incarnation is definitely a better fit for larger companies that can present more standardized workloads. You want to give each team clean and explicit interfaces that they have to work with, and not have to worry about different shaped maps. Flexibility and power just lets junior developers shoot themselves in the foot.

I agree, I wouldn’t suggest Clojure for a large company with lots of junior developers.

I wouldn’t suggest Java either, though. Go would be a better fit for that situation these days.

I look at Clojure and Go as different sides of a coin.

One is a powerful language the let’s a small team of experienced developers get a lot done with a small amount of code.

The other is a simple language that lets a lot of less experienced developers get a lot done with a lot of code.

I want to stress in this comparison though, that I don’t view one as better than the other.

Go was designed for development at scale, where you have lots of developers, all at different levels, all needing to contribute large amounts of code. In this respect, it does a great job at removing most of the rope that people can use to hang themselves. If I needed to pick one language to use at a large enterprise, it would probably be Go.

Clojure gives you a (different) set of limited functionality, but what it gives you is very powerful, and very foreign to most developers, so you can definitely hang yourself trying to force everyone into using it. If I got to choose one language for myself to use, it would be Clojure.

Go reminds me a lot of the old days of Java, that lead it to its current dominance. It used to be explicit and wordy, but there was usually just one way to do things, so a code base would stay very consistent, leading to much better portability even as code constantly changed hands between developers.

For people coming from a java background, I've had good results with other java+syntactic sugar languages as well, like Groovy and Kotlin, that have accelerated the growth of java to try to pick up some of the best parts of other more modern languages.

I would tend to disagree with the idea of "only choose one language for myself". You want to pick the right language for the job, you want to consider the business needs at hand, you want to consider the team that you have, you want to consider the experience that your team has. Even a company that would prefer to use as few languages as possible could easily end up with javascript for the frontend, ruby for the backend, go for infrastructure, python for ML. And at the end of the day, it's just code, it's just a tool to express your own skills as an engineer.

And then as an engineer personally, you need to get exposed to a range of languages, because each language teaches you a different mindset, and brings with it a different community with different priorities. Not referring to you specifically, but I've seen a lot of junior developers get into Clojure and all of a sudden it's the most amazing language because their life as a Java engineer before was horrible. But it's not Clojure specifically, it's the idea of learning new languages that is itself powerful.

A counter point is that most large companies these days operate with lots of small teams, not one big team. And most systems are composed through a service oriented architecture of some sort, not a single giant monolith worked on by all teams.

So in practice, even at a large company, you find yourself with lots of small teams.

And no such small team at a large company would be composed exclusively of junior developers. There's at minimum going to be a team lead of some sort, and most often you'll have a spectrum from senior to junior.

So in reality, Clojure can be quite a good fit even at large companies. In fact, it might be a better fit at bigger companies, because I think small startup might actually be where you could find a small team of only juniors, and a "deliver at all cost no matter the quality" business need (due to not being profitable yet).

I can't help but feel that all arguments defending how this Clojure solution supposedly can't be done as elegantly in other languages, use "idiomatic Clojure" as the judging criteria. Of course the conclusion writes itself.

Well, you can write code a lot like this in JavaScript using a library like Ramda, and get a lot of the benefits.

But then everyone that you on-board to that project has to be up to speed with how the code is being written and why.

With Clojure, that understanding is table-stakes.

So in that respect, the conclusion does write itself.

I've actually tried that: https://github.com/enumatech/sprites/blob/master/lib/__tests...

reads quite like Clojure, but looks foreign to JavaScript programmers...

I even made extensions to Ramda, so you can thread a mixture of async and non-async code with it: https://github.com/enumatech/sprites/blob/master/lib/fp.js#L...

At the end the friction was just annoying and no one else really understood the greatness of this approach, since they didn't know Clojure...

In my experience, it's not that you can't do these things in some other languages, it's that you won't, and if you try too, there will be more ceremony and friction.

That said, this is where a lot of people who took a detour through Clojure, come out of it saying when they went back to their prior language or some other language, they felt like they had suddenly become better programmers, and it helped them even in other languages to write better programs. Clojure showed them the way, and now they can recognize it and think of doing it as such in other languages as well.

Maybe a different way to think of it is that you can do OOP in Clojure if you really want, but you won't. Similarly, in Java, you could try and do FP with immutable collections and value semantics, but most likely you're not going too.

And I'm using Java here because in theory, Java and Clojure have the exact same set of available features, they run on the same JVM, same runtime, compile to the same bytecode, can share libraries, etc. So it goes to show that the language itself does influence the style that is most convenient for writing your programs in.

I am familiar with C#, and in good faith spent a bit of time trying to find examples to support this, but in the end using a class is both common and typical (omitting the many StackOverflow results with the same pattern): https://dotnetcoretutorials.com/2018/08/04/csv-parsing-in-ne...

His example is just something like .ToDictionary(column => column.First(), column => column.Skip(1).ToList())* .

The motivation to eventually use a Class would be to make use of static typing, e.g. when using an ORM like Entity Framework (a common usecase: reading a CSV file into a relational database). Static typing is usually considered a strength these days, but it does have its downsides (that debate has been done to death). Showing the parallel Clojure code for that could be very interesting.

However, reading CSVs doesn't require enforcing types, even if one wishes to use a library to deal with the complex quoting cases. CsvHelper is tilted in that direction of mapping to a static type, but it's not the only Csv reading library (I'm a pretty happy user of ExcelDataReader[0] for other reasons, and it doesn't enforce mapping. I'm sure there are more functional-style solutions out there). The original post picked the one place C# uses a powerful functional-like syntax, its generalized query language....

* From my limited understanding, not being a user of Clojure yet (I've been considering learning it for some time, just need to find time to do it seriously), each 'sequence' in the original example is a column, not a row. Were each sequence a row, in C# we could have used the positional lambda argument to get the columns.

[0] https://github.com/ExcelDataReader/ExcelDataReader

It's less lines in JS/TS, depending on how you format the parens and braces.

    let csvData = [
      ["Name", "Age", "Email"],
      ["John", 25, "john@email.com"],
      ["Mary", 30, "mary@email.com"],
      ["Anne", 40, "anne@email.com"],

    let parseCSV = ([cols, ...rows]) => 
      rows.map(row => 
        row.reduce((res, val, i) => Object.assign(res, { [cols[i]]: val }), {}))


Yes, as I said in my original post (and others in this thread), you can absolutely come up with elegant solutions to this in other languages.

In my experience though, for every one elegant solution like yours, there will be ten developers writing nested for loops.

My example was never meant to be “look at what Clojure can do! You should all be jealous! “. Only an example of how Clojure encourages a different approach to solving problems that we are all familiar with.

Its because the default abstractions provided as part of the language shape the way developers using that language approach problems. In Clojure, the sequence abstraction has been given a lot of thought and effort, which means that’s what Clojure developers will try to use. It also Clojure comes with a rich and powerful set if sequence functions which makes using sequences a breeze and lets you do super powerful things out if the box.

You could simply implement the exact same functions in other languages too and end up with similar code, butsince its not built in, most people won’t think in those abstractions.

> there will be ten developers writing nested for loops.

Ain't that the forever truth =/

nested for loops go brrrrrr

Aaaand this works on arrays, but would you know how to make it work for files? You would need to do something like how the essay wrapped the `rdr` into a sequence, but it would still require (probably) the whole data to be present in the memory at once...

I think you would need something like this: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

but I found it quite mind-bending when I tried it once. Clojure's way of building something like this out of a `lazy-seq` call feels simpler to me...

I find that much less readable than Clojure.

Is that due to familiarity or another reason?

I, too, find the Clojure version to be easier to read, but I spend more time with Clojure than JS.

I'm not really familiar with either, I think the Clojure version just makes the intent clearer.

Wouldn't this be this just as feasible and elegant in JS, Python or Scala? Maybe not as short but short enough.

Then again with Python I'd just fire up NumPy or ugh Pandas(which is basically NumPy with benefits) for this kind of transformation.

You could definitely do something like this in other languages, probably even in Java/C# using a HashMap or Dictionary, but the difference is that this is “the way” you would do this in Clojure. It’s not, generally speaking, they way you’d do it in the other languages you mentioned.

I think it’s one of the reasons why people always say that learning Clojure makes them better developers even when they’re not using Clojure. The ideas that that take away from idiomatic Clojure and use in other languages.

I think Reagent would be the easiest and most visible example.

Reagent was my intro to React, I had no experience prior nor did I care to learn it at the time. The reagent wrapper around react made it so intuitive to use that I was writing productive (albeit warty code because I had 0 experience with functional programming/lisp/clojure) my first day. I had loads of experience writing Javascript, but at that time React looked cryptic to me without reading any documentation and Reagent did not.

Another commenter mentioned lodash. If I was forced to use JS directly, lodash would be the first thing I reach out for to solve problems.

But in Clojure you also get "cljc" which is a file extension that lets you use code both in Clojure and Clojurescript land simultaenously. With this you can write things like form validation code that's exactly the same front and backend.

The are aspects of solution shapes that Clojure enables that are unique and mostly completely different than other languages, but it is hard to appreciate them if you don't know it. One's intellectual appreciation is literally limited to the languages one speaks (both human- verbal and eg visual, like art, and computer programming).

I look at the benefits more concretely. It is usually possible to express a solution in Clojure with a fraction of the keystrokes another language requires, and Clojure also gives you knobs to control how many keystrokes you devote to the solution vs the stuff under the solution.

You can be all solution domain and be really lean, or you can make some reusable infrastructure to make the solution-side shorter, at the cost of something bigger overall.

The secret of good writing is usually distilled as- tell less, say more. Take away, cut, remove, until you get to the essence.

So many languages REQUIRE you to tell more, with ceremony and patterns and boilerplate. Clojure has some too, but for any given semantic intention, it generally requires the least ceremony.

It is breathtaking to wield a tool that lets you say, in a couple hundred lines, what it takes many thousands in other languages.

It is also hard, and such tools are sharp and can be humbling. But when you arrive at a place where there is no more to take away- I have not felt that "wow" in another language in quite the same way.

(not OP)

I've got a completely contrived example for you, but those are things that are typically easier to do in Clojure than in the languages I know otherwise:

Let's say you're writing a web-application that consumes JSON form an external source. You have to send out responses that accept and reject requests and you add some meta-data.

You have to meet a bunch consistency guarantees in your business logic like: a person cannot buy more than X products at discount Y in a time-period Z. Or some other arbitrary rules.

The external source decides to add an additional field in their JSON tomorrow. With Clojure this isn't considered a breaking change, you don't change a single line of code since you don't care about their new field at all.

Next week, the external source has a bug, they send out messages that are well-formed, but inconsistent with your business logic. You already wrote function specs for those and your messages automatically "explain" why your spec deems those inconsistent. Again, you don't change a single line.

A month afterwards the maintainers of the external source went through a major "refactoring". Now they changed the structure of their messages towards you and you are forced to be compatible. In Clojure this is a matter of adding a new spec and moving around some expressions, or better: composing a new (v2) top level function to meet the new spec.

Again, this is hyperbole, but there is some truth in it as well.

None of this sounds like a problem in a statically-typed system with proper data modelling. In fact for the complete refactor of the external source we would do something very similar to Clojure, i.e. update our decoders for the external data. In fact I'd argue it would be even easier because the compiler would help check all the data is accounted for.

I agree with you in the sense that these things “done proper” would be robust. It is meant as a pointer to what is specifically one strength of Clojure in comparison.

From my experience marshaling/deserialization and bubbling up errors as an introspectable useful message are things that are more involved in statically typed languages than in Clojure.

Typically statically typed languages are very good at internal consistency. But require more ceremony and maintenance regarding outside sources. This is a viable tradeoff in some cases but not so in others.

It all comes down to proper data modelling, whether in a statically-typed or a dynamically-typed codebase. If you were consuming a JSON API that gave you this data:

    {"id": 1, "name": "Bob", "age": 55}
But your Clojure app only needed the 'id' and 'name', why would you write a parser function that also grabbed 'age'? E.g. (I don't know specific Clojure libs, so this is for illustration only):

    (defn decode-user [json-obj]
      {:id (json/field "id" json-obj)
       :name (json/field "name" json-obj)
       :age (json/field "age" json-obj)}
This doesn't make sense, rignt? Of course. And you wouldn't do it in a statically-typed language either. See https://lexi-lambda.github.io/blog/2020/01/19/no-dynamic-typ... for more on this.

Right, this makes a good point and is worth studying more closely. We have to distinguish these concepts. But from my experience this isn’t done in practice. And by that I mean libraries, frameworks, idiomatic use, culture and so on, which the post at least partially acknowledges (or hints).

This is also a reminder that type systems are not all equal at all. The range of expression matters especially in these kind of discussions.

Rich hickey explains at this timestamp in the video,


"The information programming problem"

Whole video is gold, is my fav programming talk of all time.

I feel the other answers to your comment failed to really illustrate the property.

The history paper Rich Hickey wrote alludes to this throughout it all.

Basically, data is central, everything about the language makes modeling data simple, first class, and non-ceremonious. You can take the data your domain relies on, and use it directly, in the same structure the domain structures it, no abstraction needed, no mappings, no adapters, just straight up.

It's really hard to communicate what a data centric approach feels like. But I will try. Keep in mind, there are a multitude of details and features across the entire language which all focus on this data centric approach, and they all serve a role and come together to create this property. It is not any one feature, but really the sum of all of them which enable the property to surface. I will mention only a few to give an idea.

#1 Flexible data representation

Imagine we have a business, and they have product listings, and their products are uniquely identified by vendor and name. In pseudo-model we would have:

  vendor+name -> product
Now in Clojure we can model this as:

  {[vendor, name] product}
That's it. This will now be the data-structure we will use. We're now going to write a bunch of operations over it which map to the operations the business does in its day to day with regards to its product listing.

Think about how you'd model that in other languages. In JavaScript, which also has a pretty flexible data model, this won't work, because keys to JS Objects cannot be a composite. One would need to have a string encoding and some mapping from the input to the string encoded variant and back, such as:

  {"vendor_name": product}
In Java, one would refrain from using a Map<List, Product>, and instead would model this as classes. Maybe you'd have:

  public class Listing {
    String vendor;
    String name;
    Product product;
  List<Listing> listings;
But now you can't easily lookup for a product by vendor+name. So maybe one would instead do:

  public class ProductId {
    String vendor;
    String name;
  Map<ProductId, Product> listings;
Except it turns out that for certain products, but not all, the business also distinguishes them by manufactured year. And some other are distinguished in addition not by manufactured year, but color. In Clojure you'd just have:

  {[vendor, name] product}
  {[vendor, name, color] product}
  {[vendor, name, manufactured-year] product}
In Java you'd now have:

  public class ProductId {
    String vendor;
    String name;
    String color;
    String manufactured-year;
  Map<ProductId, Product> listings;
Where color or manufactured-year can be null.

And maybe you don't find the Java one that bad quite yet. So let's now talk about a second data centric feature:

#2 Value semantics

It turns out, in Java, the above code does not work. If the business says, find me the product Kraft MacNCheese, you might be tempted to do:

  ProductId productId = new ProductId("Kraft", "MacNCheese");
But the productId you created is not equal to the productId of equal value which is currently stored in your Map. Two objects in Java are equal if they are the same object, not if they have the same value. In Clojure:

  (get listings ["Kraft", "MacNCheese"])
That's it. In Clojure, things of the same logical value are equal by default. As such, Clojure has value semantics. Or in other words, equal data is equal, as it should be in my opinion.

#3 Data is same in memory and out

So we've come to the point where we want to export our listings. In Clojure we'd do:

  (spit "/home/user/listings/my-listings" listings)
That's it. And when the user wants to import listings:

  (-> (slurp "/home/user/listings/my-listings")
Because all data in Clojure can be serialized and deserialized as is by default. This gives us back our exact listings data-structure that we had.


And like I said, there's many more such little features all thought of from a data centric perspective. When you add them all up, you start to realize everything is just data needed to be manipulated from one shape to another, and moved from one place to another.

In Java you can (and should) implement equality semantics however you want for Objects by implementing equals/hashCode. If you just want value semantics you can use a record type, which works just like the Clojure example.

I think the point is that in Clojure you get value semantics out of the box, but in Java you have to put in extra effort to get it.

Nothing really different from writing functional JS with lodash and being conscious about immutability (or use immerjs). For macros you can use Babel.

That's kind of like saying you can write immutable data structures and then bolt on STM and try to force yourself to stay in functional territory as much as possible in any language, which the paper addresses. You can do this, and Rich did it in C#, but in my experience it's not long before you start to have clashes of philosophy with other libraries you depend on and with the language itself, not to mention people you work with.

Believe me, I tried on one project to write as Clojure-y in Ruby as I could, and as soon as you reach for a library you need to interact with that doesn't play by the rules, things get weird if you want to keep things functional and immutable all the way down. You need to flagrantly violate established idioms in the core of the language, and often you wind up reimplementing things in the core of the language. People who aren't familiar with why you wrote it a particular way will come in at some point later and imperative or OOP it up.

Are you speaking from experience of Clojure?

Yes, heavily. Clojure influenced(mostly because for Rich Hickey talks) many parts of the JS functional programming ecosystem. Many functions in lodash and other libraries(immutablejs etc...) are somewhat directly or indirectly influenced from ideas Clojure help make more mainstream. Even Java the language itself has been heavily influenced by Clojure.

So this ecosystems implement the best ideas from niche influential languages, you can use Clojure's "programming model" without leaving the productivity and practical benefits of a big ecosystem. This has been more prominent in JS since the language lends itself better to FP.

This reads to me as a rather superficial take on Clojure.

It's true that JavaScript - or any language, for that matter - can implement libraries that may have been inspired by a Clojure library. But specific libraries aren't what's being talked about here.

The characteristic of Clojure in question - which, IMO, JavaScript does not answer well - is the ability to easily create libraries and DSLs that feel like an extension of the language itself.

To poorly paraphrase a quote that I can't quite remember: Lisp isn't the best language for any problem in particular. What Lisp is the best language for is being a platform on top of which you can implement your own language, that is itself the best one for the problem you're trying to solve right now.

Or, to take a concrete example: In JavaScript, you get optional static typing from TypeScript, which is actually a whole new language that is transpiled to JavaScript. In Clojure, you get optional static typing from Typed Clojure, which is just a library.

By "experience of Clojure", I meant experience of Clojure.

"By 'X', I meant X" doesn't help to explain what you mean. Can you clarify?

The first X was in quotes, so it was a purely lexical reference to the words themselves, without referring to any conventional meaning that those words might have. The second X was unquoted and so by that I meant to convey the usual meaning in English.

Another way of saying it would have been: erm right but experience of Javascript is not experience of Clojure, regardless of any influence Clojure may have had on modern Javascript.

OK. But you didn't ask him to describe his Clojure experience, you just asked whether he was speaking from Clojure experience -- and he said that yes, he was. Asked and answered?

No, on the contrary, I understood his/her reply to be saying that he/she had not used Clojure, but that its influence on the Javascript ecosystem was such that a user of modern Javascript could consider themselves to have "experience of Clojure" in a sense.

We don't have to carry on debating it, but have another read of the reply in question -- I think you'll agree that it was in fact saying that they did not have any experience of Clojure; only of Clojure-influenced Javascript.

What stack do you use for web application development for clojure?

Most notable things I use are Re-frame frontend and Datomic for back.

I’m in the opposite boat. I went from Clojure to Java.

I got really tired of “finishing” my work, only to discover that it was riddled with bugs from all kinds of weird edge cases. My favorite is having to use (map vec <thing>) all over the place, because the seq abstraction leaks like a sieve.

To be fair, I also worked in a domain that was extremely poorly suited to Clojure. The “data first” approach of Clojure falls to pieces the moment you have a bunch of data that’s syntactically similar, but semantically different. This requires a lot of cross-method coordination, creating a huge mess and a risk of missed cases.

This sounds fishy to me too... Have you actually consulted anyone, with more Clojure experience, how to avoid the need for `(map vec <thing>)` all over the place? I would guess you were working with deeply nested data structures and you had to do deep updates, while strictly maintaining the vector data-type. That's indeed kinda sucky to do just using vanilla `clojure.core`, indeed, but then there is https://github.com/redplanetlabs/specter for that. Alternatively, instead of dealing with deep nesting, you should probably work with namespaced keys, which allow flattening your data and suddenly a lot of transformations become radically simpler and such a system is more extensible too...

Your attitude is extremely patronizing.

Everyone's problems, context, and experience is different, but from where I stand it sounds like you may have had an a better experience using a data spec library like Metosin's malli.

One of the pain points I had before was different kinds of data flowing through my system. Clojure spec never made complete sense to me and was a bit difficult to write specs that matched my data. It was easier to write data that matched the specs.

Malli does this a lot more easily with human errors that have made my life enormously easier. It's been a game changer for me.

I don’t know Malli; and I haven’t used Clojure since 2017, so things might have changed. So if I’m misunderstanding the purpose of that library, my mistake.

The issue wasn’t that we would use the wrong data in the wrong place. The issue is that we had a bunch of data that was syntactically similar (huge overlap in keys), but semantically different (key numerical results must be calculated differently based on the minor differences between maps).

So the issue is less “the right data got to the wrong place” and more “in this data pipeline there are multiple places where the same data must be treated differently based on minor differences”. This is actually pretty tricky to do in a maintainable way in Clojure; technically possible but really hard to avoid breaking when you need to extend it.

In the end it actually turned out to be an issue that classes and objects solved brilliantly. The solution was to have the pipeline accept an interface (syntactic similarity) and have different classes for the behavior (semantic differences).

This reminds me of a discussion between Alan Kay and Rich Hickey on role of "data" that occurred right here on HN[1]. The TLDR (though the thread is very worth reading, IMO):

Alan Kay argues that bare "data" isn't very meaningful on its own because it leaves the interpretation too open. His view is nicely summarized by the following quote:

> For important negotiations we don't send telegrams, we send ambassadors.

Rich Hickey argues that bare "data" is a fundamental primitive that, while leaving open the possibility of misinterpretation, leaves open the possibility of re-interpretation in multiple contexts. This is the strongest quote I took away from his POV:

> None of the inputs to our sensory systems are accompanied by explanations of their meaning.

Based on your above post, it looks like you're strongly in Alan Kay's camp, which may also explain (in part) your experience struggling with Clojure's approach.

[1] https://news.ycombinator.com/item?id=11945722

That is a very succinct summary, yes. I’ll add that certain applications are more amenable to one kind of use or another.

I find the possibility of re-interpretation utterly bewildering. I have never re-interpreted my data in Clojure application, at least not in any way that I understand that word. Rewrite algorithms, yes, but you can do that in any language.

The only time I think of re-interpreting data in a different context is in a data lake or similar, which is a wildly different thing, IMHO.

The fascinating thing to me about Clojure is that it's so vocally a "practical" language, and yet is a) a Lisp, and b) purely functional[1]. Usually those two things are associated with language enthusiasts, and not """pragmatic""" development. In the end it works, clearly, it's just very interesting to see that juxtaposition.

I think it says something interesting about the practical value of higher-brow programming concepts, but also about how important the packaging/marketing/ecosystem is to conveying that value and making it accessible to the masses.

[1] I know it's technically not 100% functional, but all of its messaging highlights that philosophy as a focal point and advantage.

I think the most important part about Clojure is how ridiculously straight forward it is.

Once your REPL is set up and you got used to the predominantly functional API and its vocabulary, you are dealing with an extremely simple, data-driven environment that (almost) never gets in the way.

Part of that is due to deviations from traditional Lisps: a more human readable API (first, rest etc.) and less homoiconicity (data structure syntax). But also things like the first class support for names, a baked in, dead simple STM...

For me, this is the reason why it is so practical. I'm not good at top-down programming. I can think of and design a system or solution in an abstract way to get a sense of direction.

But when it comes to writing code I have to look at small, simple things in isolation, look at the results, change the shapes and build something up.

Clojure is very, very good at enabling this kind of dumb, simple, learning-by-doing approach of programming, because everything just works, is easily decomposed and merged together again, with very little ceremony or friction.

And when the simple approach reaches its limitations, then there are these extremely expressive tools like multiple dispatch, macros, transducers, spec etc. which all go beyond what most mainstream languages give you.

This is a minor nitpick but please check your facts about other Lisps when comparing them with Clojure.

> Part of that is due to deviations from traditional Lisps: a more human readable API (first, rest etc.)

first, second, rest ... are defined in Common Lisp

> and less homoiconicity (data structure syntax).

Clojure is no less homoiconic, you just write vector literals in your code; you could write vector literal to represent code in Common Lisp too, but in fact [], {} etc. are not used by default in CL because they are reserved for the user, so that it can be used for domain-specific purposes.

Thank you for those corrections. My limited Lisp experience before Clojure was with Scheme a very long time ago. I shouldn’t have made these false, blanket statements. Those can be harmful for discussion.

In fact I know from reading about CL that it is extremely malleable and that you would be hard pressed to find features that are not present or possible there.

> I know from reading about


> homoiconicity (data structure syntax)

Can someone explain to me why we need a murky word like "homoiconicity" and then explain it as "data structure syntax" (which doesn't clarify much) when "self-modifying code" and "metaprogramming" are crystal clear?

Homoiconicity means you're code is defined in terms of the same data structures that are used in programs written in the language. This makes self-modifying code and metaprogramming easier because the language's standard library is already full of functions that operate on those data structures.

However, you can have self-modifying code and metaprogramming without homoiconicity, and you can have homoiconicity without support for self-modifying code and metaprogramming.

Both "self-modifying code" and "metaprogramming" don't mean exactly the same thing to me as "homoiconicity"

I guess I need to learn Lisp to understand the distinction. My only reference is XLST which is also valid XML.


Well, it depends on your exact definition of "self-modifying code" and "metaprogramming", given that it's not like there's an authoratative source you can look up the definitions in. The way I understand the term without additional context, XLST is not "self-modifying code" in the same way malware, certain JIT implementation techniques etc are.

Clojure isn't actually a purely functional language. It's more of a consenting adults language that guides you towards doing the right thing by default. If you write idiomatic Clojure code, then it will be written in a purely functional style. However, Clojure won't stop you from writing imperative code or adding side effects if it makes sense in your particular scenario.

Lisp family of languages also have a lot of tangible and very practical benefits. The syntax is simple very regular, so there's less mental overhead involved in reading it. This helps avoid errors because there's less ambiguity about what the code might mean.

The syntax also makes relationships in code explicit, making it more scannable. In most languages it's not immediately obvious if the lines next to each other are related. On the other hand, s-expression syntax effectively provides you with a free diagram where you can see relationships visually.

The syntax also facilitates structural editing. Instead of working with lines of text, you work with expressions, and you think in terms of moving blocks of logic around. You can see some examples here https://calva.io/paredit/

The syntax also makes it very easy to serialize code, since code is expressed as plain data structures. This also facilitates powerful metaprogramming facilities. You can apply the language to transforming any code written in the language.

Another practical aspect of Lisps is live coding workflow where you connect the editor to the running instance of the application. This creates a very tight feedback loop where you can try things and see the results immediately. This is strictly superior to the traditional workflow where you have to restart the application and build up state to see the results. It's also much faster than using TDD. You can use the REPL for exploration, and then write tests once the code is in the state where it's doing what you want. This is a good read about the workflow https://vvvvalvalval.github.io/posts/what-makes-a-good-repl....

Here's a talk I co-presented last year that discusses an application my team works on and the specific benefits we found from using Clojure https://www.youtube.com/watch?v=IekPZpfbdaI

"... consenting adults that guides you towards doing the right thing by default..." really hits the nail on the head. I do a lot of Java interop at work, which is a necessary evil, but once I'm in Clojure code I'm happy to be gently pushed to be more functional, immutable, have less state, etc etc.

Also, thanks for all your great work Yogthos, Luminus is an amazing contribution to the Clojure ecosystem as well as all your other libraries!

thanks, glad to hear it's coming in handy

"juxtaposition"... pun intended?

You mean the "juxt" function? I had to google it, so no, but that's funny :)

Yep :)

> The fascinating thing to me about Clojure is that it's so vocally a "practical" language

I tried Clojure, but the startup times were too large to the point where it became impractical. Especially for small scripts.

If you read the history, you'll see that small scripts was never a use case targeted by Rich Hickey. It was focused on information systems programming, where a few seconds of startup time is a blip on the full running time of the application.

That said, slow startup times are only true of Clojure JVM. ClojureScript and Clojure babashka, for example, both start fast enough for small scripts.

Also, seems the JVM is slowly improving in startup time department. I was surprised to see that a hello world is now down to 800ms start time on my 2011 laptop with Java 14. It was 3s before with Java 8 on the same laptop.

For comparison, ClojureScript hello world was 550ms, and babashka hello world was 36ms. Python hello world was 68ms and Ruby was 111ms.

> Also, seems the JVM is slowly improving in startup time department.

It's not the JVM that makes Clojure JVM slow to start up, try comparing hello world in Java vs hello world in Clojure.

It's true that Java hello world will be faster than Clojure JVM hello world. That said, a big Java application with lots of classes to load, and lots of static initializers (which would approach the same number as what Clojure requires to be loaded and initialized) would take as much time as it does for Clojure.

So when people say that part of the slow startup is Clojure's fault, they mean more that it is implemented in a way that the JVM isn't very fast at starting. But if you look at Clojure implemented on other runtimes, like ClojureScript, the startup times are much faster, because the NodeJS runtime is much faster to start and initialize everything than the JVM is.

In effect, if the JVM improves startup times, so will Clojure JVM's startup times be improved. That's why I see this drastic difference between a Clojure hello world on Java 8 vs Java 14.

If you don't do aot compilation that might affect startup time negatively as Clojure will need to read and evaluate your code during startup.

And now with things like GraalVM and Babashka you can write programs and scripts that starts in a matter of milliseconds.

When have you tried Clojure?

Startup time was also a deterrent for me initially, but that was ~7-8years. I annoyed the heck out of me. I was doing Rails back then and that was too much too.

Nowadays however the situation is a lot better.

For scripts, you definitely should check out https://github.com/borkdude/babashka !

Startup time still annoys me, but I've learnt an practiced development enough that I'm leaving my REPLs open for days on end.

I use very few simple tricks to speed things up a bit:

I work Clojure cli tools (`deps.edn`) instead of `lein`.

Run `(compile 'my.main.namespace)` occasionally, when I add some bigger library to my project as recommended here: https://clojure.org/guides/dev_startup_time

And I think that's it.

You can further reduce this overhead if you are just using the the Socket REPL, but I think the best support for that (which allows evaluation interruption for example) is the Chlorine plugin for the Atom editor. I usually just accept the overhead of nREPL though.

Have you tried Babashka?

Startup time for Clojure itself has become pretty fast. My guess is you were using something like lein or boot which involves more than just starting up Clojure.

Either way, for scripting uses there are tools like babashka or lumo with sub second startup times.

You can create a graalvm native image for your Clojure app. You have to jump through some hoops to build it though.

Great story, really enjoyed reading. Rick Hickey is a great speaker (if you haven't watched his talks on youtube, do it! They are totally worth it), and certainly a great programmer, which is why I respect his opinions on dynamic typing, even if I stubbornly disagree (one day I might still dive into Clojure, but I am diving into the arguably opposite end of languages right now: Rust!!). I have even more respect for him now knowing that he actually worked on Clojure for years without any income!! This is a work of passion, and I admire him for that, and for being able to turn that passion into a company big enough to pay his bills and that of several of Clojure's early adopters.

FWIW, I don't think it really does to treat his opinions on typing as a blanket, universal, prescriptive statement that one can or should agree or disagree with according to one's own blanket, universal, prescriptive position. He generally doesn't come out and say this explicitly, but, in his talks, he's always speaking to a certain audience, about a certain class of problems.

You couldn't come up with a better pair of languages than Clojure and Rust for illustrating this sort of thing. I like them both (as far as my extremely limited experience in each will allow, anyway). But, if a language like Clojure is a candidate for the thing I'm working on, then I would never have dreamed of picking a language like Rust. And vice versa.

To add to that and concur that Rich generally doesn't have such un-nuanced strong opinions without context, see his talk Effective Programs. He explains what Clojure is good for and isn't good for, and says more clearly it so depends on what you're doing.

He doesn't organize the Clojure conferences, but I'd imagine that if he didn't see any value in any context for static typing, there wouldn't have been a talk about Typed Racket and static typing researchers as keynote speakers, etc.

And FWIW, spec is a library trying to come as close as possible to the imaginary midpoint of static and dynamic typing, but in a Clojure-y way.

>I have even more respect for him now knowing that he actually worked on Clojure for years without any income!!

Hickey mentioned in one of his talks that he actually burned through retirement savings (at least some?) while taking these years to do Clojure. I got the impression he had been thinking about it for a while prior and that it was in the neighborhood of two years. (The context was, he was thanking his wife for helping to make Clojure possible, in part by indulging in many converstions about very minute details of how Clojure would work.)

(I have no idea if this is covered in the paper as it's 46 pages, I've bookmarked it but not yet read it.)

In those 46 pages, he goes into details regarding how the idea came up to him, why he took a sabbatical of 2 years (which ended up lasting to this day, he says at the end :D ), what he did during those 2 years (not only Clojure) and how he basically has no retirement savings left because of that.

There was also a group of people who wanted to see Clojure succeed who donated money ($100 each, if I recall correctly). But I would guess that was a drop in the bucket compared to what he had to spend of his own money.

I remember being at the first Clojure/conj - I had just spent some time learning a bit of Common Lisp and stumbled onto the conference announcement somehow.

This was very fortunate for me - after attending the conj, I was able to use Clojure at work to sneak in some FP on the readily-available JVM platforms. I later used it to do some internal REST-API work. The code for that project has run for years without modification or error.

For whatever reason, I also discovered that thinking functionally with Clojure worked so much better and naturally for me than object-oriented design methods. While I have drifted away from Clojure over the last couple of years, I find that my problem solving mind is much better in other languages because of the time spent thinking functionally with Clojure.

"The code for that project has run for years without modification or error."

To follow up on this point, if you ever need ammunition to sell the use of Clojure inside an enterprise level organization, just show your Chief Architect the graphs of code stability over time for Clojure (pg. 71:26) and ClojureScript (pg. 71:30).

After working with JavaScript (not trying to bash) for the last few years, the idea of stability has become top of mind for me.

Wow, I didn't expect such differences compared to Scala. These graphs need a study on their own.

Looking at it, seems like the Scala code base found some stability in 2011?

This could be a factor:

> On 12 May 2011, Odersky and collaborators launched Typesafe Inc. (later renamed Lightbend Inc.), a company to provide commercial support, training, and services for Scala. [0]

[0] https://en.wikipedia.org/wiki/Scala_(programming_language)

I was at a talk by Guy Steele where he mentioned in passing that Clojure was a Lisp which had done everything right. This was all the more impressive to him because Rich Hickey had until then been a relative outsider to the Lisp/Scheme community.

I think what Rich got right is the practical approach. Clojure is about compromises (like all engineering). It's not a theoretical exercise, it doesn't prove a point, it's about getting things done.

This also means that sometimes things do not seem as "beautiful" as one might like. Well, that's because they aren't. There is no way to sugarcoat certain things without making the language less practical, and Clojure seems to have been designed with no sugarcoating in mind.

Incidentally, I love this quote from Rich, an answer to people complaining that Clojure isn't easy to learn: "Instruments are made for people who can play them". I've grown to appreciate that over the years. You don't expect to be able to sit at a piano of a particular brand and just start playing it.

I just wish it didn't target the JVM. I think if it had a native runtime that loaded quickly like Python it would be more popular. I love Clojure and really wish I could use it, but I just don't do the kind of things that can justify running a JVM.

This is definitely a valid criticism! Two ways that this may be getting better: GraalVM's native-image and babashka.

babashka is a little interpreter that implements large portions of "scripting Clojure" so you can write tiny Clojure scripts the way you'd write bash or Python.

GraalVM's native image is a Serious Compiler (unfortunately also with Serious RAM appetite and compile times, though this has gotten much better over the last few months) that will produce native binaries on Windows, macOS and Linux.

> babashka

I recently was looking for something like this and I also found Planck, Lumo, and Eden.

Do you (or anyone) know of a comparison between them?

Planck uses the os native webview for execution. Lumo runs on node. Eden is compatible with most clojure libraries, and is meant for embedding. Babashka is a clojure interpreter written in clojure and compiled with GraalVM. I've used Planck and babashka for shell scripting and prefer babashka.

Thank you, that was a good overview! I started giving Babashka a quick go and it looks like it'll work well for what I wanted to use it for.

He explains how he created Clojure explicitly as a hosted language. After the JVM implementation, it quickly got implemented for CLR, and then JS (ClojureScript), and both are full implementations, not just look-alike.

By all accounts, though, Clojure became as popular as it is because of the JVM, not despite it.

Btw, Rich had two versions of Clojure - Java (JVM) and .NET (CLR) and shortly before releasing it he decided to focus on the Java one, and deleted the .NET one.

ClojureCLR still exists and is somewhat maintained (last commit was last year, but then again, little has happened in the clojure world in the past year so I'd still use it): https://clojure.org/about/clojureclr

Right. What I was trying to express was that there have been 2 Clojures on the CLR:

1) The original one, developed by Rich Hickey in 2005-2007, which he deleted before releasing Clojure

2) The second one, known as ClojureCLR, which David Miller developed in 2008 from the ClojureJVM.


JVM is not to blame here, it's Clojure that takes a while to load and initialize. Java actually crushes Python in every aspect in terms of performance (which is to be expected of statically typed language).

GP specifically mentioned startup time. It's true that most of the startup time for Clojure projects is a Clojure-specific interplay of things the JVM does (but also not language-intrinsic: ClojureScript on V8 does not share that problem, for example), but default flags CPython still beats default flags OpenJDK for "start and do nothing".

> default flags CPython still beats default flags OpenJDK for "start and do nothing"

i stand corrected :)

it's not the static typing that makes a difference, it's the compilation. As an extreme example, asm is very much not statically typed (hell it's not even considered typed in most circles), and it beats the pants off of Java.

As another, less flippant, example, Julia will typically beat Java in most long-running algorithms (but probably not time-to-first-x), because it doesn't have a well-optimized-for-compilation-speed compiler, but it does have a better-optimized-for-running-speed compiler for many scientific tasks. https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

In that example, Julia also has someone who's spent many many hours writing and re-writing those few tiny programs to tune performance.

Last year I spent about twenty minutes writing mine (Julia #3) in the n-body example and it tied the naive java. I'm not even a Julia programmer anymore.

Thank you for that contribution. Since then many hours have been devoted to those programs.

I started out with that view too. Now, after many years of using Clojure, I have come to love the JVM: so many of the libraries it comes with have become critical pieces of my daily workflow.

GraalVM, ClojureScript, etc...

I find this very interesting, because it seems the programming landscape has changed quite a bit since Rich Hickey's world where he conceived Clojure.

How he describes information systems, in that world, JVM is an amazing piece of technology that can hardly be surpassed.

But today, the programming landscape goes far beyond that. ML and Data Science, infrastructure automation, serverless, etc.

I also work in the world of enterprise information systems, so I'm always surprised about people saying the JVM doesn't work well for them. For me the JVM is the best piece of tech out there, only matched by the CLR. Even Go isn't up to par yet.

But I reckon, other use cases value different execution characteristics. Low memory footprint is more important than raw performance. Throughput is more important than latency. Binary size is more important than portability. Startup times is more important than runtimes. Etc.

Well the good news is that Clojure is meant to be hosted. And the JVM is only one of its many hosts. ClojureScript can run on NodeJS, eventually Deno, it can run on JavaScriptCore and in the browser. All these offer alternative execution models which can meet a new set of requirements.

Clojure CLR is still real and robust, while it hasn't seen much use, it is ready at any time to take over. If the CLR provides to fulfill alternative execution characteristics it would be an option. Right now, the CLR has pretty much the same characteristics as the JVM though, which is why its use in Clojure hasn't seen a lot of popularity.

Babashka and Joker are Clojure based runtimes, interpreters really, so exactly the same as what Python and Ruby offer. Babashka especially is picking up pace, and already starting to offer a really compelling set of alternate execution characteristics, specifically one where startup time trumps all.

I'm also quite fascinated by Clojerl, a Erlang hosted Clojure. While less official, the maintainer is very active, and I feel it's actually now ready to be used in production.

And there are more, smaller ones, but I'll skip those.

What this shows is the landscape of possible runtimes is vast and will just keep expanding. Right now lots of experiment in having a Clojure hosted on Go, on Common Lisp, on Rust, etc.

This a testament to Rich's hosted design. Clojure is easy to build on top of other runtimes. This ease really helps it grow and reach more use case.

And there's the fact that the JVM is only getting better. GraalVM already offers native binaries with a small memory footprint, small binary, fast startup and still pretty fast performance. There are so many new garbage collectors coming along. Some for large heaps, small heaps, no heaps, tuned to throughout, latency, low memory, large memory, etc.

P.S.: Also, I really urge you to reconsider a JVM. The truth is, it's such a powerful piece of technology that it can be hard to navigate and understand even what all it can do. I believe a lot of people are turned off by that and choose to use Go or Python because they have only one easy mode of operation. But the JVM can do so much, it's a worthy investment, it's very possible it has what you actually need.

For example, you no longer need to install a JVM. Apps are self-contained. The user doesn't have to install a JVM, and the bundle contains only what the app uses, not the whole JVM. And as mentioned, compiled native binaries are also a possibility now.

You mentioned ML, so I just had to link to the awesome libraries by Dragan Djuric like ClojureCUDA, ClojureCL, Neanderthal, Fluokitten, Bayadera and recently Deep Diamond


Hopefully that'll come with GraalVM.

High praise indeed! Can you provide a source for it? It would be nice to be able to cite it.

I can't, sorry. This is my paraphrased recollection of his response to an audience question from years ago, and a quick search doesn't turn up any recordings.

Yea Clojure is like the Python of lisp style languages. Good language design and it can interact with Existing Java libraries. (Python does the same thing but with C).

> Python does the same thing but with C

Don't think that's really accurate.

You don't have to use C with Python, and there's no VM to worry about. CPython* just happens to be the most popular implementation.

Clojure (by and large)* requires the whole Java infrastructure, and that is a nonstarter for many.

> Clojure requires the whole Java infrastructure, and that is a nonstarter for many.

There are Clojure implementations that run on JS runtimes, the .Net CLR, the Erlang BEAM runtime, etc.

It requires jvm. But you don't have to interact with java, sometimes its easier to just use a particular java library, but many people report never having to touch java. JVM seems like a nice to have, a lot of the standard java tooling works for clojure as well (profilers and what not).

Nit: CPython, not Cython. Cython is a whole different thing that leverages CPython to compile Python programs to C, as well as compile Cython-the-language, effectively annotated Python, to faster C.

CPython comes with its own VM: are you saying the issue is "you need to install 2 things instead of 1"?

Common Lisp also does the same thing with C.

And Hy does the same thing but with Python.

I think, though, that the things that Clojure gets right go way beyond just the JVM, though. Clojure's approach to nil punning, for example, is really well thought out.

"nil punning" has been a feature in Lisp since the earliest days, but as far as I know it's only called "nil punning" in the Clojure community.

I don't have anything against Clojure (well, I'm a little negative on the JVM), but I personally don't see any reason to move to it from Common Lisp.

On the other hand, I'd miss CLOS, and I feel like CL has a better/more consistent design. Something about Clojure always feels like they're "winging it".

nil punning?

Eric Normand explains it better than I could: https://lispcast.com/nil-punning/

In a nutshell, nil punning is a thing in lisps (and some other languages) where the null value can have different meanings in different contexts. It's comparable to Python's concept of "truthiness" and "falsiness", but more so.

Oh that, yeah that's what I'm used to from Common Lisp as well.

A great quote from the introduction, on why Rich took a sabbatical to work on Clojure:

...to give myself the opportunity to work on whatever I found interesting, without regard to outcome, commercial viability or the opinions of others. One might say these are prerequisites for working on Lisps or functional languages.

The results are also astounding, because what he came back with produced extreme commercial viability, and is excellent in the opinions of others.

Having used Clojure professionally for a couple years now, I feel so blessed. It is really well cut out for a whole host of problems that are common but extremely labour-intensive in conventional languages.

Maybe one of these days I'll get time for my own sabbatical, and solve the two major tasks I see with Clojure: taking the performance from good to excellent; and hosting it in a robust systems language suitable for implementing well-trodden functions.

Yeah, share both common concerns, centering around the JVM (which I love, and have used and studied since it launched, but it is baggage).

There have been a number of attempts to build Clojure on top of Go (I worked on one), and another on top of Rust. Both lack the dynamic runtime ergonomics that Clojure leverages. Clojure is also seeing significant performance benefits, both runtime and startup, from the Graalvm, but that also will have limitations.

I am wondering about D, which should be sufficiently low level from a systems perspective but may also have sufficient dynamic capabilities.

I think the problem with Go is that it is invested in some oddities that are of a similar scale to Java oddities. For example, from my ignorant perspective, Go's multiple return pattern for error handling seems to have turned out in the long run to be a much bigger mess than Rust's Result.

AFAIK Rust ditched the optional GC long ago, but I think a lot of the infrastructure that has enabled async/await, in the way of hooking in a “runtime” through attributes and macros, could help with building a GC into a rust host. I think this could be a special strength if you combined the futures infrastructure with the allocation logic, maybe with a local spill/scratch pseudo-heap for allocations that don't escape.

Think ztellman's aleph and manifold interfaces, but based on tokio, with some of the allocation magic (or deallocation magic, depending on how you slice it) of BEAM.

I think I've been a bit let down by some “Clojure-inspired” languages which do not have HAMT or CHAMP datastructures, or do not have good implementations of them.

The JVM's profusion of high-quality garbage collectors is another thing that is hard to compete with; though I think there are massive opportunities with a collector similar to Shenandoah 2.0, if the allocator can be aware of the properties of Clojure.

Furthermore, a purpose-built runtime would be an amazing way to explore transparent non-volatile memory. I think there is staggering value in transparent access to datastructures that are significantly larger than memory, especially if that access is inside a future/task run by a NVM-aware executor, and can wait safely. Think MapDB, but persistent, immutable, indistinguishable from the standard Clojure datastructures, and integrated with the garbage collector.

Of course, as I've not written anything significant in Rust, I will shut up about it beyond this, because there's nothing more obnoxious than people extolling the virtues of something they've never accomplished anything significant with.

Heh, I actually like the "comma-ok" pattern and use it in Clojure.

IMO one of the areas of abstraction leak in Clojure is around error handling. Most of the time the options- out of band and out of code path, like Exceptions, and in band, like null- don't sit well with me. I usually want in-band, explicit, with the capacity for some sugar so if necessary the error cases can be organized in one place. Comma-ok does this for me without any magic, and I would like to see it as a lowercase-p protocol integrated into things like the threading macros (there has been some work in this area...)

But high level, comma-ok is just data, not a type- another area of abstraction leak.

I hear the interest in arriving at a more efficient GC but don't have much current background in that area. I did a lot of GC work at the configuration level with the JVM, and read a bunch of the source a decade or more ago, but recognize that's not my area. I similarly spent a bunch of time in the linux kernel world 15 years ago, and still check in on the mailing list from time to time, but that's a different life.

Agree with non-volatile mem opportunity, though note that was the intent with virtual memory architectures and then CPUs went and ruined it with the cache hierarchy. From a performance perspective if your code is hot, cache-aware will noticeably outperform non-cache aware. Figuring out ways to make this automatically efficient at runtime in the context of also supporting GC would be a fascinating PhD project.

Appreciate the engagement, cheers.

It's worth mentioning that there are a few options for light weight Clojure dialects nowadays





>There have been a number of attempts to build Clojure on top of Go (I worked on one)

Do you have a link? The only one I'm aware of is https://github.com/candid82/joker.

Believe it or not, this is relevant to an ongoing sabbatical project of my own. I'm grossly unfamiliar with Java, so it's often hard to read through the Clojure implementation :/

I worked on one privately back around 2014, then later picked up on some of this work: https://github.com/venantius/glojure, done by a friend of a friend. Neither one got anywhere useful. I more became convinced that to get to a useful place it would be necessary to generate (and compile) code and although Go has some of that tooling, I did not have the appetite to build with it.

The Java implementation implementation is hard to read even for Java folks, it is more of how a C++ programmer would write Java, both in terms of syntax, and semantics.

Cheers! I’ll definitely give this a read-through.

I'd love to use a light-weight Clojure hosted on Go, etc., but also wonder whether performance improvements to the JVM will solve many of these issues without requiring a full rewrite.

Many of the issues Clojure has with the JVM are shared by regular Java apps, so I think there is motivation to improve many of these areas over time, even if it takes a bit.

>I'd love to use a light-weight Clojure hosted on Go, etc.

You may find this interesting: https://github.com/spy16/sabre

It's still pretty green, but it might float your boat.

"hosting it in a robust systems language suitable for implementing well-trodden functions"

Maybe take a look at Joker [0] to start. A Clojure interpreter written in Go.

[0] https://github.com/candid82/joker

A similar point was raised in his Hammock-Driven Development talk. It inspired me to follow suit during this COVID19 lockdown.

I've never been more intellectually fulfilled, nor happier.

For working on the languages themselves for sure. But thankfully working with functional languages helps with outcomes and commercial viability.

If only languages like Clojure, F#, OCaml, Kotlin, and many more enjoyed more popularity and support.

I've been working with Clojure for a few years now, and the thing I love most about the language is how it nudges developers to write straightforward code. It's maybe closer to a well-structured procedural style than to highly indirect OO or to deeply polymorphic category-theory-inspired typed functional programming. The emphasis on mostly-first-order pure functions composed of other functions, receiving and returning immutable values, makes it easy to untangle even the worst balls of mud, while the well designed set of data-structures (few) and functions operating on them (many) allow for easy and terse expression of everyday data manipulation chores.

If we keep the majority of the code pure and straightforward, we can deal with state and I/O with other constructs, like atoms and records, only where we need to (doing this systematically tends to lead me to some variation of the functional-core/imperative-shell pattern). STM, multimethods, macros and whatnot are cool and have their uses, but after a while I think the really cool thing is how little do we need to resort to fancy stuff and how much is accomplished with just functions and data.

> functional-core/imperative-shell

Lol I’ve been learning Clojure for fun these last few days and I had to read this twice because the first time I thought I was looking at a function from a module.

> we can deal with state and I/O with other constructs, like atoms and records

Can you elaborate on what you mean here or link to specific articles? I’m talking about the representing I/O side effects as records part.

There is a component framework that builds dependency tree for stateful modules and does implicit injection of params. It is a beautified pattern(usually you can do initializations in higher levels, but you have to resolve dependiences yourself and for the whole tree) for dealing with things that require state while keeping explicit references(so that you understand where it came from and can inspect(visually) its lifecycle).

Sure, check out Stuart Sierra' (by now famous) blog post Clojure Workflow Reloaded and his Components library: http://thinkrelevance.com/blog/2013/06/04/clojure-workflow-r... https://github.com/stuartsierra/component

> When I first was learning Common Lisp I was appalled at the tolerance on Usenet comp.lang.lisp of arrogance, hostility, intimidation and other anti-social behavior of the presumed ‘smartest guys in the room’. I was determined (a) that the Clojure community would not be hosted there and (b) that kind of behavior would not be tolerated.

Thank you, Rich Hickey!

The persistent data structures and functional core library are great, but many would not bother with Clojure if they had to deal with the toxicity that is present in the Common Lisp community.

Comp.lang.lisp was awful, and the trend somewhat continues even though it has gotten better. Looking through newb threads on various lisp forums there are a few people that always seem to try their best to be mean or funny on someone else's expense.

It is not that hard: if someone writes awful code, point them in the right direction. Don't claim they have malfunctioning brains.

I stopped writing CL a few years ago even though I enjoyed it. For my personal stuff scheme is even more fun, and the communities are a lot more pleasant.

I was super excited to see this paper released. I've spent time with Python, Java, C#, Javascript and limited time on C and Rust. Clojure was the largest paradigm shifter by a large margin.

Having done concurrent programming in Java, and even implemented some concurrency primitives like semaphores, Clojure was incredible to use. For those that don't know, Clojure uses immutable data structures that mitigate concurrency issues. If you are curious how that's even practical, I highly recommend the paper and also reading about Persistent Data Structures. They reuse shared data, similar to how Git only tracks the delta's between commits, rather than multiple, full copies of the file.

On top of that, you up reducing lines of code by 60-70% of traditional Java or C#.

> They reuse shared data, similar to how Git only tracks the delta's between commits, rather than multiple, full copies of the file.

I learned about immutable data structures at Rich's conference talk years ago, and the whole concept blew my mind as well. However, what you said above is not strictly correct. The idea is not to store deltas (which is what Subversion does), but immutable copies of objects (in case of Git identified by their SHA-1) that can be used to build multiple versions of a hierarchical data structure (e.g. a list, tree or hash map). Git commit is basically a tree of other objects with some metadata. The problem with deltas is that they need to be iteratively applied to some base snapshot which requires computation (and probably a cache as well), while immutable objects are easily accessible in constant time.

Agree. Probably should’ve used a more solid example, but was just trying to convey the essence I suppose.

No worries, I fully agree with the rest and just wanted to clarify things for others, not nitpick.

Thanks to Clojure (and ClojureScript), I was able to build my self-funded SaaS business. There is no way I would have been able to tackle the complexity without Clojure.

I also love the mature approach and mature community. It is, quite simply, a different world from most other software places, and one I enjoy a lot, as I'm mostly focused on building and shipping a large and complex codebase with an extremely small team.

Can you elaborate on how Clojure helped you tackle the complexity? I've considered learning Clojure (I have the Brave Clojure book sitting not far from me) but I have concerns about the learning curve (I'm a fullstack dev by day (C#/Angular) slowing me down in my own attempt to build a SaaS.

Well, off the top of my head:

* a single language for both client-side (browser) code and server-side code

* a single data representation (maps with namespace-qualified keys), so no conversions necessary

* spec which helps validate data in multiple places (such as :pre/:post conditions)

* core.async which lets me write asynchronous code both in the JVM and in the browser, same primitives

* a library of excellent sequence manipulation functions

* transducers, which let me build composable data pipelines

* the Rum library which lets me use React as a rendering engine, basically a function of my data

* most of my domain code is shared between the browser and the server

There is more, but these were what I could come up with immediately.

I mostly spend time thinking and working on the problem domain, not writing boilerplate (there is none in Clojure, really).

I think we might be clones. My self-funded business (https://operatr.io) is built in Clj/Cljs.

From the product, to the static website, marketing emails, even the licensing runs Cljs on Node in AWS Lambda.

I would have listed those same bullet points off the top of my head.

That sounds pretty great. Thanks!

I have a similar background and in Oct 2018, I started spending lots of time learning with that exact book! I first learned Emacs (covered in the book) and it really wasn’t bad at all. I did some tiny fun projects in Clojure, and some exercises on 4clojure. After a couple of months I understood the language and could easily make sense of most of what I saw except core.logic which hardly comes up in practice. Even Heroku has first class support of Clojure, so deployment is trivial for MVPs.

You likely won’t have an issue learning the language, but from my experience, making full stack apps in Clojure is a bit more challenging (compared to .NET and Flask). It’s challenging to figure out what libraries to use for your app, since frameworks are hardly complete and usually outdated in Clojure.

I see nothing but praise for Clojure and for good reason. I experimented with it a bit a few years ago and it seemed to be a more practical and useful version of Lisp, although I say that as someone who wasn't that experienced in Lisp more generally, with only some small experiments in Scheme. Something just as significant was that the ecosystem seemed to be quite sane and I got on really well with Leiningen for example, if you were considering diving in then give it a go.

I personally won't consider it now but not because of the language or the ecosystem but because of the runtime(s). For me the layers of abstraction are too much complexity to not be justified.

It seems that rejecting all of the most popular runtimes in the world (JVM, JS, CLR) is an odd thing to have a grudge about.

(Also notable, babashka, a Clojure alternative to bash.)

I love clojure the language but hate the tooling.

Also a lack of documentation was always a problem, especially around clojure script.

I also don't like smug community. When you say about the lack of documentation and get told that's a good thing, clojure is meant to be hard to learn, it's not for everyone and maybe you just aren't smart enough...

Hmmm. That attitude might put people off.

I kind of know where you are coming from. 'Back in the day', early Clojure adopters would ask one to read SICP before asking questions. SICP is indeed the best CS book I have read but that attitude was uncalled for.

But things have changed. Clojure and Clojurescript community is among the most helpful and Stackoverflow questions are promptly answered. If you take the time , like I did, you will find that Clojurescript has awesome tooling. Go though the tutorial on Figwheel.main (https://figwheel.org/docs/ ) and then check out re-frame TodoMVC here https://github.com/day8/re-frame/tree/master/examples/todomv... or this simpler reframe tutorial from here https://opengisgal.wordpress.com/2018/05/19/re-frame-clojure... . And this here is the Reagent cookbook https://github.com/reagent-project/reagent-cookbook . Good Luck!

"'Back in the day', early Clojure adopters would ask one to read SICP before asking questions."

I was in the Clojure usenet from very early on and do not recall this happening. That's not to say that it never happened but it was certainly not a common response to questions. Instead, I remember early adopters like Chris Houser, Chas Emerick, Phil Hagelberg, and many others taking a lot of time to help people who had questions with the language.

I've also received countless hours of help from @alandipert, @micha and @jumblerg on the Clojurians slack about boot, Hoplon and Hoplon UI. Without them I couldn't have successfully built a 6 ppl Clojure team, who was consistently delivering working software!

> When you say about the lack of documentation and get told that's a good thing, clojure is meant to be hard to learn, it's not for everyone and maybe you just aren't smart enough...

Clojure is not meant to be hard to learn and you are definitely smart enough. There are many, many learning resources for Clojure in every form and it has one of the most helpful communities I'm aware of on Clojurians slack, etc.

> I also don't like smug community. When you say about the lack of documentation and get told that's a good thing, clojure is meant to be hard to learn, it's not for everyone and maybe you just aren't smart enough...

Where did you experience this? That is unacceptable in most of the Clojure communities I know, and the only big one I don't know personally is the subreddit.

(I am not suggesting the subreddit is toxic.)

Compared to many other subreddits, the Clojure one is frolicking unicorns and rainbows.

The best thing is that keeping it that way takes a minimum of moderation.

What tooling has given you problems? I think that some tooling assumes familiarity with the Java ecosystem (which is a problem for many things in Clojure land), but otherwise it seems pretty solid to me.

I have posted about this before but Clojure does have tooling problems specifically ergonomics.

Look at create-react-app. Two commands and a ton of editor integration across intellij, vscode, vim, etc gives you incredible access to a rich ecosystem and plugs together really nicely with sensible defaults.

Clojure has some answers in this space but everything feels bolted on and not nearly as polished. A big part of the problem is the size of the community, there just isn't enough hands to build out the infrastructure that other langs enjoy.

I still love Clojure and will continue to use it but in order to do so I have to understand that some simpler things will just be more work.

That might be because you've left out CIDER, which is

1) the most popular Clojure programming environment

2) solidly documented

3) built on top of Emacs, which is in itself very well documented (Just the reference manual of the vanilla Emacs is 500 nicely written, if dry, pages)

4) very featureful.

I guess the biggest problem is that most people are not familiar with Emacs, and not willing to read any documentation and guides.

I've never understood why so many people are reluctant to learn Emacs (or Vim), but then don't see any problem with learning a new language specific IDE every few years.

Probably because those language-specific IDEs all follow standard user interface conventions.

While we can suggest Cider as an option, I have had nontrivial problems with breakage on updating, Emacs falling over with long lines of text, Windows support, etc. I also know that I am not the only one who has had these problems https://twitter.com/ztellman/status/1055152840589496320.

These kinds of issues a part of the tooling gap and are real reasons people give up on Clojure and I do find the dismissive attitude of most when I have suggested this in the past off putting.

Addressing Emacs specifically, while it would be probably worth the investment it is not an answer most people want to hear and when we suggest that Clojure is this great thing it is important to show people on some of their terms because learning a new editor on top of Clojure more then we should be asking IMHO and for all the talk of Clojure being pragmatic this feels directly opposed to that ethos.

Let me emphasize, I don't want to say anyone is doing anything wrong or that Emacs isn't the right way forward but it had to be acknowledged that there are real problems that path that need improvement when it comes to onboarding new developers and with daily use.

In 6 years that I've been using Cider daily, I don't remember it being broken after update. Ditto for Emacs.

A few times Cider had major changes, but even then it was quickly resolved, and was relevant only to people that were using daily snapshots. I guess if someone is a beginner, they'd use a stable version, and these are rather stable.

Big files could be problematic in Emacs. OTOH, Clojure files are never big.

As for Emacs requiring a bit of learning at first. Well, yes. It has lots of features that beginners have probably never encountered in other tools. These features are awesome. They also help a lot. So, someone expects to unlock a great featureset, but refuses to spend even a few afternoons learning about that featureset. I understand that people would rather get something out of nothing, but it's not Emacs's fault.

Part of the problem could have been that I was using Spacemacs and package support and that may have broken it but circling back around this connects with meeting people halfway, I don't want to learn Emacs style control, I know Vim, Vim works elsewhere, and I just want to write Clojure. I can do that now with Cursive so the trade off isn't worth it. A lot of people learning feel and think this way in my experience, especially if I selling Clojure to someone who is reluctant, anecdotally at least 4 people I know have followed this path, I am sure it extends beyond them.

Per the big files, its not so much Clojure but output at the REPL. Especially as a noob accidentally grabbing way to much data for a buffer is really easy, Emacs hangs, REPL and process dies and has to be rebooted and you have to bring up everything again. This is non-trivial friction when learning and should really be handled gracefully by the editor.

As for everything else regarding Emacs and learning all these points stand and are valid but still don't address the problem of someone wanting to learn Clojure and instead having to learn Emacs + Clojure + Cider. For someone just starting on the path most won't have more than 3 files of CLJ and may not even interact with Lein or Deps. Refactoring IS important but not to them. I just don't think the trade off is worth it for someone just toying around with CLJ in their free time. It may not be the best metric but "time to hello world" and "time to blog" matter to people coming in because it shows that the promise that they are sold is something that they can reach and achieve.

Hm, I use Spacemacs particularly because of the easy config.

That said, you're right: you should just be able to use vim, and historically CIDER has been the dominant (most polished) environment. At Clojurists Together Foundation, we're trying to fix that though: a lot of our funding has gone towards IDE development including for vim. Some of it towards CIDER also benefits vim because they share internals.

[0]: https://clojuriststogheter.org

It's worth mentioning that BBatsov, the creator of CIDER, maintains a practically zero-config Emacs distribution, Prelude. A novice interested in Clojure could install Prelude, and start coding Clojure and other popular languages using pre-configured packages in, let's say, 15-30 minutes. The difference from Spacemacs is that Prelude is much simpler, does not mess with Emacs defaults, is close to vanilla Emacs, and has been super stable for years.


aaand I guess that's what ppl mean by "ergonomics issues". Emacs is very stable, quite logical in some sense, but also hasn't kept up with the times and doesn't default to those typical keyboard shortcuts, which most newer editors almost all share... but this has been discussed countless times before on twitter, clojureverse.org etc...

My bet would be that you are working with the Linux version of Emacs. That's indeed a lot less problematic and more performant too. Under macOS and Windows, the situation is not that rosy and since ppl want familiarity, they typically mix in something like Spacemacs, Doom Emacs, etc into the mixture. For a long time they would probably won't even hear about Prelude or if they do, they would not even assume that's a preset system for Emacs. I've seen this happening to a couple of ppl already just in the past 2-3 years.

I'm not super familiar -- what editor integrations does create-react-app provide? It seems like it's just a project template like Luminus for Clojure.

Tooling is definitely important, so I don't want to downplay friction here, just curious about what other ecosystems do better.

Mostly that there is an interplay between the config that gets spit out by CRA and the editors that consume it, I don't need to give intellij the run config for the 400th time, it's just there by default. I click run and it runs.

An exercise for you to try that might illustrate it. Setup create-react-app and get hello world up and running. Now do the same with Clojurescript. That should include a way to hot reload and edit because that is often the next steps.

Notice the friction that CLJS still has and the aimlessness of starting from a build script as is suggested on the Clojure site vs getting a functional build in two commands.

I don't want to come across as ungrateful for the hard work that David and Alex and the whole team put in, they are the only reason have dev tools that feel comfortable to me and I can't thank them enough for that, but I just want to point out that friction and difficulty and to try and get people who do understand to put themselves in the shoes of developers who don't and to acknowledge that we as a community can aspire to more.

I am not familiar with CRE, so I could be missing something, but based on my understanding of what you write, `lein new re-frame hello-world` should do the trick. Do `lein dev` from the project root directory to get the app running with hot reload.

npx create-cljs-app will get you a ClojureScript React app pretty similar in tooling.

Just tried this and the install size is 796Mb of which 513Mb is node modules. The initialised git repo is a whopping 274Mb - WTF? I re-initialised git, excluding node_modules and it's now 244k. I recommend the author leaves `git init` to the user.

Is your criticism that it downloads to many packages, or just that they're (I agree, incorrectly) being added to the git repo? Because if you want to create your own minimal setup, shadow-cljs' tutorial will help you do that: but OPs criticism was that there's no easy starting point that has all the bells and whistles, so yeah, this has a lot of bells and whistles :-)

That is extremely fair but again it doesn't have deep hooks and the convenience that Javascript and CRA have with, for example Intellij. This isn't a problem that is easily solved, Colin is only one person and JetBrains is a huge team, but it is still a gap that Clojure needs to have a story around to attract more developers and to make our lives as devs starting new projects easier.

Can you help me understand the deep hooks that need to happen? My understanding is that it's stuff like, intellij's "run" knows how to call `npm start`. Elsewhere in the thread you highlighted specific requirements like getting hello world running and hot code reloading with your editing, but CCA seems identical to CRA in that regard.

How is create-react-app more ergonomic than "lein new"?

There isn't enough officially supported first class tooling. Most things that are a given for Rust/JavaScript/Go are community maintained with only one maintainer in Clojure, development is slow and features take a long time to roll out. (Let's not delude ourselves that Webpack level of donations is even possible here) This is a problem with every language that is small but it is particularly problematic with Clojure as it has a commercial company behind it but you get the feeling that they are disinterested in promoting the language and ecosystem beyond selling their database software. Go also espouses "frameworks bad, libraries good" but the biggest difference is that they actually have a strong robust ecosystem of solid libraries while pretty much every Lisp claims to have production-ready libraries but expect you to cobble together the entire kitchen sink from scratch.

> development is slow and features take a long time to roll out

As Rich has said, Clojure is small and intends to remain so. Clojure has everything you need right now. It doesn't need gobs of new features to be sufficient. Most Clojurists are hard pressed to come up with things they really need in the language itself, so the rate of change in the core language has slowed. (And the rate of change in the tooling and libraries around Clojure gets more attention now.)

> they are disinterested in promoting the language and ecosystem beyond selling their database software

... is incorrect. We have every desire to promote Clojure as a language (regardless of Datomic). Could more be done, sure that's always the case. Please help promote Clojure by doing great stuff and telling people about it!

I love Clojure.

I also believe that tools.deps is a step in the wrong direction, fragmenting an already small community and forcing all tools to support yet another dependency/build system.

Leiningen is super powerful, and it was never made clear what kind of conflicting requirements / philosophy led to the birth of tools.deps etc. In absence of an official answer, the presumed answer is Not Invented Here Syndrome.

> In absence of an official answer, ...

Wouldn't you consider this and official answer?: https://github.com/clojure/tools.deps.alpha#rationale

That documentation does not explain what prompted the creation of another tool when leiningen was already (and still is to a large extent) the de-facto deps/build tool.

> Go also espouses "frameworks bad, libraries good" but the biggest difference is that they actually have a strong robust ecosystem of solid libraries while pretty much every Lisp claims to have production-ready libraries but expect you to cobble together the entire kitchen sink from scratch.

I mean, Java is pretty much the biggest ecosystem there is and it's trivial to use Java libs from Clojure, so I'm not sure this is a fair comparison.

The Go ecosystem seems pretty weak outside of the standard library. I'm not an expert, but I was trying to fix something the other week and was shocked by how few packages there were to solve my problem. The main one (which is a dep of things like k8s) is currently unmaintained.

Clojurians slack is honestly one of the most welcoming and understanding programming communities I've had the pleasure of joining. I frequently ask stupid questions and will occasionally have library maintainers take their time to fill my gaps in understanding. The tooling complaint is valid, but it has been getting better every year. There are plenty of good books for Clojure too, like Brave & True that help you understand the documentation.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact