
Why did Clojure gain so much popularity? - yogthos
https://www.quora.com/Why-did-Clojure-gain-so-much-popularity/answer/Ag-Ibragimov
======
throwawaywlabs
I work at Walmart Labs. Clojure is super popular[1] in Walmart Labs. Clojure
is fun and importantly it is a very productive language to work with. This
gain in productivity comes from less state, thus easy to debug and fix, and
functional approach, so the functions do what exactly what they are supposed
to and there are no hidden surprises.

Less state or immutability means concurrency is easy. Easy concurrency means
we can scale[2] applications with ease. I mean ease of development. It takes
less work, less mental effort to take something that works for a single thread
and make it work in a multithreaded and/or distributed manner over hundreds of
thousands of cores. Debugging is easy too. P1 incidents for a microservices
written in Clojure takes less time to debug compared to those written in other
languages. So no wonder most of our mission critical services are written in
Clojure.

For a complex environment where thousands of microservices are deployed on
hundreds of thousands of computes, Clojure has been super successful.[3]

[1]:
[https://github.com/walmartlabs?language=clojure](https://github.com/walmartlabs?language=clojure)

[2]: [http://blog.cognitect.com/blog/2015/6/30/walmart-runs-
clojur...](http://blog.cognitect.com/blog/2015/6/30/walmart-runs-clojure-at-
scale)

[3]:
[https://clojure.org/community/success_stories](https://clojure.org/community/success_stories)

~~~
didibus
I love how you mentioned that Clojure is fun!

We use Clojure at my work as well, and ya, everything else you said afterwards
is true as well. Clojure is a great productive language. It can scale, is very
comparable to Java in performance, simple to maintain, has a complete
ecosystem, is batteries included, etc.

But most convincing! It's just so much fun!!

Being fun is an undervalued quality of a language in my opinion. That joy you
get from using it actually translates in the quality of the work you do. So
it's not an intractable thing, the fun is good for you and for the product!

~~~
yogthos
This can't be said enough. At the end of the day programming is done by human
beings, and it needs to be treated as a human endeavor. If I enjoy what I'm
doing I'm going to do it better, it's really that simple. There is actual
science to back up this idea as well
[http://neverworkintheory.org/2014/05/01/happy-sw-devs-
solve-...](http://neverworkintheory.org/2014/05/01/happy-sw-devs-solve-
problems-better.html)

------
room271
Clojure is particularly well suited - at least Rich Hickey would claim - to
real world applications with unsafe or unstructured inputs. I.e. complex
programs that receive or pull data from external and loosely-typed sources. A
lot of web programming falls into this category. Why is it well suited to
these? Because it is dynamic, and offers rich runtime solutions to validation
(Spec), has immutable data structures, safe and relatively simple concurrency
primitives, and has the expressivity of a lisp. It also has access to the vast
array of JVM libraries which helped early adoption (but possibly hindered it
in the longer term?!).

~~~
cutler
Although I love working with Clojure more than any other language it's
relationship with its host is somewhat schizophrenic. Calling Java classes
from carefully-crafted functional code is like pouring tomato ketchup over a
lemon sorbet. Thankfully there are about 23,000 pure Clojure libraries at
clojars.org to help my Clojure code stay functional.

~~~
stingraycharles
It depends on what your requirements are. Through the years, I found it easier
to write reliable code by just skipping the Clojure wrapper libraries (e.g.
using Kafka directly rather than a wrapper): it's easier to "escape" into
functionality the wrapper doesn't offer, and it's typically more performant as
well.

------
iagooar
To be perfectly honest, Clojure's biggest mistake, from my point of view, was
to be so heavily JVM-dependent.

Yes, in certain environments Java is king and a ton of people use it, it's
battle-proven, etc. But so are other stacks.

Clojure, as a programming language, sparks a lot of joy and is a lot of fun to
use and reason about. Yet in reality, when working in a production
environment, you're basically fighting the always-memory-hungry JVM.

Just give me Clojure running on BEAM and I'll take it over anything in this
world.

~~~
hota_mazi
I think it's the other way around: Clojure gained more exposure than past
Lisps because it runs on the JVM.

But it's still definitely niche and hardly used anywhere.

~~~
brightball
I think this is accurate (in terms of exposure, not that it's hardly used). As
soon as it was possible for other languages to run on the JVM, the huge amount
of developers working around JVM infrastructure that didn't like the sometimes
painful experience of working with Java, started looking at alternatives.

Groovy, Scala and Clojure were the early contenders that people were exploring
and Clojure seems to have won a lot of mindshare.

The thing about Java isn't the language or the JVM it's the App Servers.
Enterprises have invested heavily in the App Server infrastructure for
deploying JVM based code from monitoring, training, etc.

Throwing all that away to use a non-JVM language is a REALLY hard sell. Those
app servers are very well done.

Running on the JVM made using Clojure, Scala and Groovy possible in places
where other languages wouldn't have been. In the early 2000s, it was Java and
everything else.

The .NET ecosystem was awful at that time. The programming world was
essentially, Java, Perl and PHP 4 in terms of enterprise usage. People said
the words "Enterprise Java Beans" without a chuckle.

People were HUNGRY for better options.

Now you've got a solid C# ecosystem, Elixir, Go, a dominant Python ecosystem,
significantly more polished PHP, K8's & Docker that make automating the
infrastructure around all different languages consistent...it's just a
different world.

~~~
joshlemer
>Groovy, Scala and Clojure were the early contenders that people were
exploring and Clojure seems to have won.

I love Clojure and hope it gains more mindshare, but I don't think this is
accurate. Scala currently enjoys quite a lot more industry use than Clojure
does, for instance there are currently roughly 15x as many postings mentioning
Scala as there are Clojure (source: Linkedin.ca).

~~~
cutler
Same in London. Search indeed.co.uk for "title:Clojure" and there are on
average no more than 6 unique results. Outside London the rest of the UK shows
next to nothing. About 6 years ago Clojure dojos and meetups were well-
attended but even then the numbers were a fraction of the equivalent Scala
events. Nowadays you're lucky to get a handful attendees at London Clojure
events which is a real shame as it suggests Clojure is disappearing into a
very small niche.

~~~
ilikehurdles
Yes, it’s niche, but remove “title:”.

------
krn
Interestingly, Clojure is associated with the highest salaries worldwide[1],
and is used by developers with the most years of professional coding
experience[2], according to the StackOverflow Developer Survey 2019.

[1]
[https://insights.stackoverflow.com/survey/2019#technology-_-...](https://insights.stackoverflow.com/survey/2019#technology-
_-what-languages-are-associated-with-the-highest-salaries-worldwide)

[2] [https://insights.stackoverflow.com/survey/2019#developer-
pro...](https://insights.stackoverflow.com/survey/2019#developer-profile-_-
years-coding-professionally)

~~~
dan-robertson
These surveys seem (increasingly) biased towards certain types of development.
Specifically, web development. I couldn’t find a single language I regularly
or irregularly use on this page (the closest thing I could find is Emacs).
Also the sample size for Clojure developers is smaller than other languages.
It’s not obvious to me why the bar should be raised at a certain point (e.g.
letting in slightly niche Clojure programmers but not other more niche
things).

I don’t want to imply that these surveys are rubbish, just that I do not think
they give the whole picture

~~~
innagadadavida
The Quora article mentions that clojure took off in the fintech industry.
People get paid a lot in this field, so if there is some way to control for
this niche, the stats will be more accurate.

------
paultopia
Absence of syntax! I really wish every language were a lisp. Like, Rust, the
lisp, Swift, the lisp. Who needs rules of precedence and associativity and
statements vs expressions and all the other stuff that languages that aren't
lisps impose on you? Just stick an expression in some parentheses! And then
put that in another expression!

~~~
lmm
(find (most people) difficult (without (infix operations) (to expressions
write)))

~~~
ken
In 99% of programming languages, infix operations are only used for arithmetic
-- and more than 99% of the programming I do doesn't involve that. Even the
numerics software I've written is >90% non-numeric parts.

Arithmetic doesn't even look _right_ in programming languages with infix
operators. Go to the "Chalk is still one of the primary tools of
mathematicians" article here, and try typing the inequality in Java.
Mathematicians aren't writing "Math.floor()" and "% 2" on their blackboards.

I think you're optimizing for the wrong thing. Arguing that we need infix
operations for 1% of our programs, so that arithmetic looks like how we
entered it on a TI-81 or Apple II, is a false economy. It was a good tradeoff
25 or 50 years ago, but the majority of programming has moved past writing
programs in terms of arithmetic on registers.

That said, there is a nifty macro [1] which adds infix support to Clojure, if
you really want that. It even supports symbols like "√", unlike any other
language I can think of!

[1]: [https://github.com/rm-hull/infix](https://github.com/rm-hull/infix)

~~~
lmm
> In 99% of programming languages, infix operations are only used for
> arithmetic

I've been spoiled by Scala, but certainly most languages allow a
"subject.verb(object)" style for general-purpose code, and that reads a lot
better (particularly for those looking from the business domain) than "(verb
subject object)".

> It even supports symbols like "√", unlike any other language I can think of!

Heh, again I'm spoiled by Scala, where √ is just another method/operator
(there's no distinction) name if you want it to be.

~~~
ken
> most languages allow a "subject.verb(object)" style for general-purpose
> code, and that reads a lot better

Why do you think this?

Programming isn't natural language, and I don't even know what the "subject"
or "object" is in most cases.

~~~
lmm
> Programming isn't natural language

It's not, but it should be close; readability is important, and particularly
when you want to express domain logic it's important to be able to hew closely
to the actual language of the domain. Arithmetic is the most obvious example
of this but it's just as true for other domains.

~~~
yogthos
You're right readability is important, and Scala fails terribly in that
department. I worked with Scala for around 6 months or so at one point, and I
was still constantly looking things up after months of using the language.
There's simply too much syntax and weird quirks in the language that it
becomes impossible to keep it all in your head.

~~~
lmm
Not my experience - I found Scala's syntax is a lot more consistent than most
mainstream languages (though, yes, it certainly doesn't go as far as lisps).
Operators are just normal methods. Rather than complex rules about which
constructs require braces, everything permits an expression or you can put a
braced block of statements anywhere you could put an expression. Rather than a
bunch of different special case syntaxes for collections / async / possibly-
absent values, you have a single lightweight syntactic sugar (for/yield) that
translates directly into plain old method calls. There are some libraries out
there that make poor choices of method name, but that's about all.

~~~
yogthos
My experience with Scala matches this pretty closely
[https://www.reddit.com/r/scala/comments/4246qc/heres_why_sca...](https://www.reddit.com/r/scala/comments/4246qc/heres_why_scala_sucks/)

~~~
lmm
That's a strange set of complaints for a clojure advocate, since most of what
they're complaining about is not too much syntax but too little - no mandatory
braces, mandatory brackets, mandatory types. Indeed "a -> b" for "(a, b)" is
not syntax or even a macro, but just a function.

_ is genuinely costly syntax, as are by-name parameters (though less costly in
an age of IDEs). I consider _ to be worth its while (indeed I miss it a lot
when writing lambdas in any other language); if you're going to have syntax in
your language at all then it's one of the most general-purpose, effective
constructs I've seen.

~~~
yogthos
The problem isn't whether there's too much or too little, it's that the syntax
is incredibly inconsistent. There are a million ways to do anything, and you
have to juggle it all in your head. Consider stuff like this:

[https://twitter.com/pembleton/status/1116343189437394944](https://twitter.com/pembleton/status/1116343189437394944)

A new line can be significant syntax in Scala!

With s-expressions, all of this mental overhead goes out of the window. I have
regular boring syntax where pretty much everything works the same way. It's
incredibly liberating. On top of that, s-expression syntax facilitates
structural editing, where I'm manipulating the code tree semantically as
opposed to thinking about moving lines of text around.

------
throw03172019
Did it though? Most of the time when I mention Clojure, people think I mean
“closures”.

Our whole stack is Clojure/Clojurescript. Clojure has helped reduce bugs and
improve stability across the board.

I enjoy writing it everyday and wish more people would give it a shot.

~~~
yogthos
I don't think Clojure needs to boil the ocean to be successful. We already
have a community of many thousands of developers with lots of very smart
people in it. At this point there's enough mind share to keep it going
indefinitely.

At the same time Clojure benefits from being a hosted language. We don't have
to build everything from scratch, instead we leverage two of the biggest
ecosystems and reap all the benefits for free.

It's still a very niche language to be sure, but from what I've seen there has
been a lot of commercial adoption in the last couple of years. There are new a
few consulting companies, such as JUXT and Metosin, that are focusing
exclusively on Clojure consulting. A lot of the libraries in the ecosystem are
now maintained by these companies as opposed to individuals. There are also
efforts like Clojurists Together for funding Clojure projects sustainably.
These efforts would not have been possible without growing commercial use with
companies willing to donate money.

So more companies using Clojure directly leads to a better ecosystem, and that
in turn makes the language more attractive. The state of tooling and libraries
is dramatically better today than it was even a couple of years go. I'm
excited to see how the ecosystem grows this year.

One of the projects I'm most excited about is
[https://cljdoc.org/](https://cljdoc.org/) which aims to provide a one stop
shop for polished Clojure library documentation. I just saw a presentation
about it at Clojure North, and I was very impressed with where it's at
already.

~~~
throw03172019
Thanks for your reply. I am very glad to hear this! Cljdoc looks great as well
- much needed :)

Btw, thank you for all your open source projects and comments on various
GitHub repos.

~~~
yogthos
Thanks, glad to hear it's coming in handy. :)

------
acd
Clojure builds on Lisp that its creator wanted a modern functional programming
language that runs on the JVM. Clojure also has immutable data structures Link
to Githut programming language popularity on Github, compare the number of
open issues per repository for different programming languages. Then you will
find that Clojure, Lisp and Haskell has relatively fewer issues compared to
other more popular languages. [https://githut.info/](https://githut.info/)

Lisp is used in AI. [https://medium.com/ai-society/the-lisp-approach-to-ai-
part-1...](https://medium.com/ai-society/the-lisp-approach-to-ai-
part-1-a48c7385a913) [https://ai.stackexchange.com/questions/2236/why-is-lisp-
such...](https://ai.stackexchange.com/questions/2236/why-is-lisp-such-a-good-
language-for-ai)

------
furgooswft13
> But that's not how programming being taught and mentored today - it's more
> like "steal it from StackOverflow, adjust it to your needs. Do what everyone
> is doing. It's okay to follow the hype".

Oof. So true, I hate this field. But I guess you could say similar of almost
anything. Maybe I just hate humanity.

~~~
Jach
Soft-misanthropy is healthy enough, just as long as you don't go full-
misanthrope. I hope.

You left out the immediately following bit, "Experienced software developers
don't chase the hype". I cringed. But maybe if my interpretation of the
overall point is correct, I can agree with a point that Clojure attracts
grumpy programmers who have Seen Things and gives little lip service to
beginning devs.

~~~
lugg
Why cringe? Hype creates burnout. The churn is absolutely untenable in the UI
space right now and hasn't shown signs of slowing down anytime soon.

Where I work we have 3-4 different ways of accessing our monolithic backend,
and on top of that 3-4 different ways of accessing and manipulating
application state.

It drives me up the fucking wall.

I'm not sure I agree with the little lip service statement. I feel like the
feelings are most likely mutual.

Stable systems that don't need constant heroism and don't diverge much from
original choices made at the architecture level don't make for good places for
juniors to make mistakes and learn from them.

~~~
Jach
I cringed because it's not my experience that experienced devs are immune to
hype, it's an orthogonal characteristic. I also find it somewhat logically
inconsistent for there to exist a push for teaching/mentoring The Way of the
Hype if seniors haven't bought into it themselves.

------
refset
For me it was the draw of Datomic and DataScript. Clojure is a neat language,
but I feel like how we store, retrieve and transmit data is vastly more
important.

~~~
ilikehurdles
Take a peek at Crux from Juxt. Really exciting times happening on the data
front.

~~~
yogthos
link to the repo [https://github.com/juxt/crux](https://github.com/juxt/crux)

~~~
dfrage
One day old, 4 comment HN discussion of it:
[https://news.ycombinator.com/item?id=19714073](https://news.ycombinator.com/item?id=19714073)

------
yingw787
I'm doing a series on concurrency models at the moment for personal
professional development, and I recently reviewed Clojure and its native
ability to separate identity from state. It's a very properties-based and
specifications-based language, and the thing about properties and
specifications of languages are that they're nigh impossible to retrofit at a
later point in time. Other languages need to have external tooling or
frameworks on top for things Clojure supports out of the box, and even then
those frameworks may not be as reliable as something guaranteed on a language
level.

It's actually a pretty sizable moat. Python broke encodings between 2 and 3
and split the community for at least a decade. Some companies will likely
never be able to migrate from Python 2 to Python 3. Now imagine breaking now
`dict` works.

[https://bytes.yingw787.com/posts/2019/01/26/concurrency_with...](https://bytes.yingw787.com/posts/2019/01/26/concurrency_with_python_identity_and_state/)

------
karmakaze
I attribute it to two things. It's the least lisp-y of the lisps meaning it
came with so much that you didn't have/want to make a (different) dsl on every
project. The second is the jvm: partly from interoperability but even more so
from being just another jvm language with packaging/deployment and some ops
already paved.

------
dwhitney
Rich Hickey gives the best talks. It's that simple.

~~~
Scarbutt
Was that a pun? ;)

------
maleghast
Add onto the very real facility for being a highly reliable and productive
tool for doing data acquisition, transformation and distribution it makes
programming FUN!

------
cutler
Clojure suffers from a lack of ambition, at least as far as adoption is
concerned, and I think that's why many leave the community. Rich Hickey is
more than happy with Clojure's current reach which says everything when you
look at Clojure's abysmal level of adoption. It could be argued that Matz and
Rich are both similarly idealistic language designers but Matz recognises the
danger of Ruby being left behind without improvements to the language and
Ruby's community is already much bigger than Clojure's.

Although there's Luminus web frameworks are generally frowned upon in Clojure-
land which concedes mindshare to other languages which recognise the
importance of frameworks and killer apps. With so many languages competing for
mindshare it is that much easier today for a language, however original and
superior, to be sidelined by paying customers. For developers looking for a
language which will also help pay the bills Clojure is unfortunately not an
option.

~~~
didibus
Well, its got a chicken and egg problem here.

The chickens, experienced Clojure devs, don't need a framework, and are easily
able to do whatever they want and use Clojure to pay their bills. Thus they're
not motivated to build frameworks.

On the other hand, the eggs, devs unfamiliar to Clojure, find it difficult to
get anything done in Clojure, and are looking for frameworks that abstract
away some of the expertise needed to manage a full Clojure web app. They're
hoping they can start using it to pay their bills before they become an expert
in it, through the use of a framework.

I'm not sure how the future will unfold here.

~~~
yogthos
The way we build web apps is changing away from back-end frameworks in
general. Things like micro-services are a good example of that.

Traditionally, web was all about serving static documents, and server side
rendering model is a perfect fit for that. However, nowadays we also have web
apps that are highly interactive.

This requires moving away from the thin client model. Server-side rendering
approach is inherently at odds with that because HTTP is a stateless protocol.

So, the modern way to write web apps is to have a thick client that manages
things like views, business logic, and state in one place. With this approach
you no longer need a complex framework on the back-end because vast majority
of the complexity moves to the client. The server is mostly responsible for
things like data governance. The other driver here is horizontal scalability.
It's much easier to scale small stateless services than a giant monolith.

Clojure started getting a lot of use in the wild right around the time when
SPA model started taking off, and I think that the ecosystem reflects that. We
tend to have fairly light weight back-end architecture coupled with a thick
client on the front-end.

~~~
danmaz74
Moving business logic to a js client looks like a crazy gift to any ill
intentioned actor.

~~~
yogthos
Not sure how that's the case since authentication and data governance is done
by the server. This is just the difference between thin and thick client
models. Any app on a phone is a thick client as well, and uses exactly the
same model as SPA.

------
billfruit
I thought that Clojure was losing its buzz in recent times. Perhaps it is that
even though the core language is elegant its ecosystem of leiningen, maven for
dependencies, etc, and the whole brittle emacs-cider-nrepl chain is detracting
from its core strengths.

And poor startup times, and unhelpful error messages (though I hear there has
been some improvement in recent versions).

~~~
yogthos
You can use pretty much any popular editor, such as Atom, VS Code, IntelliJ,
and even VIM, for Clojure development nowadays. I personally use IntelliJ and
I've never touched Emacs for any serious projects.

I'm also not sure what the problem with Leiningen is, it's a hell of a lot
better than most build tools I've used. If you want to see brittle, you should
take a look at NPM clusterfuck. Yet, people happily use that mess all the
time.

Poor start up time is typically not an issue for web applications, which is
the primary domain Clojure is used in. Meanwhile, at dev time you're using the
REPL, so you're not restarting your app all the time. I often have an instance
of the app running for weeks on end as I work on it. For environments where
start up time matters we now have ClojureScript on Node and GraalVM.

The error messages are indeed a huge improvement in Clojure 1.10.

~~~
billfruit
Leiningen on Windows seemed like an after thought.

Even on Linux I have run into package management issues with lein1/lein2. I
should say a build tool should largely be stable, not changing it behavior. On
many functions like how you use local jars the features and methods have
changed drastically at various points of time in Clojure/lein. This kind of
churn has even effects like invalidating much documentation already written,
and many stack overflow answers too. Why should an application developer be
distracted by issues like these?

Most importantly if Clojure development is essentially depending upon lein,
then I feel it should be distributed with Clojure itself. And emacs, cider,
nrepl and lein chain has so many parts which are getting version upgrades
independently, it breaks frequently for the user, functionality that was
working suddenly stop working after an upgrade and you have to debug why it
stopped working. Also I wonder how complicated it might be to set up the whole
chain on a Windows machine.

~~~
Zekio
The Windows experience with leiningen can be pretty good when kept up to date,
which can be done easily using [https://scoop.sh/](https://scoop.sh/) to
install it and update it once in a while

------
amai
It didn‘t gain any popularity with me, since I had to debug it once. Most
Clojure code is just a collection of overly clever recursive subroutines. In
the end I rewrote the code in Java. It was shorter, easier to read and
probably more performant than the same code in Clojure. Don‘t believe the
hype. Clojure has some nice constructs for concurrent programming, but most of
them are available also in Java since version 8, or are available as some
external package. In the end programming in Clojure will just make your
project more expensive, as good Clojure programmer are a rare species. The
small advantages of smart Clojure constructs won‘t outweight the costs of
maintaining Clojure code in the long run. The same is actually true for Scala.

~~~
yogthos
I'm sorry, but as somebody who's been working with Clojure professionally for
nearly a decade I have to say that this is just utter nonsense.

------
elamje
Summary: The JVM made Clojure popular. The JVM made Clojure hated.

~~~
iLemming
Yeah, haters gonna hate - it's human's nature. Give a man a powerful tool that
solves lots of problems with two* most popular platforms today: Java and
Javascript and they still will be complaining.

* actually three - if you include CLR

------
elamje
I love clojure. Would it be possible to get it running on a rust runtime? That
would be awesome

------
k__
How much did it get?

------
lincpa
Clojure is a functional programming language based on relational database
theory.

The Clojure system can be treated as a RMDB using the Lisp language, RMDB
Schema (Clojure Spec) is a static type, and SQL (LISP) is a dynamic type.

It is an example of the best combination of dynamic and static types.

It's innovation, simple, practical and reliable.

[https://github.com/linpengcheng/PurefunctionPipelineDataflow...](https://github.com/linpengcheng/PurefunctionPipelineDataflow/blob/master/doc/Clojure_is_FP_based_on_RMDB.md)

~~~
sandGorgon
could you elaborate on that ? how is it based on relational DB ?

~~~
lincpa

                   Clojure -> DBMS, Super Foxpro
    
                       STM -> Transaction，MVCC
    

Persistent Collections -> db, table, col

    
    
                  hash-map -> indexed data
    
                     Watch -> trigger, log
    
                      Spec -> constraint
    
                  Core API -> SQL, Built-in function
    
                  function -> Stored Procedure
    
                 Meta Data -> System Table
    

In the latest spec2, spec is more like RMDB. see:
[https://github.com/clojure/spec-alpha2/wiki/Schema-and-
selec...](https://github.com/clojure/spec-alpha2/wiki/Schema-and-select)

The main development goal of clojure is to write the database. The development
idea is actually from the database, not the FP.

With reference to the database, as long as I use spec to strictly define
(standardization) the core data model, I can ensure the correctness of the
system.

I've turned the traditional work of ensuring code correctness from in-code
type system validation to data self-validation. Turn the work into verifying
the core data model instead of validating the input and output data types of
the function.

Similar to industry, verify that all finished products meet the standards
before entering the warehouse. Also similar to databases, verify their
compliance before data enters the database.

That is "Data as a service, Warehouse as the core, operates around it".

A system requires only a few core data models, and development is built around
the core data model.

Persistent data structures ensure that the modification of the immutable big
data model are high performance and memory efficient.

In addition, using my pure function pipeline data stream
([https://github.com/linpengcheng/PurefunctionPipelineDataflow](https://github.com/linpengcheng/PurefunctionPipelineDataflow))
makes debugging, parallelism and extension very simple.

from:
[https://github.com/linpengcheng/PurefunctionPipelineDataflow...](https://github.com/linpengcheng/PurefunctionPipelineDataflow/blob/master/doc/Clojure_is_FP_based_on_RMDB.md)

~~~
weavejester
_The main development goal of clojure is to write the database. The
development idea is actually from the database, not the FP._

This seems a tenuous theory. Clojure has been described as a language for data
processing, but this isn't the same as basing the design on relational
databases. Clojure isn't even relational; its data is generally kept in
standard collection types, rather than relations.

I'd describe Clojure as a dynamically typed, functional Lisp that emphisizes
open data models and explicit time.

~~~
collyw
its data is generally kept in standard collection types, rather than
relations.

Isn't that the same as relational databases once you dig a layer deeper?

~~~
cgrand-net
No, it's hierarchical (because we nest collection types) vs relational.

~~~
lincpa
In RMDB(For example postgresql), Database is hierarchical nest collection
types. It is also consistent with Clojure.

```clojure

{:table01 {:row01 {:col-array [0 1 2]

    
    
                       :col-json  "{\"a\": \"Hello\"}"
    
                       :col-text  "abc"}
    
               :row02 {}}
     
      :table02 {:row01 {}
    
                :row02 {}}}
    

```

In addition, postgresql supports inheritance, which is also hierarchical nest
collection types.

~~~
weavejester
While Postgres supports hierarchical data in the form of JSON, that doesn't
mean it's a form of relational data, it just means Postgres supports both.

Clojure doesn't have good tools in its core library for working with
relational data. There's no core type that explicitly represents a relation,
and Clojure lacks functions for many basic relational algebra operations. For
example, how would you perform a natural join across your data structure?

~~~
lincpa
In programming, data modeling is the most important.

Convert(or design) data to hash-map, join (or merge) by key.

it can write commonly used operations as functions, try to row (or col)
operations as much as possible, and join all data only when necessary(reduce
the row-join).

```clojure

(def a {:a-id-01 {:a-name "a1"}

    
    
            :a-id-02 {:a-name "a2"}})
    

(def b {:b-id-01 {:a-id :a-id-01 :b-name "b2"}

    
    
            :b-id-02 {:a-id :a-id-02 :b-name "b2"}})
    

(->> b

    
    
         :b-id-01
    
         :a-id
    
         a
    
         :a-name)
    

;=>

;"a1"

(let [x (b :b-id-01)]

    
    
       (->> x  
    
            :a-id
    
            a
    
            (merge x ,)))
    

;=>

;{:a-id :a-id-01,

; :b-name "b2",

; :a-name "a1"}

```

~~~
weavejester
You're describing a language whose data model is hierarchical, not relational.
In your code you first convert a relation into a hierarchy of collections. Why
would you need to do that if Clojure used a relational data model?

~~~
lincpa
In RMDB, database data model (db,table, row, col, value[array,json, int, text,
etc.] ) is hierarchical.

Relation is a logical model mapping of data structures, it is just a logical
thinking that exists in the brain.

SQL, Prolog, clojure.core, minikanren can be used for relational operations.

~~~
weavejester
The relational model isn't arbitrarily hierarchical. We can place a map inside
a map, or a map inside a vector, and keep going indefinitely until we run out
of memory. Conversely, in a relational database we can't place a database into
a table, or a table into a row.

It's true that we can represent relations using collections. In Clojure we'd
write:

    
    
        (def a
          #{{:a/id 1, :a/name "a1"}
            {:a/id 2, :a/name "a2"}}
    
        (def b
          #{{:b/id 1, :b/name "b1", :a/id 2}}
    

But these structures don't allow for efficient lookup or joins, and we lack
inbuilt functions to easily deal with data modelled in this way.

Relational databases are based on relational algebra. If Clojure is based on
relational databases, then we'd expect to be able to do relational algebra
easily in Clojure. But we can't: the core library isn't designed for it, and
the built-in data structures aren't designed for it.

~~~
lincpa
In general, I think:

1\. arbitrary layering and deep nesting are not good engineering practices.

2\. refer to the data-model & code of my latest two posts. I prefer to use
hash-map as the table with the primary key hash index, with key as the primary
key and val(colname-colval-hashmap) as the row content.

I also don't think relational algebra operations must be implemented in the
form of RMDB and SQL. It can also be implemented very elegantly with
clojure.core. using hash-map operation is simpler, clearer, smoother and high
performance.

There are many ways to implement relational algebra. The thinking is not
limited by the "information structure" displayed by the traditional RMDB
interface. In clojure, the hash-map(NoSQL) is the underlying physical model,
and the relational model is the upper logical model.clojure core function acts
as a data manipulation language, I named this architecture SuperSQL or
SuperRMDB.

In fact, the original data manipulation language of posgresql and foxpro is
not SQL. Clojure core function is closer to foxpro's commands (DML).

3\. set, vector, list is generally not a good default data container, only
used when needed. you use the set as container, it's difficult to operate data
(table, row, column, value).

4\. In summary, I think: programming is the process of designing a data model
that is simple and fluent in manipulation. To have open thinking, not to be
restricted by traditional thinking, to be flexible, adapt to local conditions,
and design as needed.

~~~
weavejester
_" arbitrary layering and deep nesting are not good engineering practices"_

Perhaps, but that's irrelevant; I'm describing the difference in how
hierarchical and relational models are designed.

 _" I prefer to use hash-map as the table with the primary key hash index,
with key as the primary key and val(colname-colval-hashmap) as the row
content."_

And what if you need a second index? Your indexing should be separate from
your data model, otherwise you can't write performant relational algebra
operations that apply in the general case.

 _" It can also be implemented very elegantly with clojure.core. using hash-
map operation is simpler, clearer, smoother and high performance."_

No it can't. Suppose I have a relation with keys: _a_ , _b_ , _c_ , _d_ and
_e_. I want to index on _a_ , _b_ and the pair _(c, d)_. How would I do that
in Clojure? What happens if I later decide I also want to index on _e_?

This is the sort of problem that's trivial to solve in a relational database,
and extremely hard in Clojure, because Clojure doesn't have the functions or
data structures to support data modelled in this way.

That's not to say that Clojure _can 't_ have these tools; just that they
aren't built into clojure.core, because that's not what it's designed for.

 _" set, vector, list is generally not a good default data container, only
used when needed"_

Yes they are. Sets are the basis of relational algebra.

You're complecting the ideas of data representation with data indexing. Sets
are a good _representation_ of a relation, but a poor _index_.

We can get the best of both worlds by combining the two:

    
    
        (def a
          (let [r1 {:a/id 1, :a/name "a1", :b/id 1}
                r2 {:a/id 2, :a/name "a2", :b/id 1}]
            {:relation #{r1 r2}
             :index
             {:a/id   {1 #{r1}, 2 #{r2}}
              :a/name {"a1" #{r1}, "a2" #{r2}}
              :b/id   {1 #{r1 r2}}}}))
    

A data structure like this allows us to start writing efficient relational
algebra. For example, with a natural join we can look for the smallest index
two relations have in common.

So we _can_ begin to construct the infrastructure we need to perform
relational algebra in Clojure, but it's not there to begin with, and therefore
Clojure isn't designed around the relational model.

~~~
lincpa
If you want to build an RMDB with Clojure, you are right, but this kind of
project is very rare, including Datomic, which is built on top of RMDB, I
don't think it needs to deal with such low-level and general operations.

Implementing an RMDB is not equivalent to relational algebraic operations. I
think it should be to use simple, direct and lightweight the relational logic
model to solve real-world problems. Don't over-optimize, over-generalize and
over-complicate, keep it simple and direct.

Just like we design a Database in RMDB to solve real-world projects, This is
the normal way to use relational algebra and models. After the database design
is complete, we don't need care how the index is implemented, and we don't
need care the underlying storage of the data. I mean, clojure is used as RMDB
, is not used as tool of construct RMDB .

Therefore, my method is to design the application-level data model. The
problem you said does not exist. When I get the data from the database (or
elsewhere), I simply transform the data to the target model, you can think of
this model as a table or view, you don't need to transform again, so you don't
need multiple indexes.

~~~
weavejester
I'm not talking about building a database; I'm talking about using a
relational model for data.

Like most general-purpose programming languages, Clojure has a hierarchical
data model. We have a number of collection types, and we can put any
collection into any other collection.

A relational model takes a fundamentally different approach. Relational data
is represented not by nesting collections, but by a flat set of tuples.
Efficiency is achieved through indexing, not by rearranging collections.

There's some interesting research around on using the relational model outside
of a database, but that's not a design goal of Clojure.

~~~
lincpa
My method can also generate a "derived index" based on the "primary key hash
index",it's Higher performance, not by rearranging collections.

```clojure

(def table01 {:t1-pk1 {:pk :t1-pk1

    
    
                           :name    "t1-r1"
    
                           :manager :m1}
    
                  :t1-pk2 {:pk      :t1-pk2
    
                           :name    "t1-r2"
    
                           :manager :m2}
    
                  :t1-pk3 {:pk      :t1-pk3
    
                           :name    "t1-r3"
    
                           :manager :m3}
    
                  :t1-pk4 {:pk      :t1-pk4
    
                           :name    "t1-r4"
    
                           :manager :m2}})
    

(def t1-manager-index {:m1 #{:t1-pk1}

    
    
                           :m2 #{:t1-pk2 
    
                                 :t1-pk4}
    
                           :m3 #{:t1-pk3}})      
    

(->> :m2

    
    
         t1-manager-index
    
         (select-keys table01 ,)) 
     

; =>

; {:t1-pk2 {:pk :t1-pk2, :name "t1-r2", :manager :m2},

; :t1-pk4 {:pk :t1-pk4, :name "t1-r4", :manager :m2}}

(->> [:m2 :m3]

    
    
         (select-keys t1-manager-index ,)
    
         vals
    
         (apply clojure.set/union ,)
    
         (select-keys table01 ,)) 
     

; =>

; {:t1-pk2 {:pk :t1-pk2, :name "t1-r2", :manager :m2},

; :t1-pk4 {:pk :t1-pk4, :name "t1-r4", :manager :m2},

; :t1-pk3 {:pk :t1-pk3, :name "t1-r3", :manager :m3}}

```

Your point of view is mainly to emphasize that Clojure is a multi-paradigm,
general-purpose functional programming language.

The postgresql development team is also this view, so postgresql is not only
RMDB (relational modeling), but also supports OO and json (NoSQL). But
postgresql default data modeling is relational modeling

My point of view is mainly to emphasize the best practices of data modeling
and programming.

Both views are correct and can exist in parallel.

~~~
weavejester
_" Your point of view is mainly to emphasize that Clojure is a multi-paradigm,
general-purpose functional programming language."_

No, that's not my point at all. I'm saying that Clojure's core library and
data structures are built around a hierarchical data model and not a
relational one.

If you want to model your data as a relation, then you need to build the tools
and structures for it. Look at your code, then consider how it would look in a
language designed around relational algebra:

    
    
        (def table01
          #rel [{name "t1-r1", manager :m1}
                {name "t1-r2", manager :m2}
                {name "t1-r3", manager :m3}
                {name "t1-r4", manager :m2}])
    
        (select table01 (= manager :m2))
    
        ; => #rel [{name "t1-r2", manager :m2}
        ;          {name "t1-r4", manager :m2}] 
    
        (select table01 (or (= manager :m2) (= manager :m3)))
    
        ; => #rel [{name "t1-r2", manager :m2}
        ;          {name "t1-r3", manager :m3}
        ;          {name "t1-r4", manager :m2] 
    

An indexed selection against a relation would just be a single function or
macro. We wouldn't need to mess around with select-keys and set union to
achieve such a simple operation, as you needed to do in your code. It would be
built into the core library or the language.

I want to emphasize that I'm not saying you _shouldn 't_ model data the way
you are. There are plenty of advantages to it. But the more you go down the
relational rabbit hole, the less suitable clojure.core is to handle it.

Clojure _is_ about data modelling and processing, but it isn't based on a
relational model, as you suggest:

 _" Clojure is a functional programming language based on relational database
theory"_

Clojure is a functional programming language, but it's not based on relational
database theory.

~~~
lincpa
If Clojure is the sea, programming is to sail on the sea, I use the relational
model as a lighthouse and route, as a reference model for simple programming,
because the relationship theory is simple and scientific..

I had implemented a DataFrame with hash-map, which implements relational
operations. A relational operation is just a function. The advantage of hash-
map is that key-chain can be used as a pointer, and processing data elements
is simple and efficient. Therefore, DataFrame has advantages of RMDB and
NoSQL.

A strict relational model will lose the flexibility of data element
operations.

------
_pmf_
For me, Clojure was full of false promises and dead ends / false starts
regarding concurrency[0]. Elixir delivers where Clojure failed.

[0] for example, nobody uses the STM implementation

~~~
weavejester
Where did Clojure fail? People don't use Clojure's STM because it turned out
not to be necessary most of the time, not because of any inherent issue with
the implementation.

------
armitron
The thing that kills Clojure for a lot of teams I worked with is the JVM. It's
too limiting and constrains the problems one can solve.

~~~
iLemming
Whenever I ask to explain JVM criticism, people either really have nothing to
say, or most of the time it turns out to be hatred of Java. JVM is a very
solid piece of tech and it is hated for all the wrong reasons.

I'm baffled: give people a powerful tool that can simplify and bring enormous
joy to work with two, most popular platforms of all time, even making it
possible to share code between completely different worlds - they'd still be
complaining about it.

------
shay_ker
Clojure is popular? It's at the bottom:

[https://insights.stackoverflow.com/survey/2019#technology](https://insights.stackoverflow.com/survey/2019#technology)

As much as I love Clojure, it's still a harrowing language for beginners to
learn and use. The learning curve is just too high. If you have to learn
category theory in other to handle basic programming flows, then there's just
something wrong.

Elixir, for instance, has many functional primitives, but isn't dogmatic about
it. It can "feel" like traditional OO-style programming without the cruft.

~~~
dragandj
How the heck Clojure has any relation to category theory? Clojure explicitly
does NOT require nor use any explicit CT. Maybe you confused Clojure with
Haskell?

~~~
shay_ker
Perhaps my understanding is outdated! IIRC you needed CT because you couldn't
do some basic conditional flow things, but I could be wrong. It's been a
while.

~~~
projectileboy
With respect, I think you may be confusing Clojure with a different language.

