
A Personal Lisp Crisis (2012) - llambda
http://blog.jacius.info/2012/05/29/a-personal-lisp-crisis/?
======
breckinloggins
I love Clojure and I think it's the first Lisp to take the language family out
of "cons cell and parenthesis hell". My personal opinion is that Clojure with
the following changes would take over much of the Lisp space:

\- Rewrite most of Clojure in Clojure (which has been talked about since the
beginning, really) to find the "small core". Thankfully ClojureScript has
helped hasten the CiC movement.

\- Base FFI and code gen on more abstractions so that Clojure living in a .net
world or a C/C++ world or a Python world would feel just as cozy as Clojure
living on the JVM. Rich Hickey talked about why he abandoned primary CLR
support early on. He said he "didn't want to keep writing everything twice".
This was probably the best strategy at first, but now one hopes that with
ClojureScript + Clojure enough has been learned to discover more ways to
factor out the commonalities so that Clojure is more easily retargetable.

\- Speaking of targets: there should be a first class, fully supported Clojure
frontend to LLVM. There are research projects and personal projects in this
area, but that's about it. I'd like to see an official LLVM target.

\- Introduce or surface more abstractions and macros such that the
immutability and pure functional underpinnings could be extended, removed, or
replaced. This is likely to be the most controversial change, but it would be
essential to the possibility of Clojure running on small embedded systems and
other environments where the choice of underlying data structures and
concurrency algorithms can't come baked in. This is mostly done already (as
most things in Clojure are built on top of abstractions), but it still needs a
deep look.

\- Open up the reader. A Lisp without full reader macros is unlikely to win
over all potential converts, not because reader macros are this incredibly
common thing, but because the personality of most Lisp hackers tends to
thinking like "well I don't think I'll need a reader macro, but why would I
want a language that won't even give me the option?"

~~~
reitzensteinm
One interesting thing about having entirely immutable data structures is that
you can allocate a temporary heap, and flip the GC around. Copy off just the
result and junk the rest without looking at it. All without blocking the main
thread.

In certain circumstances - for instance a web framework creating a heap for
each request - this will make GC dramatically cheaper. Though of course your
cache locality will be poor.

~~~
cynicalkane
This is a common practice in GC. For example, the JVM already does this by
default, and you don't need to give it immutable data.

~~~
reitzensteinm
But immutability makes it conceptually a hell of lot cleaner, and feasible to
do even if you're investing orders of magnitude less in your GC than went into
the JVM.

Which is important if Clojure gets ported to bare metal runtimes, as the
parent was suggesting.

------
agentultra
First there are a lot of programmers who come to CL to debate. They read, they
study, and then they complain. I've sat in #lisp since about 2007/8 and there
have been numerous nicks that show up just to ask a few rather philosophical
questions and debate the merits and technical decisions that went into CL.
They usually end up leaving and writing blog posts about how terrible a
language CL is and how caustic the community is.

There isn't a problem with the CL community. There's a problem with the
culture surrounding CL that seems to attract trolls.

Newbies get answers to genuine questions. I've received a great deal of help
over the years from experience CL programmers. Anything from beginner
questions to code reviews of various snippets I've been working on. I have
received a lot of great advice over the years that has helped me tremendously.
And I've seen other newcomers receive the same treatment as well. If you
arrive in #lisp of c.l.l and have a genuine question about a problem you are
stuck on you will likely attract someone willing to help you. It's a small
community but it is friendly.

Second, CL was in the end designed to outlive itself. You get full
programmatic access to the compiler and reader. It is trivial to add any
feature a modern programmer may require that wasn't accounted for in the
original specification. CLUEL could certainly have been useful if the author
had finished it and put it out into the wild (maybe waiting until it was
_finished_ was the wrong choice).

I think if you are genuinely interested in learning CL you need to check your
ego at the door, so to speak, and just start from scratch. Write programs that
solve yours or your users' problems and ask questions in #lisp and on c.l.l if
you get stuck. There is a wealth of great literature, papers, tools, and a
small, friendly community to help you.

 _Edit_ : I meant to note that the reason the specification hasn't been
updated, afaik, is because it doesn't have to be updated. All the tools you
need to add the features you require are there in the spec. You can make it
into the language you need.

~~~
jlarocco
> Edit: I meant to note that the reason the specification hasn't been updated,
> afaik, is because it doesn't have to be updated. All the tools you need to
> add the features you require are there in the spec. You can make it into the
> language you need.

That's a big part of the reason I haven't used CL for anything bigger than
some toy projects.

It's a crusty, old language, and instead of keeping up with the times,
everybody has to build the features they need. I can do that, or I can use
language that has the features I need and that I don't have to assemble
myself.

And to make it worse, it's hard to even use libraries that solve the problem,
because nobody in the CL community can agree on anything. There's always a
dozen incompatible solutions, and it's never clear which one should be used.

~~~
agentultra
I don't really understand the, "crusty, old language," argument either. C++
became an ISO-certified specification in 1998 which is roughly around the same
time that CL became an ANSI standardized language. We still use C in many
systems for very valid reasons.

However unlike the C++ and ANSI C standards, the CL specification left room
for programmers to extend the language as they need it. Instead of waiting for
decades of expensive committee debates and meetings as you do with
C/C++/ECMAScript/etc the programmers using the language in the field can go
ahead and add the features they need and implementors can fill the gaps
between new OSs and hardware. How is that bad?

~~~
jlarocco
It's bad because every implementation fills the gaps in their own incompatible
way. And then instead of converging on the "best" solution, the community
argues endlessly about each unique solution is the best.

And it's also bad because CL is ancient and it's filled with things that
simply don't make sense any more, or have been replaced by "better" solutions
in every other language. Look at filename and filepath handling to see what I
mean. 2 or 3 of the fields haven't been used in real systems in 20 or 30
years, yet they're kept around because they're in the unchanging CL standard.

------
betterunix
I am not sure it is true that comp.lang.lisp _barely tolerates_ newbies. I
have posted there on several occasions and when I say something wrong, I am
politely corrected. I have seen a lot of less experienced programmers come
there, ask an obviously beginner question, and get helpful answers. Reading
through the past few days of posts, I can see a few beginner questions which
all received helpful answers.

The problem in comp.lang.lisp is the number of trolls -- in other words, the
universal problem on Usenet. People come in trying to prove that some other
language is superior, or asking questions like "If Lisp is so great why is
nobody using it?"

~~~
LanceH
It has been absolutely years, but I recall they didn't like homework problems.
This made it really difficult to ask newb questions.

~~~
serf
homework problems are different than newbie problems..

a homework problem is a problem you're charged with solving so as to reinforce
elements and critical thinking from the chapter or segment of teaching that
you experienced before the problem. Very few homework problems are given that
cannot be solved by the user, if they paid attention to the work. There is a
very thin line between someone trying to crowd-source their homework to a
programming forum, and actually asking for help. This can usually be
determined by how the question is asked.

Example: "How do I write this..?" is not as respectable a question as "I am
writing this, i'm this far and I ran into this problem, can anyone help me to
understand my problem?"

a newbie problem is a problem generated through normal use of a language to
achieve a goal, in which the language user ( the newbie ) lacks the personal
experience to conquer some goal post standing between him and his reason for
writing what he/she's writing.

Example : "I'm coming from such and such a language where we can do this...
... will this ... work the same in CL?" or "This piece of code behaves in this
way ... can anyone help me work through as to why it does this?"

The problem (that people have) with the lisp community is that they tend to be
highly sensitive in regards to hand-holding, and can usually sniff out
homework problems with relative ease.

------
protomyth
Perhaps the author will find the answer to the "why" of how unwelcoming a
community can be in his choice of opening paragraphs. A story that calls some
number of his audience as stupid[1] or ignorant[2] for holding a belief is
probably not the best way to introduce your soul searching on why the
community he was a part of is not a welcoming one.

1) the implication if you have read your holy book and still believe then you
are a poor thinker.

2) if we were in the Middle Ages, the story might be more relevant. I do
believe most believers of any religion in the modern era have probably read
their own holy book.

~~~
breckinloggins
Coming from the Bible Belt, I can assure you that point 2 is absolutely false.

That said, even as a non-Christian I was also uncomfortable with that as an
introduction. I'm sure many Lispers are of the opinion that almost 100% of
their peers are atheists or other "nones", but that's (I suspect) more a
function of geography than language choice.

It's unwise to turn off what could be a significant portion of your readers in
the first paragraph!

~~~
hga
Indeed, you can catch more flies with honey than vinegar. This author has told
us up front attracting people to LISP/Clojure is _not_ his primary goal.

There's also the _minor_ detail that if you're a Christian the really canon
stuff is in the New Testament, a lot if not most of the problematic stuff is
in the Old Testament and not longer strictly applies (surely most of us
remember "Let he who is without sin cast the first stone"???), so I suspect
the whole premise is false.

~~~
protomyth
Well, certainly the procedure for going to the bathroom is a bit different in
modern times[1], and what parts of the Old Testament is a matter of debate
between branches of Christians. Heck, the english translation of "thou shalt
not (murder|kill)" is a bit of a thing. By the time you get out of bible
school and listen to the sermons, you have gone through the whole bible[2].

I just found it very tone deaf to open with that kind of story when it
basically has the same effect as what he is complaining about. Its just odd.

1) Although if you happen to hit a disaster, reviewing a bit of the health
guidelines in Leviticus couldn't hurt. It will also probably help if you
remember a lot of it is basic how-not-to-die health guidelines.

2) I came from the Catholic and 7th Day Adv family (yeah, not exactly the most
agreeable to mix), and both seem to have gone through the whole thing. Plus,
they have a bit of disagreement as to what constitutes the whole thing. Luther
did a bit of pruning.

~~~
AutoCorrect
the Catholic church did a bit of pruning too, in the 3rd century AD. Council
of Nicea

~~~
protomyth
The Council of Nicea didn't deal with bible canon - that was made up for "The
Da Vinci Code". Various religious sites debunked that one during the books and
movie introductions.

------
tom_b
As someone who learned a little Lisp in the last few years, I think there is a
good collection of books available for the newbie who wants to learn. For
Common Lisp, _Land of Lisp_ , _Practical Common Lisp_ , and _Ansi Common Lisp_
all seem good enough to get hacking quickly.

Thinking about community effects, I'm not as convinced a community represents
a gating factor for hackers. I attend the occasional Clojure meetup (and
Clojure/Conj for that matter). The meetups in my area are well-attended and
run, although they could be improved with a better introduction and ice-
breaker for more introverted attendees (yes, I've just outed my personality
type).

But from an opportunity mindset, the Clojure community seems to have done an
admirable job getting the message out. Multiple good books, the Clojure/conj,
and a well-attended meetup in my area all make me feel like I may not only
stretch myself and learn new programming techniques, but that I have some hope
of advocating for Clojure in my workplace or, good grief, applying for an
actual day job where I would use Clojure.

It's unclear to me that Common Lisp can offer that.

~~~
Adrock
Hi tom_b, I'm one of the co-organizers of the NYC Clojure meetup. I don't know
if that's the one you attend, but I'm definitely interested in any specific
ideas you might have for better introductions and ice breakers.

~~~
tom_b
Thanks for organizing a Clojure meetup, it is much appreciated.

My only recommendations are quite generic - for smaller meetups, make sure a
group introduction is on the agenda and actually happens. Might be nice if
there were easy ways to become more active in the group, but this is hard for
the meetup/talk format.

I'm definitely not being critical though - I feel lucky enough to have a
Clojure meetup happen in my general area. Mostly it falls back on me, as an
interested-in-the-community hacker to find a way actually be social and
integrate myself into that group.

------
stephenpiment
Honestly, I don't know that CL was ever exactly beloved among Lispers. (Scheme
may fit that description better). CL was a pragmatic construct from the
beginning, responding to many conflicting demands.

------
pnathan
This is a strange article.

Let's address his complaints in two basic areas:

1\. Community.

The Lisp community as typified by #lisp and comp.lang.lisp can be remarkably
acerbic to the point of toxic, depending on who feels like contributing at the
time. Part of this are the annoying trolls that show up; one's patience wears
thin. A FAQ document might go a ways. However, there are a large number of
people who don't ream newbies out and really are helpful. lispm here on HN is
one of the friendliest Lispers out there and actually has helped me out tons.

Addendum: r/lisp on Reddit is good for generalish Lisp questions.
StackOverflow's common lisp tag almost always gets an answer to most any
reasonable question. sbcl-devel mailing list is useful if you want to take the
time to posit in-depth technical questions to the SBCL developers. Lispforum,
oddly enough, I haven't used much. I simply don't go there much. I should, I
suppose. :)

2\. Technical. I can do no better than quote my /r/lisp post on this (r/lisp
is _really_ good for lisp help).

\---

First of all, consistency. A variety of things are peculiar in how they are
named. TERPRI? Really?

Second, Common Lisp is a (the?) programmable programming language. CLUEL-esque
platforms could be used to drive CDR adoption and cleanups of the original
language.

My primary gripes with CL are the lack of CDR-2, lack of CDR-8, the lack of
hierarchical namespaces, the funky naming conventions & expectations[1], and
the lack of deeper CLOS integration[2].

[1] ELT vs AREF vs NTH, for instance. Or, LOOP for lists vs arrays vs (your
data structure here).

[2] See CDR-8.

------
outworlder
Props to the author for mentioning Chicken Scheme. The community is very
friendly indeed. There are lots of eggs and the easy C integration means that
one is not usually left wanting for libraries, assuming a minimal knowledge of
C.

Also, it runs basically anywhere. One could embed it inside an iPhone app, for
instance.

There are some big things missing still. There is no standard database access
library, or an equivalent to Python's WSGI (or Ruby's Rack). Or MongoDB
drivers (that's my fault, I promised the community one a couple of months
ago).

There's also some missing tooling. Better debugging support, for instance.
That's mostly avoided by the use of unit tests and the REPL, but is
occasionally required. Or profilers. Or code coverage tools. Due to the fact
that everything is compiled to C, there are some tools that may be used, but
it is still not ideal.

------
fusiongyro
There isn't, as far as I'm aware, another language with CL's popularity that
provides either its condition system or the full power of the CLOS (method
combination, for instance). These two major subsystems are still significant
improvements over what modern languages have to offer.

~~~
hga
Good point about CLOS, but for a lot of us who've gained an allergy to OO over
the years, especially in its difficult relation to functional programming,
that's of limited appeal unless you're working in one of the areas its
fruitfully employed.

Would have to look at it's condition system, it's been way too long.

~~~
Theriac25
Have you really looked into CLOS? Because IMHO it doesn't have much common
with any of the OO systems in any of the mainstream languages.

~~~
hga
Yes, I have, and starting from the Lisp Machine Flavors precursor. While I
haven't used it in anger, it's in theory by far my most favorite OO system.

This is in part philosophical, the "prefer 10 data types and 90 functions that
operate on them, often all of them" sort of thing, in part my horror at
smearing behavior _and_ data all over the place, and pragmatically, how that's
much more of an imperative than functional approach. And functional is where
its at for what I want to do. And experience in using OOSE to develop a
greenfields system. That was utterly cool and amazing, but ... see my last
sentence below.

Which is not to say that the next time I do a GUI I'm not going to use OO to
model the widgets et. al. on the screen. I believe it has its place, but that
it's used _much_ more widely than makes sense.

------
wes-exp
Should be tagged [2012]. Rubyist briefly falls in love with CL, then falls out
of love. News at 11.

------
jff
This reminds me of my experiences with Plan 9. A very cool system, but it's
managed to attract some of the absolute worst people in computing. Dare to
criticize something and they'll leap down your throat. In the end, I just gave
up. We tried to drag things kicking and screaming into the 21st century with
source control, 64 bit kernel, automated builds and testing, etc. but nobody
wanted it.

On the other hand, the mailing list did get a lot of posts that kind of remind
me of this: "Hi, I just started using Plan 9, where's emacs? Anyway I have a
lot of great ideas on how to fix this OS so you should all start doing these
things... I'll set up the wiki and write the newbie guide"

~~~
bitwize
Maybe this:

 _On the other hand, the mailing list did get a lot of posts that kind of
remind me of this: "Hi, I just started using Plan 9, where's emacs? Anyway I
have a lot of great ideas on how to fix this OS so you should all start doing
these things... I'll set up the wiki and write the newbie guide"_

and this:

 _Dare to criticize something and they 'll leap down your throat._

are related?

------
baddox
> the surest way for a Christian to lose their faith is to read the Bible,
> cover to cover.

I doubt that, based on my observation of a strong correlation between intense
academic study of the Bible and fortitude of Christian faith.

------
adestefan
Is this not just another point in a decades long quest to save LISP?

Maybe LISP doesn't need saved? Maybe it's okay that a language isn't popular?
Maybe It's okay that a language isn't hip? Maybe it's okay if a language
doesn't have the latest gee wiz social status? The fact that people keep
trying to save LISP every 4 or 5 years could mean it's perfectly fine how it
is. I don't see people trying to save the COBOL, or FORTRAN, or <insert
favorite esoteric language here> community.

~~~
hga
Maybe ... prior to the R6RS debacle I had my hopes on Scheme, but it's still
digging itself out of that mess, plus the long term failure to advance from
R5RS that prompted the process change.

Common Lisp has had 3 decades to get its act together and is an abject failure
in my view, plus it's got a half century of accumulated backwards
compatibility accreted messiness.

Clojure is not a LISP, as in LISt Processing is not its alpha and omega. For
now I won't go into details on I think it's being so successful, expect to
point out that its architect has exquisite taste and savvy pragmatism, has
made some great innovations in e.g. making maps and vectors syntactically
first class citizens, and of course if you like serious functional programming
and/or multi-core SMP/ccNUMA programming is has a great story. And most of all
it's developed an ecosystem that has left Common Lisp and Scheme in the dust.

BTW, I don't think anyone needs to "save" COBOL, it's still used a lot, and as
I understand it FORTRAN is still going strong in its niche, with regular new
standard. Either are very far from being esoteric languages, although COBOL
action is mostly hidden in the IT departments of organizations that write and
maintain their custom systems.

~~~
mrottenkolber
Am I the only one who thinks Clojure is extremely ugly, especially compared to
CL?

I work a lot with clojure, and it reminds me a lot of when I was forced a hack
ruby. There is this guy, or even a group of people, who get caught up in the
glamourous idea of reinventing programming. They then jerk their favourite
features and design artifacts into a language. The main point of the design
seems to be being able to express the thing in fancy words.

Some people love that stuff, they think its beauty and good design because
it's somewhat consistent (usually not after the second glance).

I think good design is whats works, and what has evolved due to the needs of
practicioners. And that is the beauty of CL, it's shaped by evolution and
consensus, not by some vison.

Edit: Also, ever taken into account, that CL is maybe simply good enough?
Every possble feature can be included by libraries. There is no need for an
updated CL, what we need is more standards like bordeaux-threads, usocket,
closer-mop...

~~~
hga
Sort of. On the surface I find Clojure infinitely more ugly than true LISPs
... but if hairing up the syntax with vectors results in more adaptation its a
price I'm willing to pay. On the other hand I'm _really_ pleased at its making
vectors, maps and sets first class syntactic citizens, and the other things
that have unified them with lists.

Underneath the hood I think there's no comparison, then again it addresses
exactly the sort of things I'm interested in. Common Lisp just can't win there
due to it being the current version of mainline LISP going back to, well, I
supposed the punched card days when it was implemented as FORTRAN subroutines.
Yeah, car and cdr are neat, and even useful in combination, but ... well, no
where can you assume TCO.

(Have to run, more later.)

~~~
mrottenkolber
> well, no where can you assume TCO.

(recur ...) Is an ugly hack. Yes in CL you can assume TCO from good
implementations.

You know you can add vector/map whatever syntax to CL in like 50 loc? Are you
seriously saying that syntactic sugar is in any way relevant to anything? Let
alone an important feature of Clojure? Thats just sad.

> car and cdr are neat, and even useful in combination,

Actually wether you call them car cdr first rest or whatever, it's actually
about CONS or call it a pair if you like. A very simple but infinitely
powerful datatype. This is the first thing I teach when I teach programming
for a very good reason.

~~~
ScottBurson
"A CONS is an object that cares." \-- Bernie Greenberg

~~~
hga
Which is followed by: "It cares about its CAR and its CDR." Then as I recall
he went to the blackboard.

Still trying to find a copy of his MIT IAP Lisp course notes (haven't tried
very hard yet); they and his delivery of them were very very good.

At the same time Jeff Schiller (JIS) would give a short neat talk on how
macros worked and why they were neat.

------
cageface
I came to precisely the same realization about ten years ago, after struggling
in vain with the culture and quirks of Common Lisp. There's no question that
learning Lisp in some form will make you a better programmer but don't get
caught up in the cult that surrounds it. Virtually all of Lisp's innovations
now been validated and adopted by more mainstream languages and the fact that
nobody has picked up the s-expr syntax is telling.

These days I focus more on writing useful code and making end users happy and
I've found that to be much more satisfying than wringing my hands over the
inelegance of everyday languages.

------
avodonosov
heresy

