
The Tragedy of the Common Lisp, Or, Why Large Languages Explode - error54
https://esdiscuss.org/topic/the-tragedy-of-the-common-lisp-or-why-large-languages-explode-was-revive-let-blocks
======
lisper
One cool thing about Lisp is that you can easily embed new languages in it,
and those languages can be small and beautiful. For example, I have a Python-
esque FOR macro that uses an iterator protocol, and a universal binding macro
that subsumes all of Common Lisp's binding constructs (LET, LET*, LABELS,
FLET, MULTIPLE-VALUE-BIND, etc.) So for me, Common Lisp has actually shrunk
without losing any functionality. This is not possible in languages without
macros. Such languages are indeed doomed to either grow forever, or change in
non-backwards-compatible ways (e.g. Python3). But with macros you can shrink a
language as well as grow it. This is one of the reasons Common Lisp continues
to thrive.

[UPDATE]: You can find my code here:
[https://github.com/rongarret/ergolib](https://github.com/rongarret/ergolib)

Also, I forgot to mention another language-shrinker included in that library:
REF. REF is a universal de-referencer that subsumes NTH, ELT, SLOT-VALUE,
GETHASH and probably a few other things that I can't remember right now. It
also lets you build abstract associative maps (a.k.a. dictionaries) with
interchangeable implementations (see the DICTIONARY module), which lets you
get rid of ASSOC and GETF.

~~~
collyw
I don't know Lisp, so correct me if I am wrong. The whole idea of embedding
your own language sound pretty much the same as a growing the language, except
that you are doing it yourself, in a non standard way.

~~~
lisper
If you build one new feature that subsumes N>1 old features then you're
effectively shrinking the language because you never need to use those N old
features any more. You can then support legacy code with macros that compile
the old features into the new feature. At that point you've actually shrunk
the language. You can even have those legacy support macros give compile-time
warnings that the old, now-deprecated feature is being used and that the code
ought to be changed. In fact, you can even easily write tools that will do
this translation automatically. So yes, you really can shrink the language.

~~~
avn2109
That sounds cool, but I have a deep-seated conviction that programming
language design and implementation should be done by someone smarter than me.

~~~
bad_user
Yet you're designing your language whenever you write a function.

~~~
Marazan
I am ashamed to say that I only had this incredibly important insight very
recently, after 20 years or so of programming .

I think it is one of those leveling up moments - a thing that once you know
allows you to write better code at all times.

------
hydandata
There is a great book about Lisp from Christian Queinnec titled "Lisp in Small
Pieces" or LISP. Here is a little excerpt from it.

"There are subjects treated here that can be appreciated only if you make an
effort proportional to their innate difficulty. To harken back to something
like the language of courtly love in medieval France, there are certain
objects of our affection that reveal their beauty and charm only when we make
a chivalrous but determined assault on their defenses; they remain impregnable
if we don't lay siege to the fortress of their inherent complexity. In that
respect, the study of programming languages is a discipline that demands the
mastery of tools, such as the lambda calculus and denotational semantics.
While the design of this book will gradually take you from one topic to
another in an orderly and logical way, it can't eliminate all effort on your
part."

Lisp has some of the best literature around of any programming language.
Anybody who really cares about craft of programming should make use of wisdom
therein.

~~~
mkramlich
seeing this comment on HN, made my PG, a Lisp book author, is amusing. I have
to admit that while I don't do Lisp day-to-day probably my favorite Lisp
book(s) were written by him. The practical hacker in me prefers Python, Java
and C.

But the elegant hacker in me? Prefers Lisp. And PG captured that in his
writing.

~~~
PuercoPop
IDK where the elegant but impractical (as it was a trade off). CL is a
practical language. Consider loop, format, multiple values or the standard
methd combination in CL.

While the only practical thing about python is that it has more libraries.
Nevermind the fact that python scope is misdesigned even in Python 3!

------
davelnewton
It was painful watching Common Lisp happen; too many competing interests, and
commercial stakes. Unfortunately languages went a different direction. (I also
mourn Smalltalk's "loss", but Java had much more money coming into it.)

Lisp50 went into this somewhat ([http://www.nhplace.com/kent/Papers/cl-untold-
story.html](http://www.nhplace.com/kent/Papers/cl-untold-story.html)) and,
unrelated, was a freakin' awesome good time. I sat next to Guy Steele for one
talk but was too in awe to even say anything.

(Anecdote about same: I IMed a friend and said "I'm sitting next to Guy
Steele" and he replied "Cool, ask him who Guy Steele is." Damn kids.

~~~
lukego
I really did not enjoy Lisp50. I was surprised to discover such a disconnect
between the old-school and new-school Lispers. Guy Steele and co really didn't
seem to have any interest at all in what people have done with Common Lisp
these past 20 years or so. That is a pity because I had always really valued
the perceived continuity of the Lisp community.

~~~
davelnewton
I think Clojure's reception was great--the oldies, overall, were very
positive.

~~~
lukego
Agreed. That was classy. But that is also a sign of them having no interest in
the modern Common Lisp community :).

~~~
davelnewton
I read it differently; I think they thought it was an interesting direction,
but only that, another direction. The oldbies are pretty CL-oriented, although
obviously many of them have moved on (e.g., Fortress for GLS).

------
kazinator
Though the 1994 ANSI standard is 1153 pages, Common Lisp somehow doesn't feel
large. A lot of it is library pieces that can be understood more or less on
their own and work independently. Somehow you can know the language well,
without reading 1153 pages cover to cover. If you see anything in someone's
code which is standard, but which you don't know well (or at all), it's not
going to throw you a big curve ball.

C is a "small" language and is pushing 700 pages now.

Projects written using small languages tend to use lots of extensions. So do
projects in larger languages, too; they use some subset of the core language,
probably a small one, and then other libs which address problems not covered
in the language at all.

How big is Perl? How much of CPAN should be included in that measurement? If
the answer is "none", how realistic is that? Do you know Perl if you don't
know any CPAN module?

How about Scheme? The base standard is small. But then there are SRFI's. It
seems disingenuous not to count htem. And then there are implementations and
their environments and extensions, which projects depend on. What better
represents "Scheme size"? The R6RS document, or some measure of the size of,
say, Racket?

~~~
realityking
I think the post was more about syntax, not standard library. Functions and
methods are reasonable easy to lookup, syntax much less so.

Also try googling for some unknown syntax.

~~~
kazinator
> _I think the post was more about syntax, not standard library_

If that is so, it has no point.

The bulk of the 1153 pages of the Common Lisp standard is in fact describing a
standard library, so if the definition of "large language" is one that has a
large core syntax, excluding standard library, then it's a small language, in
fact.

Most of the syntax of a typical Lisp dialect (Common Lisp included) takes the
form of a standard library. If you seen an unfamiliar syntax, it consists of a
form with an unfamiliar symbol in the leftmost position:

    
    
       (unfamiliar-symbol ... stuff (you (do not)) understand)
    

You search your help resources for "unfamiliar-symbol".

The lexical syntax ("read syntax" in Lisp terms) is quite very small. It
consists of elements like what constitutes a symbol token, what numeric and
other constants look like, and other such elements. Stuff like:

    
    
      #(this is a vector)
      #c(3.0 4.0) ;; complex number 3.0i + 4.0.
      `(quasi ,quote)
      '(quoted list)
      package::symbol

------
PaulHoule
In the case of Java it is not that Java is a "large language", it is that to
get anything useful done with Java you need to know about Maven and Spring and
Log4J and Apache Commons Logging and SLF4J (because if you're using a lot of
libraries surely all of those will be in use.)

That is, it is the complexity of the ecosystem, not of the language.

~~~
falcolas
When I was learning Java recently in anticipation of a job programming Java -
I was surprised by this reality. The core of Java is remarkably simple to
learn - there's not really all that much to it.

The complexity is indeed in all of the libraries and build frameworks and well
intentioned but silly HammerFactoryFactoryFactoryFactories.

~~~
jerf
I think the case can be made that Java was _too_ simple. It's inability to
express very much within itself is what led to explosion of external tools to
make it "better", or indeed, "work".

I think it was deliberately designed as a simple language to be used by large
groups of people in simple ways, but actually failed so epically at that goal
because of being _too_ simple that it actually destroyed the entire idea of
building a language deliberately for large corporate use. (Note that it has
grown a lot since then; it had to.) Go's the first language I've seen since
Java try for that niche. I've said it before: In the short term Go may be
stealing from Python and Node, but in the long term, Java's the one that needs
to be worried about Go.

Edit: Literally six minutes later, my feeds produce for me:
[http://www.businessinsider.com/google-go-update-from-
jason-b...](http://www.businessinsider.com/google-go-update-from-jason-
burberel-2015-6)

~~~
aikah
> In the short term Go may be stealing from Python and Node, but in the long
> term

Go is a niche language.As a niche language it will perform well in its niche,
but it will never be as big as Java or C#. Go total lack of expressiveness
makes it unfit for a wide range of applications.

Java is rigid, but I think version 8 makes it more enjoyable. But it will not
make all the terrible java core apis and framework go away.They are still
here.

~~~
bad_user
Java was also a niche language. People never learn unfortunately.

------
orthecreedence
I think lisp could benefit from a small core and building out a standard
library. You could pack all the features it _needs_ (packaging,
lexical/dynamic scoping (defvar), let/lambda, defun/defmacro, multiple values
(via values, multiple-value-call), setf (w/ setf expansion), simple
arithmetic, declare/declaim/proclaim, maybe a few more) into the core and have
standard libraries: cl.bind (multiple-value-..., defparameter, etc), cl.math
(sin, cos, etc), cl.clos, cl.collections (arrays, hash tables), cl.io, etc
etc.

I think this would clean things up a lot, still preserve the spec (aside from
documenting what's in which libs), and make things more approachable.

Shoving everything into the "common-lisp" package _works_ but it's cumbersome
and you have to have the _entire language_ sitting there to use anything.

~~~
tjr
I don't have the exact quote/source right here handy, but I believe that was
Guy Steele's intention with Scheme.

~~~
tikhonj
He gave a brilliant talk about it: "Growing a Language"[1].

The idea is to make languages that _grow_ —ones that provide a small, uniform
core that can be extended by the user. Ideally, these extensions feel like
first-class citizens: things added by users should feel on par with built-in
language features.

It's still one of the best technical talks I've ever come across.

[1]:
[https://www.youtube.com/watch?v=_ahvzDzKdB0](https://www.youtube.com/watch?v=_ahvzDzKdB0)

~~~
agumonkey
He even humorously implied that a growing language shrinks, by replacing many
specific constructs with a single general one.

------
chubot
I'm glad someone said this. I'm an occasional JavaScript programmer, and I
looked over ES6 last night and was surprised by how large it's become. And I
learned that ES7 is already on the way.

That said, most of the features seem nice, and many are borrowed from stable
languages like Python, so perhaps it's not too much. I'll have to try it and
see.

It made me wonder what Crockford is up to, and what he thinks of this.

[https://github.com/lukehoban/es6features](https://github.com/lukehoban/es6features)

[http://es6-features.org/](http://es6-features.org/)

~~~
mkozlows
Yeah, I share the linked author's opinion of ES6 -- it's good stuff, but it's
also a dangerous direction.

Part of me thinks that maybe what's needed is an updated version of "use
strict" \-- "use es6" or whatever -- that would let you use the new features,
but also prevent you from using some deprecated features, to keep the surface
of the language somewhat smaller even as new stuff gets added.

~~~
dangoor
That was seriously considered some years back and thrown out as likely to
cause poor adoption and poor intermingling of language features.

[http://www.2ality.com/2014/12/one-
javascript.html](http://www.2ality.com/2014/12/one-javascript.html)

------
gongador
The plan for Common Lisp originally was to have a "core" and a standard
library. From Daniel Weinreb's blog post "Complaints I’m Seeing About Common
Lisp":

    
    
        It’s just too big. Actually, the real problem is that the core of the language is not cleanly separated from the built-in libraries.  The Common Lisp designers had originally intended to do this separation, but there wasn’t time enough.
    

[https://web.archive.org/web/20100706204555/http://danweinreb...](https://web.archive.org/web/20100706204555/http://danweinreb.org/blog/complaints-
im-seeing-about-common-lisp)

(Daniel Weinreb was, among other things, one the designers of Common Lisp.)

Zach Beane has some more information on this at
[https://xach.livejournal.com/319717.html](https://xach.livejournal.com/319717.html)

EDIT: "time enough", in the quote, may seem strange; after all, work began in
1984 and the standard was finalized in 1994. But remember that many
stakeholders were companies with jobs to do, and they had to assign employees
to the design/standardization work at real costs for said companies.

------
braythwayt
I like the author’s remarks and philosophy about keeping JavaScript small, but
I thought the opening was remarkably uncharitable. The specific person and the
specific feature are quite irrelevant to the point he’s making here.

I am left with some admiration for his goals, but also a great deal of
trepidation about ever suggesting anything or even talking about
JavScript.next. Will I be the next one called out by name if I make the
mistake of asking whether traits might be a good addition to JavaScript?

~~~
inglor
For what it's worth - the authors know each other from before and knowing both
parties Mark did not intend to mean any offense.

~~~
braythwayt
I came back to note that Mark has subsequently clarified that he meant
absolutely no slight against Kyle. He is a gentleman.

------
Grue3
Interestingly hardly anybody uses Algol, Smalltalk, Pascal and early Scheme
anymore, while people still use Common Lisp. Perhaps "being small and
beautiful" is actually a bad thing for a programming language?

------
Animats
I didn't realize that Scheme had become bloated. I haven't looked at it in
years, and thought it was still the basic language described in SICP.

~~~
hga
See now the R7RS which is split into a small and large standard (the latter
still in progress). R6RS was largely ignored by the community, and there
wasn't huge growth prior to it.

------
newuser88273
Common Lisp actually has a core of a mere thirteen "special operators". You
can think of everything else as standard library.

------
TazeTSchnitzel
ECMAScript 6 is a shame, there's a lot of stuff added which is unnecessary.

"let" is unnecessary. JS now has two kinds of variable scoping! "var"'s
hoisting is annoying, sure, but we don't need two kinds of variable scope. If
you want to scope something to a block, you can just use an IIFE.

"class" is unnecessary at best. JavaScript has a bunch of ways of constructing
objects to choose from, and that's not a problem. Why lock users into one
paradigm, and obscure what's actually happening underneath? This will just
confuse people when they have to deal with code that doesn't use "class"
syntax or the OOP model it presents.

Object property shorthand is confusing. Why the hell is {bar} equivalent to
{bar: bar}? Isn't that a set literal (Python, math)? Why isn't there the
colon, if it's an object? What the hell? Try explaining that to newcomers.

Computed property names looks weird and is misleading. You'd logically expect
{[1+1]:2} to be an object with an Array (coërced to string?) key, because []
is an Array literal. But instead it means "compute this expression". In which
case, why isn't it ()? That's what you'd intuitively expect. I've tried to use
() before and was surprised it didn't work, even.

Method properties, e.g. { foo(a,b) { ... } }, are unnecessary given =>
functions.

All that being said, I think ES6 has some quite positive additions. Maps,
sets, tail-call elimination, =>, modules and symbols are all very important
and useful features JavaScript really needed.

~~~
Arnavion
>If you want to scope something to a block, you can just use an IIFE.

Now try doing that in a loop that you want to break out of. Edit: To save you
the trouble -
[https://github.com/babel/babel/issues/644](https://github.com/babel/babel/issues/644)

>"class" is unnecessary at best. ... Why lock users into one paradigm?

It canonicalizes one of the popular ways of doing classes (the other being the
same but without `new`).

>Why the hell is {bar} equivalent to {bar: bar}? Isn't that a set literal
(Python, math)?

{ ... } in JS has never meant set literals. It does however mean objects
(dictionary literals) which is also how Python uses it.

>You'd logically expect {[1+1]:2} to be an object with an Array (coërced to
string?) key, because [] is an Array literal.

[] has also always been used to index objects and arrays, so using it when
generating the object with keys and values follows as an extension of that.

>Method properties, e.g. { foo(a,b) { ... } }, are unnecessary given =>
functions.

Arrow functions capture lexical this, which method properties do not. Compare
`({ x: 5, foo() { return this.x; } }).foo()` with `({ x: 5, foo: () => this.x
}).foo()` Arrow functions also do not have an arguments object.

~~~
TazeTSchnitzel
> Now try doing that in a loop that you want to break out of.

Fair point, although this can be worked around. Though it begs the question of
why you need block scoping anyway. If you have a function large enough to need
it, you should probably break it down into smaller functions, and compose
them.

> It canonicalizes one of the popular ways of doing classes

But there are other popular ways, and this way new users will have the
underlying details hidden from them, meaning they'll encounter problems later.
It's also potentially misleading.

> { ... } in JS has never meant set literals.

Yes, but it's never been { a, b } - there's always been a colon. Python also
uses {} for dictionaries, but _with colons_. Having { a } magically use the
variable's name as a key name, and also use the variable's value, is
unintuitive. { a, b } in another language would be an array (C, C++) or a set
literal (Python, mathematics). Nobody would expect it to do what it does here
in ES6.

> [] has also always been used to index objects and arrays, so using it when
> generating the object with keys and values follows as an extension of that.

I suppose that makes some sense, but we don't use [] for string keys in
literals.

> Arrow functions capture lexical this, which method properties do not.

Oh, right, good point.

------
pjmlp
The common fallacy about simple languages.

Yes the language might be simple to understand, but then the result is the
complexity lands in the shoulders of developers and an ever increasing library
of workarounds to compensate for missing features.

Hence why every simple language that achieves mainstream use, ends up becoming
like the ones it intended to replace.

------
jaunkst
My prespective could be very wrong but isn't this a sort of quality is in the
eye of the beholder issue. I get building a silo isn't very beneficial to
another. But isn't building monolithic library just as destructive. I don't
think anything is perfect even if it's perfectly executed.

------
ericbb
Here's a title for a rebuttal in case anybody wants to write it. ;)

The Tragedy of ISLISP, Or, Why Small Languages Implode

------
upofadown
Whenever I encounter people arguing about Python3 I am reminded that I still
miss Python1.

------
raymondh
Python also suffering in this regard. It has moved from being "a language that
fits in your head" to a language where very few people on the planet know most
of what's in it.

------
a8da6b0c91d
You hear this "safe subset" thing about the syntax rich languages all the time
and I really don't get it. I really do use all of C++. There are some pseudo
deprecated bits and some extremely esoteric bits I don't hit, but beyond that
I do wind up jogging most pieces of the language and standard library. Pretty
much the same with my usage of perl. I've never written a source filter or a
format, but otherwise I use a lot of the "niche" features regularly to great
effect.

This whole sentiment comes from people who work on large rotating teams with
enough inexperienced people, I guess? Sorry, learning a language properly
takes a couple years or more. The features aren't wrong or bad, your team just
doesn't know the tools well enough. You can't play modal jazz with the big
boys until you can do your scales. I guess have fun doing your pop medleys
with "simple" languages.

------
dschiptsov
The Arc language (which runs this site) is a remarkable attempt to "fix" what
went wrong with CL. It lacks a decent runtime and native code compiler (it
offloads everything to mzscheme, the way clojure did to JRE) but it is already
more than a proof of concept.

The problem is that there is no more DoD or other grants for creating new
Lisps anymore (particularly due to Java mass hysteria and prevalence of
packer's mentality).

BTW, making something similar to SBCL (everything written in itself, except
for a tiny kernel written in C) for Arc (a core language, without the kitchen
sink syndrome) is of moderate difficulty compared to meaningless piling up of
more and more of Java crap.

