

Ask HN: What's the deal with Common Lisp? - jff

So I'd really like to try using Common Lisp more often, but I'm having a hard time getting into it. To start with, I thought I'd install SLIME, but of course it turns out the SLIME that ships with Debian is broken, so I have to check out the repository version. No big deal. Then, I thought I'd try fiddling with an IRC bot, everyone's favorite programming project. CL-IRC couldn't install (using ASDF-install) because one of its dependencies wouldn't build. Ok, I said, I don't mind working at a socket level. Oh, wait, here's half a dozen different socket libraries, some unmaintained, some still works in progress. I finally settled on usocket. Then, when Freenode's MOTD included a pound sign, which caused my reads to crash, I had to fiddle about some more until I finally found flexi-streams, which seems to be working ok for now.<p>Right now, Lisp is feeling like a massive twisty world of broken dependencies and half-finished libraries. Have I just hit a bad corner of the Lisp world? C may not be fancy but at least the standard libraries can talk to the network. I'd appreciate hearing from other people who came in, were similarly disgusted, but have come to understand the whole thing better.
======
hga
(Note, I myself started with Lisp prior to Common Lisp by a few years, but I
used it near its inception and have vaguely followed it since then (I soon
switched to Scheme and now am playing with Clojure).)

" _Have I just hit a bad corner of the Lisp world?_ "

Yes, you have; for the things you want to do you should try either a Scheme
that has the libraries you need (I'm not sure if there are any (probably not,
but maybe try one that e.g. already has IRC stuff?)) or Clojure, which by
virtue of sitting on top of the JVM and having fast and clean interoperability
handles the libraries problems.

Dependencies are handled by existing parts of the Java ecosystem and/or the
relatively new Leiningen system (the name is a play on ant, the first Java
build system).

Warning, in terms of functional programming, there's a continuum of CL <
Scheme < Clojure. Unlike the others, Clojure is not multi-paradigm, you _will_
use it functionally (well, unless you use it as syntactic gloss for
programming in Java, which misses much of the point).

Here's the most recent HN item on getting up to speed with Clojure:
<http://news.ycombinator.com/item?id=1086562>

Note that getting SLIME up at this very moment might require a bit of
trickery, a recent SLIME commit broke it for Clojure's use of it. Be sure to
check out the Clojure Google group and feel free to ask any sorts of questions
on it if you get stuck (the community is a lot better than CL's).

(Historical notes: CL is bad for networking and concurrency because it was
formalized before there were standards for those areas and it has been stuck
in amber since then. Various implimentations have non-standardized libraries
for them (there may be something of a standard for threads, I forget, but in
many CL implementations threading support itself is iffy at best). And you
already know the networking story (which CL are you trying?).

Scheme standardization is _very_ slow; it is progressing, there are optional
SRFI (Scheme Requests for Implementation) standards for each of the above (I
think, not sure about networking), but I can't recommend it for what you're
trying. If you e.g. wanted to start from the bottom up by going through the
SICP or another book, that would be fine, or if you wanted to research
language stuff it's great, but it just doesn't yet have a general modern
programming ecosystem and won't for some years if ever. In general, you match
your needs with the implementation and its packaged libraries and go from
there. If you want to try that route anyway, ask me for some pointers.)

------
sedachv
Bad corner of the Lisp world? Socket libraries are the Skid Row of Common
Lisp.

The reason it's hard to create a good library comes from an interplay of 3
things: streams, lack of standard character representation, and lack of
standard foreign function interfaces. The latter two are sort of advantages in
that if they were standardized in 1989 or whatever, we would probably have
wound up with something stupid involving DOS code pages and Fortran on VMS.

Streams are convenient and most CL functions expect them. They're also a
really inefficient way of using BSD sockets. Especially when your Lisp
implementation's character encoding isn't UTF-8. So if you want to write fast
network code in Lisp you basically need to use the raw POSIX FFI functions and
byte arrays.

Then you get into fun stuff like vectored I/O (which you may or may not be
able to do depending on whether your GC moves arrays around), and event-based
I/O (which people disagree as to how to do properly). But it's ok, because
none of the existing socket libraries try to support this anyway, so you're
back to calling POSIX functions via FFI.

(I gave a presentation about this stuff last year:
[http://carcaddar.blogspot.com/2009/11/november-montreal-
sche...](http://carcaddar.blogspot.com/2009/11/november-montreal-schemelisp-
user-group.html))

If your socket library is dependent on any third-party libraries, pray that
the latest versions will not completely change their interface and make your
library fail to compile. Then multiply all this by the half-dozen or so Common
Lisp implementations people expect to be supported.

Just like Skid Row, this has been a recognized problem for many years (well
over a decade for certain), and it's kind of hard to make it better. Flexi-
streams over usocket is the best off-the-shelf solution, so you're probably
doing the right thing.

------
gruseom
_I'd appreciate hearing from other people who came in, were similarly
disgusted, but have come to understand the whole thing better._

Ok. The larger CL world is terribly fragmented. There's a smaller world inside
the larger world which is really good. The trick for the beginner is
distinguishing between the two, i.e. between what is detritus to be ignored
("massive twisty world of broken dependencies and half-finished libraries"
describes that perfectly) and what is the real stuff (e.g. Edi Weitz's
libraries like flexi-streams). Once you find your way to the smaller world,
the fragmentation and dependency issues don't go away, but they become minor
and occasional relative to the mind-blowing amazingness of what CL is good
for.

How do you find your way to the real stuff? Trial-and-error and asking
questions. Is that an acceptable answer? No; many people unfortunately never
get over the confused/disgusted stage and bail. Does it work? Yes, very well,
_if and when_ you get over those initial hurdles.

I fully agree that the above situation is upsetting when you compare it to
things that just work out of the box. There's so much half-baked and half-
working stuff in the CL world that I can only compare it to spam. A lot of
Lispers are unhappy about this, and many attempts have been made to fix it,
but it isn't quite clear what a definitive fix would require.

My personal hope is that the work being done by the Clozure-with-a-Z people
(CCL) will blossom into a well-functioning, self-sufficient, obvious place for
people to begin with Common Lisp. They do seem to be getting there.
Importantly, they have the nucleus of something that CL has been lacking for a
long time (since, I suppose, the implosion of comp.lang.lisp into various
forms of nastiness): a nice, normal community.

------
CyberFonic
I'm no expert in any of the Lisps, but find that the learning curve with CL is
massive. Of course, the return on your investment in learning is huge as well.

Depending on what your applications are, you might find something that really
hangs together well, like newLISP a better fit. And it's very simple to
intergrate libraries written in C.

~~~
hga
I'm a bit concerned by any Lisp that still uses dynamic scoping (Emacs Lisp is
the only other current Lisp dialect that does). I gather that newLisp tries to
mitigate the issues, but I'm not sure how well.

------
anonjon
"Have I just hit a bad corner of the Lisp world?"

Sort of.

The real problem is that it is difficult to know what the current 'best'
implementations of different libraries are, and what versions of common lisp
they are intended to build under (and operating system!)

This is compounded by the fact that the higher level libraries have more
dependencies.

As there are dozens of libraries and a smallish CL community, this leads to
the problem where author of sub-library a makes change which breaks dependent
library b, and no one realizes this for months and months. When someone such
as yourself comes along (even if it is possible for the library to work quite
well with minor changes), the build process fails is not so obvious about what
is wrong. Your reaction is to say 'CL is totally fucked' and run screaming to
the hills is only natural.

That said, I have a few tips:

1.) If you are going to use an open source lisp, linux is generally the easier
operating system to run it on.

2.) If you are going to download a library, don't use asdf-install.

ASDF itself is great for actually structuring the project and laying out your
directories in a coherent manner (managing dependencies), but asdf-install is
a waste of time (in my opinion, anyway, I'm sure someone will disagree with
me).

Instead, go to the library's page/repo and download the latest version.

Make sure that someone has been working on the library recently (I try for < 1
year since last update). (Another good way to check if a library works is to
search Github... EX: [http://github.com/search?type=Code&language=common-
lisp&...](http://github.com/search?type=Code&language=common-
lisp&q=IRC&repo=&langOverride=&x=0&y=0&start_value=1) ).

Once you get a version that works, hang onto it somewhere in case the guy
working on it breaks it with an update. If you can find a version that almost
works, it is (a lot of the time) worth it to salvage it and push the updates.

Just read the error messages very carefully and it should be fixable.

asdf-install is great in theory, but there are too many synchronization
issues, and not enough humans pushing fixes/updates to manage them.

3.) Keep your lisp image up to date. The people developing libraries are
generally using the latest build of clozure/sbcl/etc. You should too.

Scheme implementations don't have these problems because there is no
expectation of write once run anywhere with regard to libraries. Different
implementations have their own package managers and BDFLs.

Clojure doesn't have this problem because it hasn't been around long enough to
have a lot of dead libraries.

I'd like to point out that it is perfectly reasonable to program functionally
in common lisp. I take a fairly functional style in most of my code.

edit:punctuation, grammar...

~~~
hga
" _I'd like to point out that it is perfectly reasonable to program
functionally in common lisp. I take a fairly functional style in most of my
code._ "

Absolutely; my point on FP WRT these three Lisps was that you don't have any
(realistic) choice not to go all-in functional with Clojure, whereas Common
Lisp is the most multi-paradigm and you can get away with an imperative style
should you so choose.

------
wendroid
I don't know mcuh about LISP. But what I know about OSes means I know the
advantage of Everything is a File.

On Plan9 if you can open / read / write / close / remove you can do almost
anything : networking, graphics, serial, debugging, mouse control, the list
extends by design.

We have a few Schemes ported for those who had a personal itch. ALISP claimes
a port but their site is down for me.

~~~
jff
I have a few Schemes installed on my Plan 9 system, but I haven't really done
anything with them. I'd really like to see an example of integrating a Scheme
with devdraw; I've seen everything else in your list done by file operations
but only used regular library calls for graphics.

~~~
wendroid
Forth is in the same boat, no-one is going to make C based Plan9 libs for it.
The one I use 4th only has file ops anyway so I plan9 to do something with it
but it is the perennial todo list problem :)

