
Why I chose Common Lisp over Python, Ruby, and Clojure - smanek
http://postabon.posterous.com/why-i-chose-common-lisp-over-python-ruby-and
======
WilliamLP
Basically: I started with my conclusion, and then went through all the other
options and rationalized them away until it seemed like my choice was based on
objective reasoning.

~~~
smanek
I think you drawing that conclusion may be a result of how I structured my
essay (I introduced my conclusion, then my reasoning).

I freely admit my personal biases and experiences eliminated a lot of
potentially good options without ever giving them due consideration (e.g.,
Haskell, Scheme, Lua). But I really did try to give the 3 options listed in
the article a fair shake at the time.

For what it's worth - I was about 50-50 on using Python till the very end ...

~~~
wheels
I too thought the logic was weak. Here was my basic objection: you chose not
to use two popular web programming languages because of their potential
volatility, but then went with one that has been so infrequently used in web
programming that it almost certainly has opaque deffeciencies. It's not that
your reasons were bogus, they just didn't carry the decision.

~~~
smanek
That's a fair point.

For a programmer without Lisp experience, that would be true. But I've written
enough Lisp-based webapps that there aren't really that many 'unforseen'
problems for me anymore (rule of thumb: use software by Edi Weitz - everything
he writes is golden :-D)).

That being said, I still ran into trouble with Elephant, so I probably should
have taken these sort of 'unknown unknowns' (which are more prevalent in
unusual languages like Lisp) into consideration upfront. To not do so was
certainly an oversight.

~~~
ube
If you are going to be the only programmer on the project then your reasons
make sense. However, if you are going to have other people on the projects -
then your reasons do not make sense from a longer term point of view. How many
lisp programmers do you have out there as compared to python or ruby
programmers? If you and the founders decide to go your separate ways how
likely is the project going to be continued in lisp?

It seems like a somewhat precarious decision to let you use Lisp for this
project. There's nothing supremely special about the problem you are attacking
that warrants Lisp (outside of your comfort level which you clarified very
clearly in your post).

Perhaps I'm just old and cynical but I've been on too many projects where the
first (and therefore "lead") developer chose a particular language and
architecture that in the long term was not sustainable.

~~~
smanek
There are, obviously, far more ruby/python programmers. But, at the very most,
I might only ever need a dozen Lisp programmers and I already know 20 that I
would love to work with. Lisp is even pretty easy to pick up if you're smart
know other dynamically typed, functional, and OO languages.

As a point of reference, ITA Software has no trouble hiring 100s of Lisp
programmers.

Minor correction: I am a founder - and if I were to leave (while my co-
founders continued), I'd make a point to find my replacement first.

------
tjr
_After that, I was planning to dig into some specifics about the best way to
manage/deploy a production Common Lisp webapp, which I hope would help a new
Lisper get off the ground._

Looking forward to that.

------
ricree
_The fact that he proposed removing map, reduce, filter, and lambda from
Python 3._

In the case of map and filter, is there any compelling reason to use either of
those instead of list comprehensions or generator expressions?

~~~
riobard
A single, non-nested list comprehension or generator exp is basically
map(filter). You need nesting to get filter(map).

e.g.

    
    
        map(expensive_call, filter(cond, seq))
    

equals

    
    
        [expensive_call(each) for each in seq if cond(each)]
    

but

    
    
        filter(cond, map(expensive_call, seq))
    

equals

    
    
        [each for each in [expensive_call(x) for x in seq] if cond(each)]
    

note because of "expensive_call", it's inefficient (and silly) to do

    
    
        [expensive_call(each) for each in seq if cond(expensive_call(each))]
    
    

So map/filter combination gives more flexibility than list comprehension, and
for functional-thinking minds, it's just so natural to think in abstract terms
of passing functions around. List comprehension is pretty syntactical sugar to
do similar things, but it forces you think about the "how to do" instead of
"what to do".

That said, it's not really "compelling" though -- there is no real
"compelling" reason to switch from one Turing-complete language to another
given that you can do the same thing eventually. But hey, it's the itches that
drive us nuts, isn't it? :)

Just my 2 cents

~~~
amix
Most times you are interested in doing the simple, e.g.:

    
    
      filtered = [x for x in seq if x>10]
    

Python's list comprehension is much more readable than using map/filter/reduce
- at least for Python programmers :) Anyhow, I really like Guido's decision on
dropping these - it creates a cleaner language and forces people to think
Pythonic when programming in Python.

~~~
riobard
> Python's list comprehension is much more readable than using
> map/filter/reduce - at least for Python programmers

For simple cases, yes. But I wouldn't say

    
    
        [each for each in [expensive_call(x) for x in seq] if cond(each)]
    

is more readable than

    
    
        filter(cond, map(expensive_call, seq))
    

at least for functional-thinking minds. The level of thinking in abstract is
different here.

Now the problem is, some people see Python as a very functional language (with
first-class functions etc) and want to use it that way (like Lisp), but BDFL
and some core Python devs believe it is better to keep it Pythonic, thus those
functional people are kinda pissed off by this and switch away from Python.

Personally I don't think it will make Python a lot cleaner to remove two
auxiliary functions and force people to use list comprehension when it is
completely trivial to add these missing pair back (two lines of code).

(disclosure: I prefer FP, but I also think keeping things Pythonic is fine
most of the time. It's just that in this case, I think map/filter is pretty
"Pythonic" according to me. :)

~~~
ramen
I know you all know this, but I feel like it bears mentioning that nobody is
forcing you to use list comprehensions whether map/filter stay in Python's
built-ins or not. They can be defined in around 3-4 lines of code each. Lisp
aficionados, already accustomed to the bottom-up style of programming, ought
to have no problem writing functions like these as necessary.

~~~
holygoat
There's a difference between things being _possible_ and being _encouraged_.

De-emphasizing functional operations makes it more likely for libraries to
work in a non-functional style, for tutorials to do so, etc.

It's tiring fighting against a community and a (benevolent) dictator that
disagree with you.

~~~
riobard
> It's tiring fighting against a community and a (benevolent) dictator that
> disagree with you.

That summarizes the problem I guess :D

------
vseloved
It's funny, that the decision to choose Common Lisp for some project often has
to defend itself (because of so much FUD around the language). CL has one
killer feature, that will appeal to any mature developer: it's the only
production-ready language around, in which you are virtually not constrained
by any third-party design decision. That's it. Discussing other details just
boils down to the question of tastes (that differ).

~~~
pwnstigator
What about Clojure, circa now? Do you not consider it "production ready"?

I've used both CL and Clojure. I'd prefer either over Blub, but Clojure wins
in my opinion.

~~~
vseloved
Clojure is nice, although I didn't use it in production. But it is Java deep
underneath (so not completely homoiconic) and comparing to CL there are
restrictions, albeit mild.

For some specific tasks I might have preferred it (until CL catches up with
concurrency support). But in general CL gives you more freedom.

------
mahmud
Did you really _choose_ Lisp over alternatives?

Before learning CL I was a fairly decent, C, C++ and Perl programmer. Did
assembly, Pascal, TCL and Awk. Up to that point, I always had to pause a for a
minute when starting a new project/script, think about its scope, and _choose_
a language based on the necessary performance, development speed,
expressiveness, available libraries, etc. (and whether whoever was going to
read the code afterward knew the language; C was often a natural choice for
code shared with others on Unix, C++ for MFC/COM, Perl for sysadmin stuff, and
TCL and Awk for my own tools.)

I learned Lisp in over a month, to spite someone (I dared a notorious troll I
would write an AI bot of his choice if he stopped spamming us, youthful
bravado for sure, and I lost the bet) While researching "AI" I came across
Winston and Horn's "Common Lisp", then the hyperspec, then a few more books
over the course of a month. I sat down with SICP and did the exercises on my
break, while I was in school and waiting tables.

After I learned it however, specially with CLOS, there was no contest. Three
months after buying Sonya Keene's CLOS book it was fair to say I _forgot_ all
other programming languages. There were no more "projects"; I no longer had to
sketch out designs on paper or do "requirement analysis" (something I was told
in school was necessary for all software.) For once, the great ideas in my
head were a new emacs buffer away. I could write code faster than I would in
Perl, Awk or TCL, it ran as fast as C++, and it was more expressive than the
English in my head. I could type "commands" into a shell get a dialog embedded
in my window, a few more commands and it would move to the upper right corner,
I could change its name property and add text to it, then I could fold that
dialog box into a menu-item named "Help" in the menubar and call that
dialogbox "About". Amazing.

I went on hacking like this for about year when I realized I was doing the
"wrong thing". You see, I have been using CMUCL with its builtin editor and
writing GUI applications in Motif (it was 2001 and Motif wasn't open source
yet, so I got the hang on Lesstif and learned its quirks.) Right around this
time, Linux GUIs were maturing and people were being snobs about their
Enlightenment themes and dissing each other over their choices of Windows
Manager. So I was peer-pressured into learning DHTML and Web Design. I read
comp.lang.lisp and those too were snobbish condescending idiots who flamed
everyone, specially competent programmers whose work I admired (including
Scott McKay and Robert Fahlman (the very people who gave me my CMUCL.))

It was really hard to be a Lisper for a while, specially a young
impressionable one who read cll uncritically; news of corporate giants coming
with new tools and programming languages to enslave humanity were abound.
First C++, then Java, then XML, and finally .NET. You literally had to pick
your battles and choose a corporate sponsor or you would have no future in
computing! (you think I am kidding?) cll is all doom and gloom, and of course,
there is the obligatory stabs at Lisp vendors by Open Source proponents, and
stabs at Open Source for people alleging it's killing our beloved vendors.
Every once in a while there was news of a Lisp dialect that's going to kill
Common Lisp (Smalltalk, Dylan, and the ancient religions of Mesopotamia.)

Fuck, that was painful.

All the while I was following this 4-year long intellectual funeral, becoming
ever more "hardcore" and learning mathematics, there was a small group of
"Yobos" silently kicking ass and churning out great software. CMUCL got forked
to SBCL, added unicode support and threads, not to mention easy building,
SLIME was a new Emacs mode better than anything before and since, Cliki was
launched, C-L.net, and the #lisp IRC channel was born and hit puberty
overnight. Perfect ecosystem.

Today, Lisp is nothing like what it was 8,7,6, even 2 years ago. It's not just
"good" in the well-explored text book fashion; no, it's _good shit_. Get work
done good. Think, hack, ship, bill for it good. 2-3 products per month good.
You still have to know where things are, who is working on what, what's
maintained and what's obsoleted by what. Sure. But there is absolutely no lack
of libraries.

~~~
coffeemug
Come on, man. If you're doing something worthwhile, it likely takes 2-3 months
(if you're lucky) to just understand the problem. Common Lisp is an amazing
language and development environment, but it won't help you actually solve the
problem any faster, only to implement the solution. No silver bullets and all.
I love Common Lisp, but the whole "code this in a weekend" thing has got to
stop. Good software takes a damn long time to develop in _any_ language,
because the language isn't the bottleneck.

~~~
gruseom
I agree with both of you. Mahmud at that point was talking about all those
projects that _aren't_ worthwhile, by your (our) crazy definition.

Edit: but yeah, the "coding over a weekend" thing annoys the hell out of me
too... but that's orthogonal to Lisp.

~~~
mahmud
I worked 8 months on a small-business management package in MFC/C++ using the
"RAD" Visual Studio 6. That was the last piece of major code that I have
written in something other than Lisp. Immediately after that, I wrote a
similar package in CL with a web-based interface, using the free allegro
serve; this one took me about a month, though it was functional after the
first week.

I know what you're thinking; the previous project gave you the hindsight
necessary to implement the second so quickly. Well, not really. The stuff I
did in Lisp where things that have been on my requirements for the first
project but was never able to implement without huge investments in time. MFC
is the most brittle, most fickle and fidgety POS of all time; it generates the
boilerplate for you and it expects to you to use its skeletons just the way
they are. It took me months to experiment with the various data access APIs,
ADO, DAO, and ODBC, moving from one to the next as problems arose, and each
change of backend requiring a complete code overhaul.

Now I write database code in native idiomatic code; I can change the backend
with a feature. I would sometimes forget to #+postgres on deployed code and
find everything working but not see any change in the postgres logs, just to
realize I have :sqlite3 pushed into my __features __;-)

------
pcof
I predict you will get a great deal of pythonic fire. Nevertheless, I find it
very nice that you plan to deal with the specific libraries you're using in
the next posts. The greatest problem in adopting CL seems to be the extremely
decentralized (not to say disorganized) library spacs.

~~~
benhoyt

       "Python 2 is fine now – but in the coming months and years new libraries,
       features, and performance improvements are only going to be introduced
       in Python 3, and I didn’t want to get left behind or forced to take on
       an expensive and time consuming port in the future."
    

Does this argument hold some water, or is Python 2-to-3 conversion often
pretty trivial in practice?

~~~
etal
Word on the street is that the 2to3 tool is magical. Just run the Py2.6
interpreter with the "-3" flag for warnings, fix the warnings, and run the
2to3 tool to get your port.

------
wgj
Some of the conclusions are inconsistent. For example, Ruby got canned mainly
for having a heavyweight framework, since he wants to write everything from
scratch. What keeps one from doing that in Ruby? Ruby does not depend on
Rails.

His complaint about different versions and implementations is a red herring.
Lisp is not immune from this.

~~~
smanek
There are different implementations, of course. But the the multiple version
problem was worked out over a decade ago.

------
scott_s
Python's multiprocessing library
(<http://docs.python.org/library/multiprocessing.html>) provides a similar
interface that one would expect from a multithreaded library. For example,

    
    
      from multiprocessing import Pool
      p = Pool(5)
      def f(x):
        return x*x
    
      p.map(f, [1,2,3])
    

I do not have direct experience with the library, but from what I've seen, the
code you would write is not significantly different from the code you'd write
using a decent multithreading interface.

~~~
fauigerzigerk
Multiple processes cannot share regular in memory data structures. That makes
the multiprocessing library unsuitable for a growing class of applications.
However, unladen swallow aims to remove the GIL, so there's still hope for
Python.

~~~
scott_s
There is an API for both communicating data explicitly, and for allocating
shared memory data structures which can be shared implicitly.

~~~
fauigerzigerk
I know. That's why I said "regular in-memory data structures".

------
pge
I appreciated the article, but I didn't find the reasoning compelling. What I
took away from the article was the following line of reasoning: 1\. You
strongly prefer functional programming to OO programming (no reason given
other than personal preference) 2\. You don't have enough experience with any
languages that are well-suited to FP other than CL that you would be
comfortable undertaking a large project 3\. You confirmed that many languages
you are familiar with (Java, Python, Ruby) are better suited to OO than FP
style (not a surprise at all except perhaps Python as I touch on below), so as
a direct consequence of (1) and (2), you were left with CL.

CL is my first-choice language for certain kinds of projects as well, but I
didn't come away any new insights from your article. Your choice seems to
hinge around two key conclusions for which you give very little explanation:
FP is better than OO for your project, and Python is not a good language for
FP. I single out Python because I don't think anyone would argue that Java or
Ruby is well-suited to FP. Guido insists that Python is not an FP language,
but many users (myself included) would argue that it is reasonably well-suited
for FP. That FP was better for your project does not require any further
explanation. As others have noted, for a project where scalability of the
development team is not an issue (i.e. you are the primary or only developer),
it makes sense to choose the style and language you are most comfortable with.
If you were starting a project that was going to require several years and a
large number of developers, I would have found it more surprising if you
committed to a functional programming model. But, given the nature of the
project, it didn't seem like there was any need to defend that decision
further.

All in all, I'm very excited to see CL being used for more commercial web
apps, but I didn't find any insight in the thought process you laid out.

I am, however, much more interested in Pt. II - your description of rolling
out a web app in CL. I learned CL ten years before I learned Python and would
prefer to write in CL when possible, but for web apps, I have turned to Python
for a number of reasons, most notably the obvious availability of libraries. I
would love to hear what you used and hear a cogent argument that rapid
commercial web app deployment can be done in CL as easily as in Python.

Thanks for sharing your story.

------
IgorPartola
I forget where this came from but this applies 100%:

You ask a person to rate 5 paintings 1-5. You then tell them that you have
spare paintings that they happened to have rated as 3 and 4 and ask them if
they would like one. They will take a "4". Come back in a week and ask them to
rate the paintings again. The "4" becomes a "5" and the "3" becomes a "2".

Our brains will rationalize decisions to make us happy (whatever that means).

~~~
revorad
You may be referring to this TED talk by Dan Gilbert -
[http://www.ted.com/talks/lang/eng/dan_gilbert_asks_why_are_w...](http://www.ted.com/talks/lang/eng/dan_gilbert_asks_why_are_we_happy.html)

------
helium
Ok, so Rails is too heavy weight for you? What about Merb? Or Sinatra, which
was designed for just that reason? Dismissing Ruby as language because only
one of a dozen available web frameworks is too "heavy-weight"sounds a bit
short-sighted to me.

~~~
Tichy
I am not the OP, but I have a problem with Rails and alternatives: there does
not seem to be a viable alternative to ActiveRecord around? I looked into
DataMapper (I think that was it's name) which initially looked great. But then
I could not find any information on how to use transactions, and also received
no answers on the newsgroup. I guess I should have read the source, but I gave
up at that point.

Since I hate ActiveRecord, I have now subconsciously decided that Rails
probably isn't for me.

It's great that Sinatra is spawning lots of imitators for other languages,
though.

~~~
joevandyk
ActiveRecord is a small part of Rails. Lots of people use Rails without using
ActiveRecord.

See <http://whitepages.com> for an example.

~~~
Tichy
I know, but how much is left of Rails without ActiveRecord? I think
validations and forms depend on it.

~~~
joevandyk
Other parts of rails: actionmailer, actionpack, activeresource, and
activesupport.

Forms and validations don't depend on ActiveRecord at all in edge rails.

------
icey
You could probably get rid of the "(fixed)" in your title; I bet most people
didn't see your original submission, and the ones that did will know what
happened.

~~~
smanek
Thanks. Sorry for the trouble before. I had an old personal wordpress blog
that I'd originally posted this on - but there were some technical issues with
upgrading WordPress to a modern version.

Just reposted the same content on Posterous.

------
herdrick
I like the choice. I love to see people using lisps but some of the path to
that choice makes me cringe. Like this:

 _I probably could have written a ‘bare-bones’ implementation of the site’s
back-end in Rails in a week instead of two weeks, but I would rather ‘waste’
that one week up front to have more flexibility later._

YAGNI. Speed kills (the competition).

~~~
gruseom
It was the previous sentence that resonated with me:

 _I’m a bit wary of using ‘heavy weight’ frameworks like Rails (or Django) on
large custom projects. In my experience they make the first 90% of what I’m
trying to do be really easy – but then make the last 10% a living hell since I
need to modify something the framework never intended me to control._

That is a very true statement, in my experience. It's also an elusive insight,
because once you commit to a framework you begin to interpret your problem,
the universe, and everything in terms of the framework's conceptual space.

------
akkartik
No mention of mzscheme?!

------
natmaster
"The completely broken implementation of closures."

That's what the nonlocal keyword is for.

~~~
silentbicycle
When the fundamental design of the language is stacked against your style of
working, the cognitive dissonance of using so many workarounds is a serious
problem.

I just switched to Lua during the Python 2 to 3 transition, and I haven't
looked back. Real closures and tail calls, no GIL, a vastly more tasteful
design, a dead-simple C API, and the whole shebang is a tenth the size of
Python.

~~~
silentbicycle
Meant to add: Lua's not perfect, but its whole design favors embedding it in
another language, so it's a hell of a lot better about gracefully getting out
of the way when it's better to write the parts that matter in other languages.

------
babo
I don't buy the author's argument against Python and clojure but anyway,
that's they system.

