
Ask PG: Lisp vs Python (2010) - kung-fu-master
It seems that a lot of old school Lispers switching to Python (for example: Peter Norvig). What do you think on Lisp vs Python today?
======
norvig
Peter Norvig here. I came to Python not because I thought it was a
better/acceptable/pragmatic Lisp, but because it was better pseudocode.
Several students claimed that they had a hard time mapping from the pseudocode
in my AI textbook to the Lisp code that Russell and I had online. So I looked
for the language that was most like our pseudocode, and found that Python was
the best match. Then I had to teach myself enough Python to implement the
examples from the textbook. I found that Python was very nice for certain
types of small problems, and had the libraries I needed to integrate with lots
of other stuff, at Google and elsewhere on the net.

I think Lisp still has an edge for larger projects and for applications where
the speed of the compiled code is important. But Python has the edge (with a
large number of students) when the main goal is communication, not programming
per se.

In terms of programming-in-the-large, at Google and elsewhere, I think that
language choice is not as important as all the other choices: if you have the
right overall architecture, the right team of programmers, the right
development process that allows for rapid development with continuous
improvement, then many languages will work for you; if you don't have those
things you're in trouble regardless of your language choice.

~~~
edw519
_In terms of programming-in-the-large, at Google and elsewhere, I think that
language choice is not as important as all the other choices: if you have the
right overall architecture, the right team of programmers, the right
development process that allows for rapid development with continuous
improvement, then many languages will work for you; if you don't have those
things you're in trouble regardless of your language choice._

Thank you, Peter. This is how I have felt for years, but could never find
words that describe it as well as you just did.

Someone should write a program that automatically posts this paragraph at the
top of every language war thread. I think they should write that program in
php :-)

~~~
jules
OTOH, it's not completely irrelevant. If you can launch 10% faster, that's a
big advantage.

~~~
varikin
What makes it possible to launch 10% faster? Is it familiarity with the
language, tools, libraries, and frameworks? Is it that the language is more
dynamic and requires less to accomplish task X? I would say yes to both
questions.

Now with the first question, familiarity with the language, lets discuss
Python. Python is my language of choice. I know it, I use it, I pay attention
to what is happening with the language, the community, etc. I organize a
Python User Group. If I were to work on a new project, I would choose Python.
I believe I could launch 10% if not 50% faster with Python than with Ruby.
Does that make Python a better language? No. Just that I don't know Ruby,
Rails, Sinatra, etc.

I think the second question about how dynamic the language is plays into this
as well. Java is not very dynamic. Some tasks in Java are downright ugly to
work with. If you want to prototype fast and spit out a product fast, you
might not want to use Java. At that point, there are several great languages
to choose from like Python, Ruby, Groovy, Scala, Lisp, Smalltalk, Javascript,
Closure, etc. The list just goes on. You could also argue that .net is more
dynamic. But Java has some good points to it as well. You have less risk of
bugs due to it being statically and strictly typed. You can build off some
great libraries. You get the benefit of the JVM and JIT, which are amazing.
Maybe you can't launch faster with Java, but maybe you can launch with a more
robust stable product. First doesn't mean best; remember the article about
Wasabi and Mint?

The way I read Peter's comment was you don't want to spend you time finding
the _best_ language. Instead you want to just grab the most convenient
language and spend your time creating the best product. Think through your
architecture, work on your design, test the hell product, study the market,
your competitors, work on your marketing, and on and on. The list of things to
do never end, and if you were still debating about which language to choose,
you won't launch.

~~~
jules
I agree. Language choice is definitely not the most important factor in
success, but it's not as unimportant as some think it is.

Some languages make it easier to become familiar with the libraries. Compare
the standard libraries of Ruby and Python with PHP. Even though I programmed
in PHP for several years I always had to look up argument order, or whether
the names of functions contained underscores between words or not.

------
francoisdevlin
I'm a Clojure guy that just wrote my first Pylons app. Here's my impression:

1\. Python doesn't suck. I was able to mix FP & OOP approaches to get to my
goal fairly quickly.

2\. iPython was fun to use, helped out a lot, but it's not SLIME.

3\. Guido has an excellent goal with making code readable, and significant
white space is not a bad choice. However, I find being able to analyze active
data structures in a Clojure namespace to be a superior way to learn about a
system.

4\. Python's libraries are pretty good, and it's already been written. As a
first impression, Python libs are much better to use than Clojure wrapped java
libs. I'm going to look into porting SQLAlachemy to Clojure, it rocks.

5\. Paster has a ton of functionality. I'd like to see a similar Clojure tool,
maybe Lien can evolve into that.

6\. I would like to see more FP constructs natively available in Python.

7\. __method__ is an interesting convention. You can have an object implement
the right method, and your object works with Python syntax. However, I find it
to be a poor man's version of Clojure's protocols (Full Disclojure, I have a
conflict of interests here).

8\. Decorators are an interesting way to do functional composition, but I
prefer comp and monads. Way more versatile.

9\. INSERT MACRO RANT HERE

That's all I've got for now. I'm sure I forgot something.

SFD

edit: grammar & spelling

~~~
pjscott
It's funny, but the thing I miss most from Common Lisp when I write in other
languages is the LOOP macro. It's ugly, and non-lispy, but most loops I have
to write can be expressed clearly and concisely using LOOP, and writing the
equivalent code in another language is annoying.

I'm tempted to create a LOOP clone for Clojure, then laugh villainously as I
unleash it upon the world.

~~~
sedachv
Loop, like structured editing (Paredit), is one of those Interlisp things
that's very controversial and divisive.

There's tons of really wild ideas in Interlisp that seemed to be the half-
baked acid trip ideas of West Coast hippies at the time, that are just
starting to become rediscovered in the past couple of years (pervasive undo ->
reversible debugging, DWIM-like autosuggestions in more places), and even the
implementation techniques used are still innovative (for example the error-
trapping implementation of Conversational Lisp
([http://docs.google.com/viewer?a=v&q=cache:4GnEnGS2XXkJ:c...](http://docs.google.com/viewer?a=v&q=cache:4GnEnGS2XXkJ:citeseerx.ist.psu.edu/viewdoc/download%3Fdoi%3D10.1.1.99.5401%26rep%3Drep1%26type%3Dpdf+%22conversational+lisp%22&hl=en&gl=ca&pid=bl&srcid=ADGEEShXWxn-
mRgD0X-rMQN8IFezuWx7CRduPKQjuZWX2Qo1-BneEpeKTwpl5mMBxygVHRFiPkx37Sq3sY-hZN-
GusSJAlZAXqxpSwjgF4uK0sXwFAb_zoygkiNO9MinjQxudj8C0nF5&sig=AHIEtbQiXOK77aVM2PKV9gYI-
ek1oG1q1A)) is quite similar to how Geoff Wozniak approached auto-defining
functions ([http://exploring-lisp.blogspot.com/2008/01/auto-defining-
fun...](http://exploring-lisp.blogspot.com/2008/01/auto-defining-
functions.html\)))

~~~
lispm
LOOP is not from Interlisp. It comes straight from Maclisp.

'LOOPS' from Interlisp is something entirely different: an object-oriented
extension to Interlisp.

~~~
sedachv
I'm going by the Hyperspec and what I remember from reading Kaisler's
Interlisp. From the former:

"One of the Interlisp ideas that influenced Common Lisp was an iteration
construct implemented by Warren Teitelman that inspired the loop macro used
both on the Lisp Machines and in MacLisp, and now in Common Lisp."

[http://www.lispworks.com/documentation/HyperSpec/Body/01_ab....](http://www.lispworks.com/documentation/HyperSpec/Body/01_ab.htm)

~~~
lispm
It influenced it the Maclisp LOOP. The idea. That's all. The CL LOOP macro
OTOH is a straight version of the Maclisp version. The MIT version of LOOP
came from the same sources, even.

The Interlisp iteration facility looks slightly different. There are Interlisp
manuals as PDF at bitsavers...

------
pg
This question sounds like it's from 2005 rather than 2010. Lisp seems to have
become fashionable again now, thanks to Clojure.

I'm sure Python has very good libraries, but I would find it constraining to
program in a language without proper macros.

~~~
metageek
The problem with the Blub paradox is that there's no total ordering.

I do Common Lisp and C++ at my day job (ITA), and I do much of my personal
hacking in Python. In Python and C++ I miss macros; in Lisp and Python I miss
RAII and strong typing; in Lisp and C++ I miss dictionary literals.

And, in all of them, I miss algebraic datatypes.

~~~
pg
I did mention in the original essay that it was only a partial order, in
footnote [4].

~~~
tome
Ah, but metageek's point is that it's not an order at all. It contains cycles.

------
poet
An approximation of some of Norvig's recent thoughts (Feb 2010):

"(1) It just turned out that when Google was started, the core programmers
were C++ programmers and they were very effective. Part of it is a little bit
of culture. (2) Early Lisp programmers (Erann Gat) at Google actually noticed
that other programmers were equally or more productive. It has more to do with
the programmer; we're getting to the point where language choice is less
important (as opposed to 20 years ago). (3) Lisp is optimized for a single
programmer or a small group of programmers doing exploratory work... If I want
to make a change in a weekend I'd rather do it in Lisp than anything else, but
by the time you get up to hundreds of programers making changes are not a
language problem but a social one. (4) Libraries."

Paraphrased from: <http://www.youtube.com/watch?v=hE7k0_9k0VA#t=03m20s>.

~~~
ced
That reminds me of a cool story, in Norvig's talk about Python...

 _When he finished Peter [Norvig] took questions and to my surprise called
first on the rumpled old guy who had wandered in just before the talk began
and eased himself into a chair just across the aisle from me and a few rows
up.

This guy had wild white hair and a scraggly white beard and looked hopelessly
lost as if he had gotten separated from the tour group and wandered in mostly
to rest his feet and just a little to see what we were all up to. My first
thought was that he would be terribly disappointed by our bizarre topic and my
second thought was that he would be about the right age, Stanford is just down
the road, I think he is still at Stanford -- could it be?

"Yes, John?" Peter said.

I won't pretend to remember Lisp inventor John McCarthy's exact words which is
odd because there were only about ten but he simply asked if Python could
gracefully manipulate Python code as data.

"No, John, it can't," said Peter and nothing more, graciously assenting to the
professor's critique, and McCarthy said no more though Peter waited a moment
to see if he would and in the silence a thousand words were said._

[http://smuglispweeny.blogspot.com/2008/02/ooh-ooh-my-turn-
wh...](http://smuglispweeny.blogspot.com/2008/02/ooh-ooh-my-turn-why-
lisp.html)

~~~
bootload
_"... My first thought was that he would be terribly disappointed by our
bizarre topic and my second thought was that he would be about the right age,
Stanford is just down the road, I think he is still at Stanford -- could it
be? ..."_

I've often wondered why McCarthy has never been asked to Startup school to
talk about developing and using Lisp and the advantages?

~~~
metacontent
He is probably not interested in talking about stuff when he could be doing
stuff.

------
jerf
The Lisp vs. Python story really hasn't changed terribly much in the past five
years or so. Both are still great languages once you learn to speak the idioms
of the language. Both languages have persistent problems with people refusing
to do so and then bitching that it's not $SOME_OTHER_LANGUAGE. Both languages
have places where I'd suggest one of them over the other. Neither language is
even close to a replacement for the other, and programming Lisp in Python is
as big a mistake as programming Python in Lisp.

(Of the two though, Python seems to have more problems with people refusing to
speak the native idioms and insisting on writing $LANGUAGE in Python instead.
Python Is Not A Functional Language. It is a multiparadigm language where the
functional is definitely the foreign and borrowed paradigm on top of an
imperative/OO core. Ignoring that will bring you grief, but it won't be
Python's fault.)

Later edit: In fact, refusing to speak Python's native idioms has been getting
noticeably worse in the last six months. If you want a solid OO language with
some decent functional borrowing, learn Python. If you want a pure functional
language for whatever reason, do us all a favor and don't learn Python. Or at
least don't add to the chorus of people complaining Python isn't Haskell, just
_go learn Haskell_. Or Clojure, or whatever.

~~~
lisper
> The Lisp vs. Python story really hasn't changed terribly much in the past
> five years or so.

Actually there have been three significant developments in the last five years
that IMO tilt the scales back over to the Lisp side:

1\. Clojure

2\. Clozure Common Lisp a.k.a. CCL (a very unfortunate confluence of names --
the fact that Clojure and Clozure differ by only one letter but otherwise bear
almost no resemblance to each other causes no end of confusion).

3\. The state of Common Lisp libraries has gotten a LOT better in the last
five years.

~~~
jerf
Fair about Clojure, I was assuming Common Lisp. Point 3 I consider not much
net change, though, as the same is true of Python, and pretty much every other
competitive language.

~~~
lisper
Before, Lisp had the edge in native code compilation and functional
programming, Python had the edge in libraries. In the modern landscape, that
made Python win IMO. Now CL has caught up to Python (mostly -- it's 90% of the
way there) in libraries and retains its edge in the other two areas, so in my
book CL has pulled back into the lead. YMMV.

------
DanWeinreb
I tried writing an extensive Common Lisp vs. Python paper (yes, I have seen
all of the existing ones), but it got too big and out of hand. One of
interesting developments in Python recently are decorators, which allow some
interesting metaprogramming. This can do some things that Lisp macros are used
for. It's still not Lisp macros, but it;s picking off the low-hanging fruit,
which helps Python coders a lot.

~~~
fogus
Any chance we might see the draft? :-)

~~~
DanWeinreb
Oh, it's so ugly, and it hasn't been checked for mistakes, etc. I'd be
embarrassed to have my name associated with it.

~~~
kroger
I read you comment quickly and only now realized it's from you (Dan Weinreb,
über lisp guru of ITA fame). I understand your felling (to put an early draft
out). But, there is any chance you could send a copy to interested people who
know the quality of your work (the man worked on the Common Lisp specification
for god's sake!) and will read it as an early draft (no citations allowed ;-)
and will promise not to send the draft to anyone? ;-)

~~~
cma
You would probably entangle him in a two-hundred thousand dollar bet with
Scott Aaronson.

------
drcode
I think most people switching from Lisp to Python do so for practical reasons.
I suspect Peter Norvig probably wouldn't use as much Python if he didn't have
to adapt to the Google culture.

I think Python is a surprisingly nice language in many ways. It falls into the
category of being an "acceptable Lisp". But I think most Lispers still prefer
the real thing and only use Python when they need to be pragmatic.

~~~
poet
I don't know of any Lisp programmer who thinks Python in an acceptable Lisp.
In fact, there are some things about Python that are completely unacceptable
as far as a Lisp goes: (1) crippled lambda, (2) no easy way to pass chunks of
code around, etc.... Obviously not having macros is a given in anything but
Lisp. Ruby (because of blocks) is far more of an acceptable Lisp.

~~~
bad_user
> _Obviously not having macros is a given in anything but Lisp_

At least these languages (that I know of) have macros:

    
    
        http://boo.codehaus.org
        http://nemerle.org
        http://www.perl6.org

~~~
drcode
True, but the syntax for all three of these is pretty hairy.

Lisps might not be the only languages with macros, but they're the only ones
with elegant and easy to use macro systems.

You could argue of course this is a moot point, since even Lispers only write
macros infrequently. So who cares if it's hard to write one on the rare
occasion you need to?

~~~
middayc
I am amazed that people say Lisp is about macros. I think it's about being
homoiconic. You can make some ugly form of macros in ANY language.
_Homoiconic_ -ness is what makes you want to make macros with Lisp. But it is
much more than just about macros. Language REBOL is for example fully
homoiconic.

    
    
      >> a: [ 300 3 ]
      >> insert a 'add
      >> probe a
      [ add 1 2 ]
      >> print first a
      add
      >> do first a 100 1
      == 101
      >> do a
      == 303
      >> f: func [] a
      >> f
      == 3
      >> change second :f 'subtract
      >> f
      == 297
    

all code is data (blocks of words and values).

From * Io * example below I would say it's a little different. It's more like
it has a runtime api to change/generate it's runtime. I think SmallTalk has
something similar to this.

* Factor * has compile time macros. At runtime it has quotations, which are blocks of code (separate from it's other data strucures I think, but don't shoot me if I'm wrong) that can be modified and executed by your code with few specific words like _curry_ and _compose_. This means you have a little less freedom than in rebol where block of code is in no way different than a block of data. What is awesome about factor is that it also compiles quotations at runtime.

Otherwise Factor is very very cool, and I envy some runtime features of Io a
lot.

And Python has as little to do with lisp as Visual Basic. Python is the
world's best dynamic _Imperative_ lang IMHO :)

~~~
draegtun
_From Io example below I would say it's a little different. It's more like it
has a runtime api to change/generate it's runtime. I think SmallTalk has
something similar to this._

In Io, _all code is data_. Below is an example of changing a _functions_
behaviour (from addition to subtraction):

    
    
        Io> plus := block (a, b, a + b)
        ==> method(a, b, 
            a + b
        )
        Io> plus call (1, 2)
        ==> 3
        Io> plus message next setName ("-")
        ==> -(b)
        Io> plus
        ==> method(a, b, 
            a - b
        )
        Io> plus call (1, 2)
        ==> -1
    

ref: _Io Has A Very Clean Mirror_ (WayBackMachine copy) -
[http://web.archive.org/web/20080212010904/http://hackety.org...](http://web.archive.org/web/20080212010904/http://hackety.org/2008/01/05/ioHasAVeryCleanMirror.html)

~~~
middayc
Very cool!

There are few properties about concurrency, coroutines, embed-ability, and I
suppose nice process of making bindings that I value really a lot and Io HAS.
Looking at your example, I will definitely look again at Io. Thanks!

------
Goladus
I've used Python for about 4 years and am just starting to use Clojure, so
I'll just add a few comments that others haven't mentioned. I'm not trying to
offer a definitive comparison.

Clojure's data structures seem a lot like Python's but are a bit more elegant
and avoid many of the little python headaches that come up often like d['k']
vs d.k vs d('k'). In clojure it would be (d :k) or (:k d) and both work. If
you need memoization there are functions to help you. In clojure there
definitely seems to be an attempt to make all the core data structures as
compatible as possible (even having a formal abstraction (iSeq) for all of
them)

Culturally, Python seems to care more about a minimal core language.
Clojure.core has probably 3-4 times as many built-ins as Python. Many of the
clojure functions are supporting features Python doesn't have or handles with
syntax, like macros and conditional expressions, but there are also clojure
functions like even?, that probably won't ever be a Python built-in.

Especially for predicates, functions like even?, every?, ffirst,

~~~
masklinn
> and avoid many of the little python headaches that come up often like d['k']
> vs d.k vs d('k').

When does that come up? d['k'] is a key access (to a collection), d.k is an
attribute access (to an object) and d('k') is a method call. I'm not sure
where the headache/confusion would be here, unless you're trying to do very
weird things with Python (which you should not)

~~~
Goladus
They are all mapping functions, though. You have a key 'k', which is accepted
by d and will return a unique result. Why should I have to care whether d is a
collection, object, or method? In fact in many cases it's pretty easy to
implement all three.

    
    
        class months(object):
            def __init__(self):
                m='jan feb mar apr may jun jul aug sep oct nov dec'.split(' ')
                n=range(1,13)
                self.__dict__.update(dict(zip(m,n)))
            def __getitem__(self, key):
                return self.__dict__[key]
            def __call__(self, key):
                return self.__dict__[key]
        
        d=months()
        
        print d['jan']  # 1
        print d('jan')  # 1
        print d.jan     # 1

~~~
masklinn
> I'm not sure where the headache/confusion would be here, unless you're
> trying to do very weird things with Python (which you should not)

~~~
Goladus
I never said there was confusion. The headache comes when you're trying decide
whether to present an interface to a mapping function as a collection, an
object, or a function.

------
manish
Quotes from PG's Essays: _If you look at these languages in order, Java, Perl,
Python, you notice an interesting pattern. At least, you notice this pattern
if you are a Lisp hacker. Each one is progressively more like Lisp. Python
copies even features that many Lisp hackers consider to be mistakes. You could
translate simple Lisp programs into Python line for line. It's 2002, and
programming languages have almost caught up with 1958._ _Macros (in the Lisp
sense) are still, as far as I know, unique to Lisp. This is partly because in
order to have macros you probably have to make your language look as strange
as Lisp. It may also be because if you do add that final increment of power,
you can no longer claim to have invented a new language, but only a new
dialect of Lisp._ I think it is true even today

------
xsltuser2010
When I last tried Lisp (SBCL) i was surprised how hard it is to find standard,
common sense 'batteries' as you would in python, and how hard it is to get
things going if you're not experienced. I think the point where I jumped out
of the boat was trying out 'Hunchentoot' or something like that (webserver).

I haven't seen that with Python, Ruby, C++, Java, even Haskell feels 'modern'
in that way. Why must it be so hard to get simple stuff going ?

(Note: I like function programming concepts, but I would expect a programming
language to be easier to bind into a context of reality.)

~~~
dons
> standard, common sense 'batteries' ... even Haskell feels 'modern' in that
> way

Glad you noticed! We've been working hard on this:
<http://haskell.org/platform>

------
draegtun
For the macro side of things you may find these two old HN posts of interest:

* _Why Lisp macros are cool, a Perl perspective (2005)_ <http://news.ycombinator.com/item?id=795344>

* _Can a language have Lisp's powerful macros without the parentheses?_ <http://news.ycombinator.com/item?id=9172>

------
someonetwo
Python versus Lisp is not so much a dichotomy. Some bridges:

1.- Mathematica: you can use both infix and prefix form, Fullform[a+b] =
[Plus,a,b]. Mathematica internally use prefix notation. Evaluation is more
flexible than Lisp, you can define a function and decide whether it evaluates
some, all or none of its arguments. 2.- Maxima: A layer over Lisp to define a
infix language,in which you can define operators to resemble math notation,
for example f(x):= x^2 similar to (defun f(x)(* x x)) 3.- Dylan. A lisp with
infix notation. 4.- Willem Broekema cl-python, python in Lisp. 5.- Clojure.
Clojure brings some nice syntax for getters and setters, function arguments
and much more. 6.- comp.lang.lisp versus clojure. Clojure has a great
community, lisp has some problems with lords. 7.- abcl is here, that is Lisp
in java. abcl can run maxima without errors and that is great. 7.- Ruby,
jruby, ioke, duby, those are efforts to achieve a very expressible language.
8.- javascript, the good parts. javascript with some anotations can be the
next lisp. 9.- quick-lisp for a better installer than asdf.

------
ntoshev
There is a language and there is an ecosystem surrounding it (libraries,
community, etc). Ignoring the ecosystem, the question "to lisp or not to lisp"
pretty much boils down to "syntax or macros" - if you want macros, you go with
a lisp, if you want syntax, you go with Python or another modern language.

I used to think macros matter more than syntax, because you can freely define
your own micro-languages. I didn't really practice it, because of the
practical limitations of the available lisps [1]. Now I think the opposite,
that syntax matters more. Syntax helps you parse the code visually and you use
lower level parts of your cortex to understand the code [2]. You can build
arbitrary DSLs in lisps, but they all have no syntax, so they are of limited
cognitive help. I think the real win are modern languages with syntax, that is
malleable enough to facilitate the cognitive apparatus of the programmer in
most cases, or at least most cases that matter. For example, an obvious DSL is
the mathematical notation - Python / Ruby handle it well enough with operator
overloading, Lisp actually does worse because of the prefix notation.

It is important to understand that you can approximate the bottom-up style of
building abstractions with libraries (instead of DSLs), parameterizing the
proper things, with minimal syntax noise. The remaining difference between
macros and using higher level functions is mostly in run time optimization.

I guess seasoned lispers learn to "see through" all the brackets and engage
the lower-level part of the brain in parsing lisp code. Ironically, something
similar happens to Java developers - after enough hours looking at the code
they start to ignore the ugly try/catch clauses that can't be properly
abstracted away because of language limitatons.

[1] with the exception of one big project in Common Lisp, but I did only a
little programming in it, and this was before I fully appreciated macros - but
the guy before me used them extensively to build two layers of domain specific
languages

[2] L Peter Deutsch talks about this in Coders at Work and this is probably
more valuable than what I have to say:

Deutsch: I can tell you why I don’t want to work with Lisp syntax anymore.
There are two reasons. Number one, and I alluded to this earlier, is that the
older I’ve gotten, the more important it is to me that the density of
information per square inch in front of my face is high. The density of
information per square inch in infix languages is higher than in Lisp.

Seibel: But almost all languages are, in fact, prefix, except for a small
handful of arithmetic operators.

Deutsch: That’s not actually true. In Python, for example, it’s not true for
list, tuple, and dictionary construction. That’s done with bracketing. String
formatting is done infix.

Seibel: As it is in Common Lisp with FORMAT.

Deutsch: OK, right. But the things that aren’t done infix; the common ones,
being loops and conditionals, are not prefix. They’re done by alternating
keywords and what it is they apply to. In that respect they are actually more
verbose than Lisp. But that brings me to the other half, the other reason why
I like Python syntax better, which is that Lisp is lexically pretty
monotonous.

Seibel: I think Larry Wall described it as a bowl of oatmeal with fingernail
clippings in it.

Deutsch: Well, my description of Perl is something that looks like it came out
of the wrong end of a dog. I think Larry Wall has a lot of nerve talking about
language design—Perl is an abomination as a language. But let’s not go there.
If you look at a piece of Lisp code, in order to extract its meaning there are
two things that you have to do that you don’t have to do in a language like
Python.

First you have to filter out all those damn parentheses. It’s not intellectual
work but your brain does understanding at multiple levels and I think the
first thing it does is symbol recognition. So it’s going to recognize all
those parenthesis symbols and then you have to filter them out at a higher
level.

So you’re making the brain symbol-recognition mechanism do extra work. These
days it may be that the arithmetic functions in Lisp are actually spelled with
their common names, I mean, you write plus sign and multiply sign and so
forth.

Seibel: Yes.

Deutsch: Alright, so the second thing I was going to say you have to do, you
don’t actually have to do anymore, which is understanding those things using
token recognition rather than symbol recognition, which also happens at a
higher level in your brain. Then there’s a third thing, which may seem like a
small thing but I don’t think it is. Which is that in an infix world, every
operator is next to both of its operands. In a prefix world it isn’t. You have
to do more work to see the other operand. You know, these all sound like small
things. But to me the biggest one is the density of information per square
inch.

Seibel: But the fact that Lisp’s basic syntax, the lexical syntax, is pretty
close to the abstract syntax tree of the program does permit the language to
support macros. And macros allow you to create syntactic abstraction, which is
the best way to compress what you’re looking at.

Deutsch: Yes, it is.

Seibel: In my Lisp book I wrote a chapter about parsing binary files, using
ID3 tags in MP3 files as an example. And the nice thing about that is you can
use this style of programming where you take the specification—in this case
the ID3 spec—put parentheses around it, and then make that be the code you
want.

Deutsch: Right.

Seibel: So my description of how to parse an ID3 header is essentially exactly
as many tokens as the specification for an ID3 header.

Deutsch: Well, the interesting thing is I did almost exactly the same thing in
Python. I had a situation where I had to parse really quite a complex file
format. It was one of the more complex music file formats. So in Python I
wrote a set of classes that provided both parsing and pretty printing. The
correspondence between the class construction and the method name is all done
in a common superclass. So this is all done object-oriented; you don’t need a
macro facility. It doesn’t look quite as nice as some other way you might do
it, but what you get is something that is approximately as readable as the
corresponding Lisp macros. There are some things that you can do in a cleaner
and more general way in Lisp. I don’t disagree with that. If you look at the
code for Ghostscript, Ghostscript is all written in C. But it’s C augmented
with hundreds of preprocessor macros. So in effect, in order to write code
that’s going to become part of Ghostscript, you have to learn not only C, but
you have to learn what amounts to an extended language. So you can do things
like that in C; you do them when you have to. It happens in every language. In
Python I have my own what amount to little extensions to Python. They’re not
syntactic extensions; they’re classes, they’re mixins—many of them are mixins
that augment what most people think of as the semantics of the language. You
get one set of facilities for doing that in Python, you get a different set in
Lisp. Some people like one better, some people like the other better.

~~~
lispm
Lisp has a lot syntax. It is just a bit different and it looks different
externally.

 _Lisp has a 2-stage syntax._

 _The first stage is the syntax of s-expressions_ , which is surprisingly
complex. S-Expressions provide a textual syntax for data: symbols, lists,
pairs, strings, various number formats, arrays, characters, pathnames, ...

The first stage is implemented by the 'reader' and can be reprogrammed by an
ancient API to the reader via read tables.

 _The second stage is the syntax of the Lisp programming language._ This is
defined on top of s-expressions and is really a syntax over data structures
(not text). This Lisp syntax deals with: data items, function calls, special
forms (thirty something) and macro forms.

This syntax stage is implemented as part of the interpreter/compiler (EVAL,
COMPILE, COMPILE-FILE) and can be extended by writing macros, symbol macros
and compiler macros. In earlier dialects it could also be extended by writing
so-called FEXPRs, functions which get called with unevaluated source code (->
data in Lisp).

So, we get a lot of complex syntax due to special forms and macros. It just
looks a bit different, since the data syntax is always underneath it (unless
one uses a different reader).

For example a function definition would be:

    
    
       (defun foo (a b) (+ (sin a) (sin b)))
    

The syntax for that is:

    
    
        defun function-name lambda-list [[declaration* | documentation]] form*
    

With more complex syntax for 'function-name', 'lambda-list' and 'declaration'.

Lambda-list has this syntax:

    
    
        lambda-list::= (var* 
                        [&optional {var | (var [init-form [supplied-p-parameter]])}*] 
                        [&rest var] 
                        [&key {var | ({var | (keyword-name var)}
                          [init-form [supplied-p-parameter]])}*
                          [&allow-other-keys]] 
                        [&aux {var | (var [init-form])}*])
    
    

Not every valid Lisp program has an external representation as an s-expression
- because it can be constructed internally and can contain objects which can't
be read back.

Not every s-expression is a valid Lisp program. Actually most s-expressions
are not valid Lisp programs.

For example

    
    
       (defun foo bar)
    

is not a valid Lisp program. It violates the syntax above.

~~~
sedachv
"The first stage is implemented by the 'reader' and can be reprogrammed by an
ancient API to the reader via read tables."

Readtables aren't any more ancient than the rest of ANSI INCITS 226-1994
(R2004) (the language standard formerly known as X3.226-1994 (R1999)), but the
interface to them is very low-level and non-modular.

The current readtable is specified by a dynamic variable, so the readtable
facility can be made modular with a nicer interface, in a portable manner.
This is exactly what the library Named-Readtables does: <http://common-
lisp.net/project/named-readtables/>

Now realize the significance of this: Common Lisp is the only language
allowing total unrestricted syntactic extension and <i>modification</i> in a
modular and (somewhat) composable way.

I've been using named-readtables for the past month, and between it and
Quicklisp, I haven't been this excited about programming in CL since I started
(which is 8 years ago, not that long, but I'm not a total noob either).

~~~
lispm
Read tables existed before CL. CLOS for example not, it was developed for ANSI
CL (based on experience with LOOPS and Flavors). See for example READTABLE in
Maclisp:

<http://maclisp.info/pitmanual/io.html#16.2.7>

Also note that I wrote that the API is ancient. It is. It is old and could be
easier to use.

'Named readtables' are related to 'syntaxes' on the Lisp Machine. For example
source files have a syntax attribute in the header, which switches between the
various Lisp dialects (or other languages), including using different readers.
This is for example used by the file compiler and Zmacs.

~~~
sedachv
Cool, I didn't know readtables were in Maclisp, or about LM syntaxes.

"It is old and could be easier to use."

Aside from something like named-readtables, how would you design the lowest-
level interface to readtables? Or you wouldn't do that, and just specify
something like named-readtables to be the interface? I'm curious because this
could be something for
<http://www.cliki.net/Proposed%20Extensions%20To%20ANSI>

~~~
lispm
I haven't thought about it much, but the character level interface is very
primitive. Second, what about things like symbols, numbers, etc.? There is no
sane way to specify number syntax or symbol syntax. That might be useful.
Currently the reader provides an interface on a character level, but not on
the level of s-expressions components.

~~~
sedachv
What I think you're saying is that the reader should be customizable in terms
of some DSL for a grammar. It would be nice, but I'm not sure how composable
it would be (in the general case, I think it would come down to having the
behavior of READ dependent upon a black-box _current-parser_ procedure).

The nice thing about readtables is that it exposes what in essence are
transition hooks for each character, so you really don't need to care about
the grammar of stuff you're not interested in parsing.

OTOH like you said, extending the syntax for numbers or symbols becomes quite
hairy. But even with a DSL grammar approach, you'd need to change major parts
of the grammar (which means copying and modifying the normative grammar of CL
syntax - not that different from grabbing a portable CL reader implementation
today (<http://twitter.com/vsedach/status/26484049015)>).

The big downside is deciding on which class of grammars that will support, and
how they will be represented.

------
Keyframe
I find it hard to maintain large-ish python codebase - it tends to wire up
itself into a mess (in my case). It's probably due to my inability to do so,
but I don't have same problems with C code.

~~~
spenrose
Is it possible you're comparing lines of code directly? The "size" of a code
base is tricky to define, but many people accept a rough rule of thumb that a
project of complexity X that might require, say, 10K LOC in
competent/straightforward C would require say 2K LOC in
competent/straightforward Python/Ruby/etc. The implication is that an average
10K LOC Python program _would in fact be a more complicated program_ than an
average 10K LOC C program, and therefore "messier", all things being equal.

~~~
Keyframe
I can't really describe it. It's as if literal size grows, it's harder for me
to wrap my head around what is where and why - I suspect it is due to the fact
I don't write very pythonic code... more like I'm writing C code with Python
syntax, if you know what I mean.

I am not particularly familiar with python too, I should educate myself
properly, dwell more into it though. I only get by for what I need to do in
Maya with it.

~~~
spenrose
Ah, right, different problem. There's much to be said and disputed about what
makes a 10K program Pythonic; my oversimplification would be something like:
\- between half a dozen and two dozen files in 2-3 directory levels \- lots of
small classes with short methods \- some short functions \- lots of
dictionaries \- no God objects \- some comprehensions, generators, and
decorators \- about 2-4 levels of abstraction between the problem statement
and the core routines in inner loops \- pivot away from any of the above if
the problem calls for it, but many problems don't

Even simpler: if you're writing short callables (functions/methods, mostly)
and using them as the values in dictionaries, you're most of the way to
writing intermediate Python.

------
pronik
Am I missing something, why does it have to be either Python or Lisp? After
all I know, Peter Norvig has embraced Python in addition to Lisp, rather than
switched completely as in "no more Lisp". Same goes for everyone else -- some
tools are more fit for some tasks than others, especially with corprorate
restrictions being a constraint.

------
garply
Paul, this would be a great opportunity for you to plug Arc, an increasingly
active Anarki (<http://github.com/nex3/arc>), and the growing Arc community
(<http://arclanguage.org/forum>).

~~~
lacker
I feel like the Arc project pivoted until it became YC.

------
fdr
I used to write a reasonable amount of Lisp at my first job, in an industrial
research lab, in addition to a fond exposure to it at my institution of higher
learning.

I really, really like Lisp. The syntax, as a result of it having been
introduced to me fairly early, never was a hangup of any kind. SLIME is quite
wonderful. I always yearn for quasiquotation.

However, I cannot justify using it for a small-scale project that has to ship
relatively soon with limited resources. The reason: Python has useful projects
with good release culture and documentation that are not seen in the Lisp
world, if only for the lack of contributors. (Considering the number of
authors, I found common lisp documentation to be quite good, in fact, but I
still find them pretty hard pressed to compete with, say, the Django manual,
or most of the Python standard library itself)

Software — especially at the small, early stage scale — is still loaded with
problems that are not matters of extensive, inspired, and unique programming.
Or, they could be, but did you really want to write your own whiz-bang form
validation instead of building all the other features of your product?

Clojure I think represents a very interesting escape from this trap, and if I
had a lot of Java dependencies (whether bequeathed or because of unique
library advantages Java may have. WEKA comes to mind for some things) I would
most certainly want to use it. But in a project I control from the beginning,
Python wins, hands down, in a number of domains.

I will also expose one heretical opinion of mine about Lisp Macros: they're
great, but easy to overuse, and seldom So Necessary if one makes good use of
higher order functions. Some code, admittedly, comes out nicer using macros,
but I can almost always get the expressive power I need with little contortion
using higher order functions.

If I had the time to follow my heart, I'd want to do something to continue to
vitalize Lisp. But not while I need to ship.

To round out this post, here are some minor peeves on the Python side though:

* No symbol data type. I don't like using strings for these. I use SymbolType for this, but it's still comparatively clunky compared to CL.

* Hyphens are the best intra-token-delimiter (see what I mean?)

* No macros

* No quasiquotation

* Slow

* Have to avoid stepping on myself because of the mutable environment frames you see a bit more frequently as opposed to say, Scheme or Erlang where tail recursion is the name of the game. I also make use of "labels" in cl.

* Did I mention tail recursion elimination? No? Well...sometimes this would be nice.

------
kqueue
Banana vs Oranges?

~~~
Goladus
Bananas have much more starch than oranges, but oranges are easier to juice.

~~~
justlearning
@Goldus - you seem like a old timer here. why post comments like these? Are
you adding any value to the main conversation - python/lisp.

~~~
Goladus
The point of the comment was to demonstrate that sometimes, it's worthwhile or
at least interesting to compare (or actually, contrast) bananas and oranges,
like Python and Lisp.

(Also, I had already posted my small contribution to the original topic)

