
The Problem with Lisp - heinrichhartman
http://heinrichhartmann.com/blog/problem-lisp.html
======
Tomte
I don't understand the author.

The evaluation order in Lisp is not "highly unpredictable", it is dependent on
the form.

I don't understand why he even constructs this strawman of "you must start in
the innermost sub-form".

That's not a rule that anyone ever made. That's the rule for arithmetic
expressions, sure, but what does that have to do with Lisp?

The rule in Lisp – as he later admits – is "look at the first symbol in the
(outer) form".

And the whole point of special forms is to make your own rules about
evaluations. In order to _help the programmer_.

Can you obfuscate the meaning? Hell, yes! Is that a reason not to allow it and
to force obfuscated code by using unsuitable evaluation orders everywhere?
Hell, no!

And even in this bad example Lisp is certainly not more unpredictable than any
other language.

Let's take C:

    
    
      int func(int a, int b)
      {
        return a + b;
      }
    

If you start at the innermost point "a + b" you get exactly the same
questions: "Oh gosh, what is the + operator?", "How do I apply + to a number?"
(really?) and "where do I store the result?"

I think the author has misled himself into some extreme notion of the divine
simplicity and purity of Lisp that nobody ever claimed to exist.

~~~
heinrichhartman
Author here. Thanks for your comment. I appreciate your feedback!

Let me first comment on your C example:

> "a + b; // Oh gosh, what is the + operator?"

There is no problem with the "+" operator. It's a language builtin. You can
look that up in the manual.

In the Lisp case, I did not pick on the "+" operator, at all. The problem is a
line below:

> "(sum x) ;; ??? Where does "sum" come from?"

There is no function "sum" at all! It just looks like a function call, which
is it not. You have to know what the enclosing special form "let" does to it's
arguments.

You could have those problems with C macros as well, but not in this example.

> Can you obfuscate the meaning? Hell, yes!

> Is that a reason not to allow it and to force obfuscated code by using
> unsuitable evaluation orders everywhere? Hell, no!

I am not picking sides here. I am pinpointing a source of confusion. The
Python/Ruby authors took another path and made the language less powerful, so
they don't give developers the power to obfuscate the code in this way.

~~~
kazinator
Your article is confused between code walking (recognition of forms) and
evaluation order. Code is generally walked top down for the purposes of
expansion of macros, compilation or interpretation. Evaluation of nested
expressions is generally bottom-up, and in well-behaved Lisp dialects, left-
to-right.

You cannot simply let your eyes randomly land on some _(a b c)_ in Lisp code
and assume that a is a function being called with arguments b c. You have to
read the whole sentences and phrases, so to speak.

No language is immune to this. Can you assume that __* X is a pointer
dereference? No, it could be a declarator. Or a fragment of a multiplicative
expression. Or the interior of a comment as in / __* X.

(a, b) could be a comma expression, or the argument list of a function call.

I can't think of any language in which I can extract _any_ substring of a
sentence that could be a syntactic unit on its own, and correctly understand
its meaning without knowing the rest of the sentence.

Lisp has to be scanned from the outside in: we have a defun, which encloses a
let, and so on. When you're familiar with the piece of code you're working on,
you can then take considerable shortcuts. You know you're in the middle of a
function, even though the _defun_ is off the screen. You know you're in the
middle of the _progn_ -like body of some construct (a loop, or let or
whatever), so all the lines of code you see are evaluated forms.

~~~
heinrichhartman
> Your article is confused between code walking (recognition of forms) and
> evaluation order.

Yes, to some extend.

> Can you assume that * X is a pointer dereference?

[Aside: The C declaration syntax is particularly terrible. In a declaration
'star' means "address of", elsewhere it means "dereference".
[https://books.google.de/books?id=bXGhBAAAQBAJ&printsec=front...](https://books.google.de/books?id=bXGhBAAAQBAJ&printsec=frontcover&dq=21st+century+C+pointer+declaration+syntax&hl=en&sa=X&ved=0ahUKEwjil_ShpYTeAhWNiKYKHfZ5CAYQ6AEIJzAA#v=onepage&q=21st%20century%20C%20pointer%20declaration%20syntax&f=false)]

> I can't think of any language in which I can extract any substring of a
> sentence that could be a syntactic unit on its own, and correctly understand
> its meaning without knowing the rest of the sentence.

Well, most languages have expressions/statements that roughly correspond to
source lines. When I read a line of python like this:

    
    
        self.raw_requestline = self.rfile.readline(65537)
    

[https://github.com/django/django/blob/master/django/core/ser...](https://github.com/django/django/blob/master/django/core/servers/basehttp.py#L139)

I can be 100% sure that:

\- `self` is an object with an attribute rfile

\- `=` is the assignment operator

\- there is a method `readline` that get's called with an integer argument
65537.

All this without checking any enclosing socpes.

A random line of lisp:

    
    
       (setcdr current (list :args (cdr current)))
    

[https://github.com/emacs-
mirror/emacs/blob/master/lisp/wid-e...](https://github.com/emacs-
mirror/emacs/blob/master/lisp/wid-edit.el#L742)

could mean _anything_ depending on the enclosing scopes. Pretty sure
setcdr,list,cdr are function calls but who knows. You don't have enough
information to decide this from the snippet provided.

~~~
kbp
> I can be 100% sure that: `self` is an object with an attribute rfile, `=` is
> the assignment operator, there is a method `readline` that get's called with
> an integer argument 65537.

I think this is overstating things, eg, the class `self` belongs to might have
defined __getattribute__, __setattr__, etc, so the things you know for 100%
certain about that line of code in isolation don't really tell you anything
about what it does (self does not definitely have that attribute, and that
might not be assignment, at least); most of that you assume based on common
sense, conventions, figuring the code is "normal", etc, the same as in Lisp.
And that's ignoring a trivial case like if the previous line was `f('''`,
where f may or may or not call eval.

------
lispm
It's a bit like claiming that riding a bicycle is hard because steering,
moving forward and keeping balance at the same time is too difficult. Yet,
people learn it and do it just fine. They just don't think consciously about
it. Riding a bicycle takes a bit of training but after that the body movements
are virtually effortless.

Lisp is slightly more difficult than some other languages because
lists/symbols are both used in code and data, because there are a bunch of
different types of lists, and because code is a token tree.

    
    
      (defun silly (a b c) (let ((my-sum (+ a b c))) (* my-sum (+ 1 my-sum))))
    

Now what's wrong with that? Nothing. But people don't write that.

Lisp is written like that:

    
    
       (defun silly (a b c)
         (let ((my-sum (+ a b c)))
           (* my-sum (+ 1 my-sum))))
    

There are a limited amount of tokens on a line. Syntactic constructs are not
one line.

Any definition has this pattern:

    
    
       (DEFsomething name ...)
    

Oh, it begins with DEF, then it is a macro, followed by the name.

Function definitions have this pattern:

    
    
       (DEFun name arglist body)
    

The code layout is:

    
    
       (defun name arglist
         body
         ...)
         

A method defintion:

    
    
        (DEFmethod name arglist
          body
          ...)
    

The difference between a function and a method is in the arglist.

LET is:

    
    
       (LET   binding-list
         body
         ...)
    

Now you need to know what a binding list is:

    
    
          ((arg0 value0)
           (arg1 value1)
           ...
           (arg2 value3))
    

Such a binding list is used in other places, too.

If you give shortcuts to it:

    
    
      df function definition
      bl binding list
      nm name
      bd body
      al arglist
    

DEFUN:

    
    
      df nm al
       bd
    

LET

    
    
      LET bl
       bd
    

PROG

    
    
      prog bl
       bd
    

lambda

    
    
      LAMBDA al
       bd
    

DESTRUCTURING-BIND

    
    
      DESTRUCTURING-BIND al
         form
        bd
    

BLOCK

    
    
       BLOCK nm
        bd
    

CATCH

    
    
       CATCH nm
        bd
    

etc.

Basically much of Lisp consists of a small number of basic patterns which are
being combined. lists, assoc lists, property lists, arglists, binding lists,
body, ...

You've learned then a new alphabet of visual/structural patterns. Beyond that,
code reading gets intuitive. Names give a clue what kind of pattern
combination follows.

The IDE can then help with syntax highlighting, syntax documentation display,
etc.

Lisp can even show you useful code layout:

    
    
      CL-USER 68 > (let ((*PRINT-RIGHT-MARGIN* 30))
                     (pprint '(defun silly (a b c) (let ((my-sum (+ a b c))) (* my-sum (+ 1 my-sum))))))
    
      (DEFUN SILLY (A B C)
        (LET ((MY-SUM (+ A B C)))
          (* MY-SUM (+ 1 MY-SUM))))

~~~
heinrichhartman
Thanks for this elaboration!

Obviously there is lot of community know-how, that I am not aware of. Thanks
for explaining them in detail. Learning to spot all those pattern is crucial
if you want to be come effective at reading lisp. For the untrained eye lisp
code often looks horrid.

I don't think we disagree, though. All I am saying is that: "Riding the Lisp
bicycle has been difficult for me. Here one thing I struggled with a lot."
Maybe other people have the same difficulty and find this helpful.

~~~
lispm
Right, you need to train yourself with these visual/structural tree patterns.
The code does no longer look horrid and the parentheses will go away.

You were not knowing that there is actual structure and thus you could not see
the forest through the trees.

