

Reading On Lisp: Then and Now - gnosis
http://lispy.wordpress.com/2007/12/27/reading-on-lisp-then-and-now/

======
pge
The Little Schemer book is also a deceptively deep introduction that is a good
preparation for On Lisp. If you work your way through it, you'll really
understand functions that make functions, which is a good foundation before
diving into On Lisp (and, of course, as a bonus, you'll have written an
implementation of the famous Y combinator:)).

------
rickmode
Having beat my head against Common Lisp, then spent a non-trivial amount of
time working in Clojure, I totally agree with his premise.

Learning Lisp requires real, significant use of the language. Eventually you
begin to think in the language and that's when profiency begins.

Merely reading about the language is not enough.

I'm sure this applies to learning any new programming language paradigm (as in
procedural like C, OO like C++, Java and C#, and functional like CL, Clojure
and Erlang) or any natural language for that matter.

[Edit: adding following sidenote.] Reading "On Lisp" was a big part of why I
recently bought the $139 Kindle. Reading the "On Lisp" PDF isn't bad with the
divice in landscape orientation.

~~~
adestefan
> Learning Lisp requires real, significant use of the language. Eventually you
> begin to think in the language and that's when profiency begins.

This applies to any programming language. It's just more pronounced when
switching paradigms. When I was learning python I would write python like it
was C. The programs work, but it's not a very good way to write python. Even
now when I write something quick in any language it ends up being very C-like
since that's the language I use the most.

------
barrkel
"So yes, reading a bottom-up program requires one to understand all the new
operators defined by the author. But this will nearly always be less work than
having to understand all the code that would have been required without them."

I think this quote (from On Lisp) is not true in practice, for most people.
People find it easier to grasp an instance of a pattern or abstraction they
already know, than to grasp a new abstraction altogether. It's only after a
certain amount of repetition of a new pattern of instances that the need, and
understanding, of the new abstraction is understood. Below a certain multiple,
it's better not to use the more compressed encoding.

One measure of how maintainable any given piece of software is might be the
average portion of the whole one needs to understand to modify a part. There's
a tension between reuse (of all kinds) and modularity; reuse increases
dependencies, while modularity bottlenecks them. Modifying something that has
a lot of dependencies is a hazardous undertaking, because you need to
understand more of the program to ensure you get everything right. A piece of
software without such intense reuse, with more redundant encoding, is easier
to modify without understanding the whole - and will probably be less work to
modify too, as the less connected nature of the dependency graph will reduce
the impact area.

~~~
ohyes
"People find it easier to grasp an instance of a pattern or abstraction they
already know, than to grasp a new abstraction altogether."

But the idea is to build the program up until you are working in a set of
target abstractions that fit well with the problem domain. If I don't know the
problem domain (or have materials to help me understand it), I have no
business writing a program for it.

You are always looking for a high level of cohesion and a low level of
coupling. If you already have a natural language that experts talk about the
problem in, it is likely that the set of terminology derived from it has these
properties. (It is well specified, it is precise, work has been done in it). A
trivial example might be the difference between doing statistical work in C
vs. statistical work in R. R is pretty clearly better for it.

Modifying something that has a lot of dependencies is only hazardous if what
that particular something does is underspecified. If it is a bug in one use of
that operator, it should be a bug in every use of that operator. If it isn't,
perhaps the operator has special cases, or perhaps you really want two
different operators (in either case, it needs greater specification).

YMMV, but in my opinion, one should not be modifying a program that one does
not understand (at least mostly). It is easier to modify the 'non bottom up'
program without understanding the whole, because generally, you do not have a
chance at understanding the whole (not really 'whole', more like 'significant
chunk'). You have no other option but to try something and see if it works. It
also seems that you would frequently 'miss' changes that need to be propagated
throughout the code-base (things that are actually bugs in other places, but
perhaps are not manifesting).

~~~
barrkel
"working in a set of target abstractions that fit well with the problem
domain"

I 100% understand the aspiration; I just don't think it works as well as this
in commercial software development, where there are competing concerns, not
just idealized software construction. Leaving some of the less-commonly used
(domain) abstractions out, and instead building them out of more commonly used
(non-domain-specific) abstractions, is what I assert is actually desirable,
but Lisp doesn't help you much here, as it's almost _too_ easy to create new
abstractions. (I'll go further: I think _the principal cause_ of Lisp's
relative lack of success is _the ease with which it lets you build your own
private language_ , and the concomitant difficulty in teaching other people
that language, when you need to bring new people in.)

"If you already have a natural language that experts talk about the problem
in"

Your new hires have a natural language they talk about the domain in - a
programming language, like Java or C#. They are usually not subject matter
experts when they arrive, and certainly they won't be 100% up to date on the
particular dialect chosen for your architecture. That's why it's important to
choose the right abstractions, and not to move too close to the problem
domain.

"statistical work in R. R is pretty clearly better for it."

With concomitant difficulty in finding experts in that language. Whether it is
actually a better choice depends on how often those features are needed, which
again, comes back to my point: eliminate less-used abstractions, and build
them concretely out of more-often used abstractions. So if you don't use
statistics much, but do need to calculate a regression or somesuch, then build
a function or class for that specific thing, don't immediately jump to R.
Every programmer will be familiar with those; bringing in the domain
specificity too early will reduce the degree to which you can leverage human
talent, and ultimately reduce the value of the business.

"if what that particular something does is underspecified"

In commercial development, everything is underspecified and underdocumented.
This won't change.

"one should not be modifying a program that one does not understand (at least
mostly)."

This, frankly, is nonsense, and is the reason I've reacted so much to your
comment; it makes me inclined to think you don't have much commercial
experience. The best way of understanding a program is by fixing bugs in it,
and that will involve modifications to areas of it, _well_ in advance of
understanding all or most of it.

~~~
ohyes
"I 100% understand the aspiration; I just don't think it works as well as this
in commercial software development, where there are competing concerns, not
just idealized software construction. Leaving some of the less-commonly used
(domain) abstractions out, and instead building them out of more commonly used
(non-domain-specific) abstractions, is what I assert is actually desirable,
but Lisp doesn't help you much here, as it's almost too easy to create new
abstractions. (I'll go further: I think the principal cause of Lisp's relative
lack of success is the ease with which it lets you build your own private
language, and the concomitant difficulty in teaching other people that
language, when you need to bring new people in.)"

I think we agree on everything except for what is desirable. Macros certainly
allow you to do some interesting things with control constructs, but I have to
disagree that they are really _that_ big of a help in constructing your own
private language.

I can imagine doing bottom up programming in the style of lisp in a language
like C quite easily. The only difference is you wouldn't get to write your own
control structures.

"Your new hires have a natural language they talk about the domain in - a
programming language, like Java or C#. They are usually not subject matter
experts when they arrive, and certainly they won't be 100% up to date on the
particular dialect chosen for your architecture. That's why it's important to
choose the right abstractions, and not to move too close to the problem
domain."

I guess it depends on your business model. If you are a small shop and you
expect to have a high retention rate of a few experts that you pay well, I
don't see 'new hire' turnover as a big deal. In addition to that, it doesn't
seem to me that anyone really knows how to 'program' once they get out of
college, unless they were doing it before college.

Your CS student from college is going to need mentoring, just as your
mechanical or civil engineer will need mentoring at the beginning of his or
her career. If you create the language in a reasonable way (using a functional
approach rather than creating piles of macros). It seems it should be possible
to read the program just as any other. As you learn about the problem domain
and become an expert in it, the program should become easier to understand.

"In commercial development, everything is underspecified and underdocumented.
This won't change."

The specification should be in a constant flux of improvement. The initial
specification is bad. The finished product specification should be strong.
Iterate and get a lot of feedback and clarify that feedback, and the
specification has to improve. If it isn't specified as being the wrong thing
to have happen, how can it be a bug?

If I want computer programming to be closer to engineering and further from
voodoo, the only thing to do is to have better specifications and reasons for
doing things.

"This, frankly, is nonsense, and is the reason I've reacted so much to your
comment; it makes me inclined to think you don't have much commercial
experience. The best way of understanding a program is by fixing bugs in it,
and that will involve modifications to areas of it, well in advance of
understanding all or most of it."

I think I communicated badly. I agree completely that fixing bugs is the best
way to learn a program. I intended to imply that you HAVE to understand what
the code that directly interacts with the bug does, in order to actually fix
it. If you don't do that, you have possibly created new bugs.

------
Tycho
_sigh_

Still don't quite get what people mean by a 'closure.' Maybe because I've not
realy tried hands-on programming with something like Lisp. But the concept
seems to oscillate between being duh-obvious, to subtle but comprehensible, to
transcedentally complicated depending on who's explaining it or in what
context.

~~~
sayrer
JavaScript is a pretty good language to illustrate closures, if you use
functions and declare all of your variables at the beginning of each function.
:)

In the demo below, the closure we're interested in is the combination of the
code inside "munger scope", and the environment it resides in. To do its job,
munger must search its lexical environment to find each variable. The lexical
environment is illustrated by the boxes. Searching for "foo", it must first
check munger scope (local), beta scope, alpha scope, and then finally succeeds
with the global scope.

In the code at the bottom, the demo1 and demo2 invocations don't influence
each other. That combination of code+environment providing independent values
is a closure.

    
    
       |-- Global Scope ----------------------------------------------------------------|
       |                                                                                |
       |  var foo = "QUX";                                                              |
       |  var alpha = function () {                                                     |
       |                                                                                |
       |     |-- alpha scope ----------------------------------------------------|      |
       |     |                                                                   |      |
       |     |   var a = "AA";                                                   |      |
       |     |   var beta = function () {                                        |      |
       |     |                                                                   |      |
       |     |      |-- beta scope ---------------------------------------|      |      |
       |     |      |                                                     |      |      |
       |     |      |   var b = "BB";                                     |      |      |
       |     |      |   var munger = function () {                        |      |      |
       |     |      |                                                     |      |      |
       |     |      |      |-- munger scope -----------------------|      |      |      |
       |     |      |      |                                       |      |      |      |
       |     |      |      |    a = a + a;                         |      |      |      |
       |     |      |      |    b = b + b;                         |      |      |      |
       |     |      |      |    return a + " " + b + " " + foo;    |      |      |      |
       |     |      |      |                                       |      |      |      |
       |     |      |      |---------------------------------------|      |      |      |
       |     |      |                                                     |      |      |
       |     |      |   }                                                 |      |      |
       |     |      |                                                     |      |      |
       |     |      |   return munger;                                    |      |      |
       |     |      |                                                     |      |      |
       |     |      |-----------------------------------------------------|      |      |
       |     |                                                                   |      |
       |     |   }                                                               |      |
       |     |                                                                   |      |
       |     |  return beta();                                                   |      |
       |     |                                                                   |      |
       |     |-------------------------------------------------------------------|      |
       |                                                                                |
       |  }                                                                             |
       |                                                                                |
       |  js> var demo1 = alpha();                                                      |
       |  js> var demo2 = alpha();                                                      |
       |  js> demo1()                                                                   |
       |  "AAAA BBBB QUX"                                                               |
       |  js> demo1()                                                                   |
       |  "AAAAAAAA BBBBBBBB QUX"                                                       |
       |  js> demo2()                                                                   |
       |  "AAAA BBBB QUX"                                                               |
       |  js> demo1()                                                                   |
       |  "AAAAAAAAAAAAAAAA BBBBBBBBBBBBBBBB QUX"                                       |
       |                                                                                |
       ----------------------------------------------------------------------------------

~~~
Tycho
Ahhh, I see, so not only does the closure scoop up the value of variables at
'define-time', but each closure keeps track of the state of its variables (its
'free variables?') so that their values may increment over time rather than
reset.

Thanks for you (rather cool) illustration. I tested the code and it also
worked :)

