
D-Expressions: Lisp Power, Dylan Style (1999) [pdf] - mpweiher
http://people.csail.mit.edu/jrb/Projects/dexprs.pdf
======
didibus
I know at first it looks confusing, but the lisp syntax is pretty much the
best. I can teach it to someone in 1 minute:

(<function or macro name> arg1 arg2 ...)

That's it. This is 100% of the syntax.

If dealing with a function, the evaluation will always be arguments from left
to right are evaluated first, and then the function is called with the result
of evaluating the args. This is 95% of the cases.

If dealing with a macro, go read the documentation for it, because every macro
is a DSL, and evaluation can happen in different ways. This is the 5% case.

All the complexity of learning a lisp comes from knowing more functions, what
they do and how to use them, and learning each macro's DSL.

Can this syntax be improved on, I'm not so sure. Clojure added some special
syntax for some more literals, such as [] for vectors, {} for map, etc. And I
do think in one way they help, but in others they hurt.

Maybe adding something that distinguishes a macro from a function, but once
you know the names of them, the name is enough to know.

Maybe make it so there is a more visible demarcation between the
function/macro name and the arguments. Not sure how though.

All that to say, if you give the ayntax a good chance, like 2 to 3 months of
40h a week for your brain to more clearly see where the functions start and
end, I'm sure you'll like it.

~~~
seanmcdirmid
Lisp syntax is like no syntax at all, it's all just direct parse trees. It is
unviable without more secondary formatting than syntactic forms since it is
really difficult for the eyes to balance parantheses on their own.

That LISP-style (non) syntax is successful just goes to show how little syntax
really matters (I.e. the null case works), I guess.

~~~
mmalone
I get what you're saying, but that's a pretty extreme view. Parenthesis are
obviously syntax, even if they look similar to the notation used to represent
ASTs in other languages. They are minimal syntax, but they're still syntax
(also, lisps have `"@' and sometimes {}[]).

Just because this is the internet, I'll also disagree with your second point.
Lisp programmers use white space and macros to make programs more readable.
Despite this, lisp's adoption trails many less powerful languages that offer
more intuitive syntax. Since lisp's metaprogramming facilities allow it to
effectively simulate pretty much every other style of programming, the only
explanation that I can come up with for this situation is that intuitive
syntax matters a lot (at least for language adoption).

~~~
seanmcdirmid
Macros aren't so much syntax as they are semantics (inlined for performance,
but we can still think of them as special procedure calls). Lisp has been
relatively successful in its various forms, it should get some credit.

~~~
lispm
Macros serve lots of purposes. One is providing convenient syntax.

    
    
        (loop for i from 10 below 100 step 3
              when (evenp i)
              collect i)
    

Above is using the LOOP macro, which implements a lot of syntax for various
ways to LOOP. The syntax description is actually quite long:

See the EBNF description of the LOOP macro in the standard:

[http://www.lispworks.com/documentation/HyperSpec/Body/m_loop...](http://www.lispworks.com/documentation/HyperSpec/Body/m_loop.htm#loop)

There are other syntactic forms which use more parentheses. For example the
DEFCLASS macro. The basic EBNF syntax for DEFCLASS is:

    
    
        defclass class-name ({superclass-name}*) ({slot-specifier}*) [[class-option]]
    
        slot-specifier::= slot-name | (slot-name [[slot-option]])
    
        slot-name::= symbol
    
        slot-option::= {:reader reader-function-name}* | 
                       {:writer writer-function-name}* | 
                       {:accessor reader-function-name}* | 
                       {:allocation allocation-type} | 
                       {:initarg initarg-name}* | 
                       {:initform form} | 
                       {:type type-specifier} | 
                       {:documentation string} 
    
        function-name::= {symbol | (setf symbol)}
    
        class-option::= (:default-initargs . initarg-list) | 
                        (:documentation string) | 
                        (:metaclass class-name) 
    
    

The DEFCLASS macro implementation will check some of that automatically, but
much of the syntax has to be check by the macro in the implementation code.

> we can still think of them as special procedure calls

That's not a good idea. It's misleading.

------
panic
Another Algol-style language with a macro system in this vein is Honu
([https://www.cs.utah.edu/~rafkind/papers/dissertation.pdf](https://www.cs.utah.edu/~rafkind/papers/dissertation.pdf)).
It goes slightly further, allowing macros to do things like define new infix
operators, and its layered approach (with an "enforest" step interleaved with
parsing) is worth a look.

~~~
mahmud
Holly smoke! The supervisor committee on that thesis is _heavy_. I'm so gonna
study it.

------
patricksli
Hello everyone.

Since there's been interest in the D-expressions paper, I wanted to add that
Jonathan Bachrach (the first author on the paper) is actually my Ph.D.
advisor, and we just released a new programming language a few months ago
called Stanza based on similar ideas. Stanza is a spiritual successor to Dylan
of sorts: it has an optional type system, coroutines, a multimethod-based
object system, programmatic macros, and an s-expression-based natural syntax.
You can check it out at www.lbstanza.org.

    
    
      - Patrick

------
lacampbell
Dylan did so much right. Well - the syntax was too verbose - but optional
static types, pure OO, multiple dispatch.. that's an amazing set of features
that would make me very productive.

------
616c
There is a solid person who frequented here who was the last guy working on
OpenDylan and mentioned he largely abandoned that for a new passion for Rust.
Besides him, was that the last person really shepherding it?

~~~
BruceM
That is me. I haven't seen much development effort since I switched to Rust.
But ... I would still mentor someone who wanted to work on it or do something
with it.

~~~
616c
I was never at that level but still was very interested by your selfless work
on OpenDylan. I was going through a Lisp adoration phase (not that it ended,
really; one of many cycles I have) and found Dylan as "Apples prescient Lisp
without parens from the future" elevator pitch and was surprised its tooling
was better on Windows than Mac, basically because of you driving a very smally
community of volunteers.

Perhaps I need to take another look.

------
sillysaurus3
I've been working on a Lisp for the last couple of years. There's one major
downside to reader macros / syntax extension: Your editor probably won't
understand it, and in extreme cases won't know how to indent or read it.

Note that Lisp macros do not suffer this problem because it still uses the
standard `'(,"foo") syntax. (In fact it's somewhat remarkable that Lisp syntax
can be described in shorthand so compactly.)

Reader macros are a different beast. Let's say you want to add support for []
as indexing. You can do this:

    
    
      > (let l '(a b c)
         l[1])
      "b"
    

And this:

    
    
      > (let l '(a (b) c)
          l[1][0])
      "b"
    

And since the reader is under our control, it's easy to use a dynamic
expression as the index:

    
    
      > (let (l '(a (b) c)
              i 0)
          l[(+ i 1)][i])
      "b"
    

If you try this, you'll find most Lisp editors work okay-ish with it.

Now let's say you want to be able to do this:

    
    
      > (if x) {
          (print "hello")
        }
    

This is entirely possible since we control the reader. The rule is, if the
following expression is a curly-body, it gets converted into `(do ,@body) and
inserted as the last argument of the previous expression. So it becomes:

    
    
      > (if x (do (print "hello")))
    

However, this fails _hard_ inside of standard Lisp editors. You'll lose
indentation right away. Nothing understands curly notation. For example if you
turn on Clojure mode and try to auto-indent, here's what you get:

    
    
      > (if x) {
                (print "hello")
                }
    

The alternative curly style is just as broken:

    
    
      > (if x) 
        {
         (print "hello")
         }
    

So if you expose this power to users, suddenly you have an editor problem: How
can the editor know what to do with tokens, short of running the program?

Even in Emacs, which is Lisp-based, it's not really possible to execute other
Lisp variants. (I actually got both Scheme and Arc to work inside of emacs,
which was amusing. No external programs; literally scheme, but running in
elisp.)

But even if it were possible, you wouldn't want to. An infinite loop would
freeze your editor. Not to mention it should always be safe to open a file for
editing, yet this breaks that safety guarantee.

I think the solution is something like:

    
    
      > (define-reader ("{" s curly-body)
          ...)
    

Within the define-reader block, `s` is the stream, and you can read tokens
from it and return a result, or stuff the result into the previous token's
last argument, or throw an error, etc. And since it ends with "curly-body",
Lisp editors can be designed to detect this token and turn on "curly-body
syntax" for any file that imports it.

It's worth it. Would you believe this is Lisp?

    
    
      > require("fs").existsSync("/tmp")
      true
    

It's equivalent to:

    
    
      > ((get (require "fs") "existsSync") "/tmp")
    

But ten billion times more readable. And Lisp!

I have no idea why Lisp doesn't let you call functions like

    
    
      foo(1 2 3)
    

instead of

    
    
      (foo 1 2 3)
    

It's an unambiguous transformation in every case. Nobody smashes an atom into
a left-paren, ever. If there's an atom followed by a left-paren, or a right-
paren left-paren combo, it means you want to call it as a function.

It leads to some amusing situations:

    
    
      (list 1 2 3)
      list(1 2 3)
    

But there's an easy solution: If it looks confusing, don't write it that way.
Just like every other language feature.

Even though Lisp is such an old language, I feel like we've barely scratched
the surface. Clojure was a promising demo of what's possible. It's time for
the next step.

~~~
aaron-lebo
The thing is, lispers don't want more syntax. They've rejected Dylan, sweet-
expressions, and a million other experiments. There's a sizable contingent who
complain about Clojure using [] instead of all ().

So really all you are talking about is making a nice syntax that enables
macros, and that doesn't require lisp. See Elixir.

If lispers aren't going to adopt extra syntax, your only hope is to attract
non-lispers, and then you've got the issue where your lispy language has to be
better than existing languages at doing something, and that's hard.

Why go to so much trouble for something that's been rejected repeatedly?
Macros don't require lisp syntax and that's the main advantage.

My own ghetto transpiler converts this:

    
    
        sum = reduce partial(+)
        [1, 2, 3] sum + 1 println
    

into this:

    
    
        (def sum (partial reduce +)) 
        (println (+ 1 (sum [1 2 3])))
    

[https://github.com/aaron-lebo/prose](https://github.com/aaron-lebo/prose)

It's fun to think about, but not sure the general concept has an audience.

edit:

Bob Nystrom's lark is one of the cooler non-lispy lisps out there:

[https://github.com/munificent/lark](https://github.com/munificent/lark)

~~~
UIZealot
Sometimes I wonder if making the top level parentheses optional would go some
way towards making Lisp more palatable to other programmers. So

    
    
        (def sum (partial reduce +)) 
        (println (+ 1 (sum [1 2 3])))
    

becomes:

    
    
        def sum (partial reduce +)
        println (+ 1 (sum [1 2 3]))

~~~
mmalone
See
[https://srfi.schemers.org/srfi-110/srfi-110.html](https://srfi.schemers.org/srfi-110/srfi-110.html).
No idea why it hasn't shown up in a popular language (or maybe it has and I'm
just naive).

------
such_a_casual
It's pretty trivial to add macros to any language. The only thing that
separates a macro from a regular function is time. In a normal function, the
parameters are evaluated before being sent to the function. In a macro, the
function is called before the parameters are evaluated. What separates lisp
from other languages when it comes to macros is that the language is very
simple and provides lots of tools for operating on the same data structure
that you write your code in.

Of course you can add macros to any language, that doesn't mean they'll be
anything like Lisp macros, because you're still writing code that has many
different ways of expressing function calls. Importing a file? Oh, don't call
a function. Instead write this special syntax which will call that function
for you. Declaring or setting a variable? Same shit. What if you wanted to do
something really fancy like create a class, oh yeah, there's a special way of
saying that too. So now when you go to automate this stuff, you've gotta teach
your computer all the special different ways to write everything. Lisp isn't
lisp because it has macros. Lisp isn't lisp because it's simple. Lisp isn't
lisp because it has a lot of really smart ways of doing basic things that we
take for granted (like explicit vs implicit scope on local variables). Lisp is
lisp because it has them all.

You can add the same extensibility that lisp has, macros, reader macros,
operate on symbols, closures, whatever. You can do some of the really smart
things that lisp does like explicitly scoping your variables or multiple
dispatch. You can even have a really simple language, with a really simple
syntax. But it's not Lisp's individual features that make it what it is, it's
how these features interact with one another to give you something that's
truly hard to replicate.

Yes, of course you can replace a block of code with another, but how easily?
For example, yesterday I was thinking about a variable that represents the
result of evaluating the previous form. So instead of having to read a method
chain backwards: (wax (buff (rinse (soap car)))) it could read forward: (soap
car) (rinse ?) (buff ?) (wax ?) Now whether or not this is a good or bad idea
is completely fucking irrelevent. The point is that it's an idea, and until an
idea is tested, we have no idea how bad or good it is. In lisp, in 5 minutes,
I can write a macro that wraps around a block of code and implements this idea
so I can actually play around with it in the real world instead of just in my
head. You literally just take a bunch of forms in, put a let statement around
the whole block, and then put a call to set the variable around each
individual form (If you're not familiar with lisp. a form is kind of like a
line of code, it's usually a function call but it can be a value as well). In
lisp I can have very abstract programming ideas and then immediately implement
them, without having to fight through the language to do so (ie without having
to write a parser).

It's cool when other languages take stuff from lisp, it makes it easier to
write lisp like code when I have to use other languages, but I've read a lot
of articles on "Lisp macros in X Language" and none of them seem to understand
that you can't just implement macros, you have to make the structure that code
is written "first class". You have to give the programmer the tools they need
to operate on a piece of code as naturally as they would a string, a number,
or an array.

~~~
lispm
> It's pretty trivial to add macros to any language.

Not true.

> The only thing that separates a macro from a regular function is time. In a
> normal function, the parameters are evaluated before being sent to the
> function. In a macro, the function is called before the parameters are
> evaluated.

That's not really capturing it. A regular function expects to get passed
evaluated results from valid code. The arguments are being evaluated. The
function gets called and the parameters will be bound to these arguments.

A Lisp macro OTOH gets arbitrary stuff passed. A Lisp macro can enclose
anything. This anything can look totally wild and does not need to be valid
code. The macro gets this anything, it has to compute valid source code and
can create any side effect it wants.

In a macro form, the macro function gets called with the code it encloses and
an environment. It can then do anything it wants, but it has to return valid
code. This code then will be evaluated - the generated code can be anything
which is in that context a valid Lisp form: self-evaluating data, a variable,
a special form using built-in Lisp syntax, a function form or another macro
form.

> (If you're not familiar with lisp. a form is kind of like a line of code,
> it's usually a function call but it can be a value as well)

A 'form' is an expression which is also a valid Lisp expression (can be
evaluated, etc.).

> But it's not Lisp's individual features that make it what it is, it's how
> these features interact with one another to give you something that's truly
> hard to replicate.

Yep, that's a good insight.

------
bitwize
The dream of the Lisp Machines is long dead.

JavaScript Machines are our new reality.

~~~
mmalone
Worse is better, amiright? sigh...

