
Not everything is an expression - thomasballinger
https://codewords.recurse.com/issues/two/not-everything-is-an-expression
======
weavejester
I've read through the article twice, and I still have no idea what the author
is getting at.

The author suggests that "the obvious way to implement a DSL as a macro, as we
saw with if-match, hard-codes the form of the new syntax class". I disagree.
That's not what I'd consider the obvious way at all.

I'd consider the most obvious approach would be to pass the macro onto a
polymorphic function of some description:

    
    
        (defmulti if-match*
          (fn [pat _ _ _] (if (list? pat) (first pat) (type pat)))
    
        (defmacro if-match [pat expr then else]
          (if-match* pat expr then else))
    

Macros have all the same capabilities for extensibility as regular functions.
In Clojure at least, macros _are_ just functions with some metadata attached.

~~~
rntz
That's a very clever use of defmulti that I hadn't considered --- consider
that you may know more about writing extensible macros than the average lisper
:P. My article was also aimed at being language-agnostic, so a Clojure-
specific feature like defmulti wouldn't have been appropriate to introduce.
(Although of course CLOS does have multimethods as well, but that's an even
more complicated subject!)

However:

1\. The code you give still isn't smart enough. It dispatches on the _symbol_
at the head of the list, but that doesn't account for namespacing. So your
pattern-macros will all end up in one giant namespace. You could probably
invent something clever to account for this but...

2\. My overall point[1] was that writing a macro-extensible macro shouldn't
_require_ cleverness or new code - it should be in the standard library!
Indeed, ideally defining a "pattern-macro" should be accomplished via the same
mechanism as defining an "expression-macro"; you shouldn't need separate,
custom macro-defining-macros for each syntax class. I'd settle for it just
being easy to define an extensible syntax class along with a macro-defining-
macro for it, though.

[1] Admittedly, this point could have been far clearer.

~~~
weavejester
You're making a distinction between macros and functions, but the only
difference is that functions evaluate their arguments, while macros evaluate
their return value.

The idea that there should be "pattern-macros" and "expression-macros" is
confusing _how_ macros are used with _what_ macros are.

The namespacing issue can be solved in the usual way; by evaluating the symbol
in some fashion. How that's done really depends on what you want the syntax to
look like. The simplest mechanism is just to pass the macro on to another
macro:

    
    
        (defmacro if-match [pat expr then else]
          (if (list? pat)
            (list pat expr then else)
            (if-match-on-type pat expr then else)))

~~~
rntz
I'm sorry, I don't really understand what you're getting at when you say that
I'm confusing "how macros are used" and "what macros are". Perhaps we're just
disagreeing about terminology?

What I mean by a "macro" is: a convenient way to extend the syntax of a
particular syntax class by telling my language how to translate the syntactic
construct I want to add into constructs it already understands. In this sense
one definitely can have expression-macros and pattern-macros. Is there some
reason you think one shouldn't have this in a language?

Your new proposed `if-match' is strange. I'm not sure how I'd use it to, for
example, implement a "list" pattern-macro, such that `(list p1 p2 p3)' matches
any list of three elements, whose first element matches p1, second matches p2,
and third matches p3. You seem to intend that `((list p1 p2 p3) expr then
else)` should macro-expand to the code that performs this pattern match. This
is difficult for two reasons:

1\. I'd need to shadow the definition of `list' with a macro that usually
behaves like the list function, but if used inside of if-match behaves
differently. This seems difficult, and is conceptually ugly.

2\. AIUI, to get `((list p1 p2 p3) expr then else)' to macro-expand specially,
`(list p1 p2 p3)' has to macro-expand to a symbol naming a macro, call it
MAGIC, such that `(MAGIC expr then else)` expands to the desired result. This
is difficult, because MAGIC needs to know about p1, p2, and p3. So `(list p1
p2 p3)' will need to dynamically define a new macro (with a gensym'ed name)
that knows specifically about p1, p2, and p3, and return that. This may be
possible, but is gnarly as fuck.

This is certainly very far from being "the obvious way to do it" to anyone but
a wizard.

Maybe I'm misunderstanding you, or you made a typo?

~~~
weavejester
My definition of macro would be: a function that's evaluated at compile time.

In this sense, the idea of an "expression-macro" and a "pattern-macro" makes
as much sense as an "expression-function" or "pattern-function". We don't
typically have different classes of function to fulfil different purposes.

The `if-match` macro you describe could be written as a function, albeit with
less pleasing syntax:

    
    
        (defn if-match [pat expr then else]
          (if-let [match (if (fn? pat) (pat expr) (isa? pat expr)]
            (then match)
            (else match)))
    
        (if-match 0 0 (fn [_] true) (fn [_] false))
    
        (if-match (pattern/cons 0 'x)
                  '(0 hi there)
                  (fn [{:syms [x]}] x)
                  (fn [_] nil))
    

So in the above case, we allow literal values to be passed in, or a function
which returns a map of symbols to matching values. It's the function that
allows for the extensibility.

A macro can easily smooth over the janky syntax of the `then` and `else`
functions, and we'd end up something like this:

    
    
        (if-match (pattern/cons 0 x)
                  '(0 hi there)
                  x
                  nil)
    

This is essentially what my previous example was alluding to, though in that I
intended to use a namespaced macro rather than a namespaced function for
extensibility.

You might well complain that `(pattern/cons 0 x)` is more verbose than `(cons
0 x)`, but it's also unambiguous, and doesn't require what would amount to a
separate namespacing system, which a Lisp-1 like Clojure tends to reject.

Now, I don't think the above examples are magical in any way. The macro is a
small improvement over the function, and the function isn't particularly
complex. If higher-order functions are your idea of wizardry, then there's not
much in Lisp that you _wouldn 't_ consider magic.

Maybe your objection just stems from the use of something like `p/cons`
instead of just `cons`, while still desiring `cons` to be namespaced in some
way. In which case, I'd suggest that what you're really asking is not 'can I
have expression-macros and pattern-macros', but 'can I create new namespacing
systems'.

------
kerkeslager
This is an interesting approach.

I'm working on a Lisp variant that recognizes the difference between
expressions and... non-expressions? But taking the opposite approach: I found
a way to make it so that everything _is_ an expression while allowing one to
do everything you would do with a non-expression via expressions. To achieve
this, there are two kinds of expression: mutations and functions.

There are two things to understand about this:

1\. All expressions take in the environment. Most functions don't use it,
while most mutations do.

2\. All expressions run inside a trampoline that evaluates them. The
difference between a mutation and a function is that when the trampoline
evaluates a function, it places its result into the return register (where it
can be picked up by something else). In contrast, when the trampoline
evaluates a mutation, it _replaces the environment with the result_. This is
why mutations typically use the environment--rather than destroying the
environment, you usually want to build the new environment with most of the
old environment.

Some examples:

    
    
        ((mut () env (assoc env :foo 1))) ; equivalent to (define foo 1)
    
        ((mut () env
          (assoc env :my-define (mut (dest src) env (assoc env dest src)))))
        ; this is actually how `define` is defined
    
        
        ((mut () env (map)))
        ((+ 1 1)) ; throws exception "undefined symbol +" because previous line emptied the environment
    

The "everything takes env" bit is inspired by J. Shutt's paper on his Kernel
programming language:
[https://www.wpi.edu/Pubs/ETD/Available/etd-090110-124904/unr...](https://www.wpi.edu/Pubs/ETD/Available/etd-090110-124904/unrestricted/jshutt.pdf)
and a lot of what I'm working on is built on his work.

~~~
cbd1984
> non-expressions

I think "statements" is the usual term, with "sentences" being the general
term which encompasses both (a "sentence" in this context is something
recognized by your formal grammar).

Can your Lisp variant handle unrestricted continuations?

~~~
kerkeslager
I must admit I have no idea what an unrestricted continuation is. I'm not an
academic, just a guy writing a lisp on his spare time.

Here's what I'm doing with continuations:

At the bytecode level, call/cc is trivial to implement given my current
design. All you need is the current instruction pointer, the current
environment (environments right now are just immutable A-lists), and the
current return pointer. But it's not obvious to me how to translate call/cc
from the syntax to the bytecode.

Tail call elimination is the next thing I'm implementing, and unless there's
something wrong with my design I haven't noticed yet, it won't be hard to do.
Another step to take down that direction, though, is adding a pass
transforming to continuation-passing style. I think this is going to be one of
the last things I do, because I'd like to explore other optimizations so I can
compare performance.

------
ggchappell
Interesting article. A few thoughts:

The fact that Lisp does not distinguish between statements and declarations is
closely tied to the fact that Lisp is very much a dynamic language (in
particular, it is dynamically typed). The article uses the example of Python
declaration vs. statement; but actually Python declarations are statements,
too. This is typical of dynamic languages.

On the other hand, in a statically typed language there is necessarily a
distinction between code that is executed at runtime and (although we often
don't talk about it this way) code that is executed at compile time.
Declarations happen at compile time. Expressions and statements happen at
runtime. The two categories almost always use very different syntax.

Among statically typed languages, Haskell is particularly interesting,
because, while it necessarily makes a strong distinction between expressions
and declarations, it has erased the distinction between expression and
statement: the latter is represented by an expression that returns a list of
side effects.

Another interesting take on this issue can be found in Daan Leijen's Koka
programming language[1]. In Koka, whether a function has side effects is part
of its type. So _effect inference_ can be done. The result, if I understand
things correctly, is that the expression-or-statement issue becomes more than
just a yes/no thing. I think these ideas are worth further exploration.

Lastly: an extensible pattern set. My goodness, yes. That's the big lack I
feel in Haskell; I want to define new kinds of patterns. I've read that F# has
good support for this, but I know nothing about it; can anyone comment?

[1] [http://research.microsoft.com/en-
us/projects/koka/](http://research.microsoft.com/en-us/projects/koka/)

~~~
Dewie
> On the other hand, in a statically typed language there is necessarily a
> distinction between code that is executed at runtime and (although we often
> don't talk about it this way) code that is executed at compile time.

Is the distinction or separation necessarily obvious? Statically typed
languages which have a unified term- and type-level seem to get a bit subtle
in this regard, since a type can be used as a value. And Idris passes proofs
around, and has to use proof erasure in order to not let it infect the
runtime: proofs can remain until run-time if you're not careful.

I am asking this as a question since I don't have enough experience with such
languages to really know myself.

~~~
ggchappell
> Is the distinction or separation necessarily obvious?

I don't think so, but in practice it usually is obvious, in the programming
languages that actually get _used_.

OTOH, the notion that the compile-time language and the run-time language
could be very similar is an idea that might be slowly catching on. We'll see
....

------
ThatGeoGuy
I don't mean to be a pedant, but the author mentions a "syntax for patterns",
basically claiming that Lisp doesn't have one. But, isn't syntax-rules (a la
scheme) already a form for matching / macro-ing patterns? From my
understanding the author seems to want macros that can be specialized for new
forms.

I may be confused about what the exact claim is here, but I don't see how this
has anything to do with whether or not something is an expression. I don't
quite understand how having "not everything is an expression" helps solve this
problem.

~~~
rntz
Author here!

syntax-rules _is_ a form of pattern matching (mentioned in footnote 2). But
it's only for matching on syntax, not on ordinary values. I can't write the
fibonacci function using syntax-rules.

The idea behind "not everything is an expression" is that ordinary macros only
extend the _expressions_ in a language. But languages have more than
expressions to them - they also have patterns, and possibly other syntax
classes (loop formats, LINQ, monadic do-syntax). I think _those_ syntax
classes ought to be macro-extensible as well. That's what I mean when I say
that ordinary macros don't acknowledge that not everything is an expression.

It's rather a roundabout way to say it, I guess.

~~~
malisper
Well iterate[0], which is a lispy version of loop, is actually extendable
through macros[1]. So what you are looking for is just a generic way to enable
that for all macros? Iterate does it by having a code walker go over the code
and macroexpand the extendable parts. It shouldn't be too hard to apply that
method to new DSLs by specifying the syntax of the DSL to the code walker.

[0] [https://common-lisp.net/project/iterate/](https://common-
lisp.net/project/iterate/)

[1] [https://common-lisp.net/project/iterate/doc/Rolling-Your-
Own...](https://common-lisp.net/project/iterate/doc/Rolling-Your-
Own.html#Rolling-Your-Own)

~~~
rntz
Yup, I pretty much want it to be trivially easy for define macros that are
themselves macro-extensible. It's definitely _possible_ to do this in Lisp,
but:

(a) it's not a well-known technique

(b) it's not in the standard library of any Lisp I know of

(c) there are interesting open design questions to be answered in the
implementation of such a system

I hope this article will get folks thinking about these issues.

For example, walking the AST and calling macroexpand will work, but then you
can't have a macro that expands differently in different contexts - when
interpreted as an expression versus as a pattern, for example. I think this is
an important feature.

~~~
malisper
> you can't have a macro that expands differently in different contexts

This[0] may be of some help. You could set it to expand macros differently
according to the current context (some global variable).

You could also write your own macroexpand function which would default to the
implementation's macroexpand.

This is definitely something interesting to think about. Thank you for the
discussion.

[0]
[http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec...](http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/Body/var_stmacroexpand-
hookst.html#STmacroexpand-hookST)

~~~
DalekBaldwin
The macroexpansion hook isn't nearly as powerful or useful as it could be
because the process of macroexpansion is not recursive. Although expansion
functions can access information about the lexical environment via an
&environment parameter, the invocations of those expansion functions do not
share any dynamic context with other invocations at nodes above or below them
in the syntax tree.

Here's a more thorough summary of the problem:
[http://qiita.com/guicho271828/items/07ba4ff11bff494dc03f](http://qiita.com/guicho271828/items/07ba4ff11bff494dc03f)

~~~
malisper
I managed to get the code from the post you mentioned to work[0] by using
macroexpand-dammit[1].

Coincidentally, you have contributed to that, so I thought I would ask you
what is the problem with using macroexpand-dammit?

[0]
[https://gist.github.com/malisper/055848f4e593af5ca1b1](https://gist.github.com/malisper/055848f4e593af5ca1b1)

[1] [https://github.com/guicho271828/macroexpand-
dammit](https://github.com/guicho271828/macroexpand-dammit)

~~~
DalekBaldwin
The specific problem is that it's buggy. But the general problem is that it's
a massive, fragile hematoma of duplication [ctrl-f "hematoma":
[http://people.engr.ncsu.edu/efg/517/f00/syllabus/annotations...](http://people.engr.ncsu.edu/efg/517/f00/syllabus/annotations/lec13.html)],
which makes the bugs buggier.

A macroexpansion facility works more like a framework than a library. Each
macro you define is basically a callback function triggered when the compiler
sees a form with a particular symbol at the head. You want to be able to
describe what should happen in particular situations you're interested in,
without having to specify anything about other situations. A code walker
spoils that separation of responsibilities because it has to reimplement large
parts of a macroexpansion framework just to make it possible to add a few new
features which are then used in conjunction with the original framework.

The potential for bug contagion is quite high, because any single bug's
effects aren't localized to the few forms that make use of the extended
functionality you've rigged up. This is tantamount to introducing a bug in the
language implementation. If you're expanding something like (my-macro (...
(my-other-macro))), a bug in your code walker can break the intermediate code.
This might be acceptable if you're just using it to write a couple of one-off
macros that don't trigger any weird corner cases and are only used in the same
project in which they're defined. But if you're creating a general-purpose DSL
like iterate that you intend for people to use in all sorts of situations you
never anticipated, it's not. In fact, the same bug is more dangerous as part
of a code walker than it would be as part of a Lisp implementation's
internals. If it's in the implementation, at least it would show up
consistently whether the vulnerable code is wrapped in your macro or not,
making easier to identify, document, and manage the problem. And probably
easier to get somebody to fix, because the implementation ought to be better
maintained than the walker.

These are not merely hypothetical concerns. Some projects already rely on
macroexpand-dammit working in the quirky way it does; Masataro tried to have
his improved version supersede the existing version on Quicklisp, but it broke
some packages that were using it:
[https://github.com/guicho271828/macroexpand-
dammit/pull/4#is...](https://github.com/guicho271828/macroexpand-
dammit/pull/4#issuecomment-66752189)

Sometimes I use those packages too, so instead of using his version with the
fix I submitted, it's actually less of a headache for me to apply my fix using
a wrapper package on top of the Quicklisp version, duplicating some of the
code which itself duplicates what Lisp already does. The original duplication
problem led to bugs which led to the need to load two separate implementations
which led to a name clashing problem which justified additional duplication.

Also, I tried to implement that same iter example using a similar trick
before, and I see that your code generates the same mysterious output, at
least on SBCL:

    
    
      ; compilation unit finished
      ;   caught 1 fatal ERROR condition
    

A fatal error during compilation, yet it clearly compiled and ran. What was
the error? I don't know, it looks like something in the code walker muffled
the details. Does it matter? Could it matter in any other scenario? Something
about macroexpand-dammit doesn't play nice with the implementation, but we
don't know what, why, or how, and it makes it more difficult to use the very
language features that would normally help us find out.

I think a lot of critiques of the "bipolar Lisp programmer"
[[http://www.lambdassociates.org/blog/bipolar.htm](http://www.lambdassociates.org/blog/bipolar.htm)]
are overblown, but in the case of these kinds of tools, they're justified.
These kinds of techniques will always be restricted to obscure one-off
software projects unless they're supported by core language features like what
rntz and Masataro are proposing. The designs of macro systems are optimized
for making it easy to make small tweaks to the language to fit the particular
problem at hand. They're not quite as flexible as something like the MOP. If
you try to use a facility designed for making small, local interventions in
the language to write very general language extensions which themselves make
broad, global interventions in the underlying system for making small, local
interventions, you're gonna have a bad time.

------
ICWiener
Macros will expand into lisp forms, not only expressions. Whether a form is an
expression, a declaration or a pattern depends on the surrounding context.

I would say that declarations, ... are not syntax but semantic classes.

Too bad the conclusion does not offer a glimpse of what would the extension
mechanism look like. Still, nice article.

~~~
dgreensp
Yes, critically speaking, the author seemed on track to implementing an
extensible pattern matching system based on macros, before stopping to end
with the point that it wasn't the _obvious_ , idiomatic, built-in thing to do.

And in this system, patterns are expressions. So "everything is an expression"
isn't exactly wrong in this case.

In Scala, you are given full control over both how a name "applies" (i.e. what
Foo(...) does when called as a sort of constructor or factory method) and how
it matches, in the form of an "unapply" static method that you implement. It
seems like a "match" macro in a LISP could desugar a pattern into a program
that matches that pattern, the same way the Scala compiler generates code that
calls "unapply".

~~~
rntz
Author here. I have created a language, called moxy
([https://www.github.com/rntz/moxy](https://www.github.com/rntz/moxy)), which
supports not just extensible pattern-matching but defining extensible
extensions in general (of which pattern matching is one example, LINQ might be
another, and so on and so forth). However, it wasn't a lisp, because I also
wanted to see whether I could support syntactic extensibility in a non-lisp
language.

The really important point, that moxy explored and that I think I didn't make
clear in the article, is that patterns are _just one example_ of where you
want extensibility of a non-expression syntax class. Really you want to be
able to _define your own_ syntax classes and be able to extend them!

I only mentioned moxy in a footnote in the article because it's "research-
quality" at the moment - it's not well documented and I'm still having second
thoughts about its design. I've moved on to other thing and am busy with work,
so I don't imagine I'll be working on it in the immediate future.

------
endlessvoid94
If you haven't had the chance to read "The Art of the Metaobject Protocol"
[0], I highly recommend it. It deserves to be mentioned anytime something like
OMeta is mentioned.

[0] [http://www.amazon.com/Art-Metaobject-Protocol-Gregor-
Kiczale...](http://www.amazon.com/Art-Metaobject-Protocol-Gregor-
Kiczales/dp/0262610744/ref=sr_1_1?ie=UTF8&qid=1427490532&sr=8-1&keywords=the+art+of+the+metaobject+protocol)

------
taeric
While it is generally held that everything in lisp is an expression. Isn't the
more pertinant fact that everything is a list? That is, I thought macros
hinged on the fact that everything is a list, not that everything is an
expression.

~~~
arohner
Macros hinge on the idea that code is data. Macros are functions that run at
"compile-time" [1], that take unevaluated code (i.e. data) and return any
valid data, it just happens that returning a list is interpreted as a function
call. But I can also define:

(defmacro foo [x] :foo)

which always returns a keyword.

All lisps that I'm aware of only let macros dispatch on list evaluation, i.e.
when you see (foo ...), call the function defined in (defmacro foo), but I'm
not aware of any limitation preventing you from applying that to other types
of data.

[1] technically, they run at macro-expansion time, which is after reading the
expression, and before evaluating.

~~~
taeric
Right, my point is that is less dependent on "everything is an expression" and
more that "everything is a list." Right?

~~~
arohner
And my point is that everything isn't a list, but it is data. Macros are
functions that take data, and return data. Their most common usage is that
that they take lists and return lists, but that isn't required.

~~~
taeric
Makes sense. I was actually coming at it from the "everything is _in_ a list"
vantage. And can be destructured as such. (Restructured, as well, of course.)

------
spenczar5
Recurse is really publishing some fantastic stuff. This entire second issue
has been just great.

------
yawaramin
Sure, not everything is an expression; but an expressive language provides an
expression that can _contain_ other syntax classes and confine all their
effects within the bounds of the expression. E.g., SML's 'let' expression.

In a language that doesn't allow that, we end up having to do some really
weird things
([https://github.com/yawaramin/lambdak](https://github.com/yawaramin/lambdak)).

------
IshKebab
Totally off-topic, but I was curious if the recursive Rust `sum` function is
actually optimised correctly.

Code:

    
    
        fn sum(l: &[i64]) -> i64 {
            match l {
                [] => 0,
                [x, xs..] => x + sum(xs)
            }
        }
    

Assembly:

    
    
      _ZN3sum20hf66fc5855a7cf5fc3aaE:
    	.cfi_startproc
    	cmpq	%fs:112, %rsp
    	ja	.LBB2_2
    	movabsq	$24, %r10
    	movabsq	$0, %r11
    	callq	__morestack
    	retq
      .LBB2_2:
    	pushq	%rbx
      .Ltmp10:
    	.cfi_def_cfa_offset 16
    	subq	$16, %rsp
      .Ltmp11:
    	.cfi_def_cfa_offset 32
      .Ltmp12:
    	.cfi_offset %rbx, -16
    	movq	8(%rdi), %rcx
    	xorl	%eax, %eax
    	testq	%rcx, %rcx
    	je	.LBB2_4
    	movq	(%rdi), %rax
    	decq	%rcx
    	movq	(%rax), %rbx
    	addq	$8, %rax
    	movq	%rax, (%rsp)
    	movq	%rcx, 8(%rsp)
    	leaq	(%rsp), %rdi
    	callq	_ZN3sum20hf66fc5855a7cf5fc3aaE
    	addq	%rbx, %rax
      .LBB2_4:
    	addq	$16, %rsp
    	popq	%rbx
    	retq
    

So... no.

~~~
dbpatterson
You might get better optimizations if you make it tail recursive (LLVM
probably gets some of these). ie:

    
    
        fn sum_tail(v : i64, l: &[i64]) -> i64 {
            match l {
                [] => v,
                [x, xs..] => sum(v + x, xs)
            }
        }

------
escherize
I'm pretty confused by the author's definition of statements. FTA: "Statements
are executed to take some action. Variable assignments, loops, conditionals,
and raising exceptions are examples of statements."

With respect to variable assignments or raising exceptions (though examples
exist of the others...) aren't these by the author's definition statements?

    
    
        (def a "apple")

or (throw (Exception. "my exception message"))

------
robgibbons
It seems to me that it's possible to define most statements in an expressive
syntax, at least in any language which allows for both constructs.

For instance, in JavaScript one can use ternary syntax in place of an if-
statement. Is a ternary condition actually an expression? It seems more of an
expression than a statement, but one could argue it's just a simplified syntax
of a conditional statement.

------
siscia
I don't really get what the author is claiming...

I would have code his `if-mathch` in a simpler way in clojure:

    
    
        (case (f data-structure)
          0 (do-something)
          1 (do-something-else)
          (do-default))
    

Where `f` can be a function defined in a protocol or a multimethod, so you can
actually implement your own `f` for any data structure you like.

Now, what I am missing ?

------
lispm
If you look at the literature there are numerous examples of extensible
macros. Often this is done for rule-based systems, which also involves
matching or unification. Typically one wants to define these rules
individually, update them individually, etc.

One needs a registry, an interning function and a driving function. Below is
just an example:

    
    
        (defvar *patterns* (make-hash-table))
        (defparameter *pattern-names* nil)
    
        (defun intern-pattern (name if-pattern then-pattern)
          (setf *pattern-names*
                (append *pattern-names* (list name)))
          (setf (gethash name *patterns*)
                (list (compile nil `(lambda (pat)
                                      ,if-pattern))
                      (compile nil `(lambda (pat expr then else)
                                      (declare (ignorable pat expr then else))
                                      ,then-pattern))))
          name)
    
    
        (defmacro if-match (pat expr then else)
          (loop for name in *pattern-names*
                for (if-part then-part) = (gethash name *patterns*)
                when (funcall if-part pat)
                do (return (funcall then-part pat expr then else))))
    
        (intern-pattern 'variable
                        '(and pat (symbolp pat))
                        '`(let ((,pat ,expr)) ,then))
        (intern-pattern 'literal-atom
                        '(atom pat)
                        '`(if (equalp ',pat ,expr) ,then ,else))
        (intern-pattern 'cons
                        '(eq 'cons (car pat))
                        '(destructuring-bind (_ p-car p-cdr) pat
                           (declare (ignore _))
                           (let ((tmp (gensym)))
                             `(let ((,tmp ,expr))
                                (if (consp ,tmp)
                                    (if-match ,p-car
                                              (car ,tmp)
                                              (if-match ,p-cdr
                                                        (cdr ,tmp)
                                                        ,then
                                                        ,else)
                                              ,else)
                                    ,else)))))
    

Writing the macro DEFPATTERN is then trivial...

I help maintain an old Lisp-based web server, which was written in the mid-90s
on the Symbolics Lisp Machine. It literally has zillions of these
registry/intern/machinery/defining-macro combinations...

It's just: one has to program those. But it has been done many many many
times.

------
gumby
Not sure why it's a surprise that languages inherently require metasyntactic
operations. This has been clear since Church and Gödel.

There's lot of good work on programming languages that allow metasyntactic
runtime extensions, going back to Brian Smith's work at PARC.

------
zem
minor nitpick - even if "OCaml has no equivalent of Haskell's Ordering or
SML's order types" there is Pervasives.compare. it returns {0,-1,+1} rather
than {EQ, LT, GT}, but you can still say

    
    
        match compare x v with
        | 0 -> ...
        | 1 -> ...
        | _ -> ...
    

with the minor wart that since compare returns an int, you need to match the
last case with a default to prevent the compiler from warning you about
possibly not matching other int values.

