
Fear of Macros - zitterbewegung
http://www.greghendershott.com/fear-of-macros/index.html
======
tomcam
For those who don't know, Hendershott wrote Cakewalk, the most popular music-
creation software on the PC. He knows his stuff. It's also an engaging
article, esp for us old C/C++ hacks who get nervous around Lispy things.

~~~
brianmwaters_hn
I saw Greg give a nice talk at Racket Con this year. He was really humble and
talked about the challenges of writing a Markdown parser. Didn't know he wrote
Cakewalk! Cool!

~~~
fit2rule
Don't forget that Cakewalk includes CAL - the Cakewalk Application Language,
which is a bit Lisp'y, I have to say, as well as being a bit C'ish, to boot.
;)

~~~
greghendershott
Confession: I used s-expressions only because even I could figure out how to
parse that. I didn't appreciate Lisp in any meaningful sense.

Fast forward to a few years ago. I wanted to learn a lisp, finally. I
gravitated toward Scheme. PLT Scheme seemed like the best choice. Today it's
known as Racket.

~~~
fit2rule
I remember putting Cakewalk through its paces in the late 80's/early 90's on
my old DOS machine in the studio .. it was really a great way to get a lot of
music written, and I really appreciate your aesthetics for how you managed
that application over time - the arrival of CAL was a really interesting move,
and as things moved so fast in the pro audio world in the 90's, it always sort
of bothered me that more and more tools were not written with scripting - for
the user - in mind.

My friends and I wrote all sorts of CAL scripts to 'funkify' our Drum/Rythmn
tracks - I wish I still had a copy of funkify.cal around, it'd be nice to see
it again.

I moved on from Cakewalk around the DOS/Windows migration, as I've never been
a heavy fan of Windows, nor a real user, so I got set up with all kinds of
other tools after we left the DOS machines alone .. mostly hardware
sequencers.

What do you think about one day doing a hardware sequencer, Greg? With the way
things are going, it seems like you could return to the world of music-
composition, albeit with a hardware bent, and make some serious progress in
that department .. as we know the music industry/market is heavily cyclic,
soon enough there will be a return to hardware as a reliable, dependent way of
composing music. Any plans in that regard? Imagine: a hardware MIDI/Audio
sequencer .. with onboard SCRIPTING! :)

------
auvrw
i don't fear macros, but i am wary of them: when there are only a few macros,
it seems as though replacing them with functions makes more sense, even if the
syntax is slightly messier. when there are so many macros that you've
essentially made an entirely new language, writing a REPL feels like a cleaner
separation, even if it is more work.

i would be interested to see links to libraries, particularly clojure
libraries, that make heavy but judicious use of macros. it totally makes sense
that macros are useful because they capture part of this "code as data" idea
that's so central to, esp., lisps. but having more examples of actual problems
that they solve would be helpful.

~~~
Psyonic
A good example, clojure logging.
[https://github.com/clojure/tools.logging](https://github.com/clojure/tools.logging)

They look the same as normal logging function calls, but entirely avoid the
issue of "logging guards."

I'll point you at a good link, but basically if you have a lot of calls to
logger.debug, logger.trace, etc, and they involve a lot of string conversions,
you pay the penalty for those string conversions all the time even when you're
logging at warn or higher. This leads many java developers to do the
following:

if (LOGGER.isDebugEnabled()) { LOGGER.debug("last node was: " \+ node); }

... which makes logging significantly heavier visually.

With a macro you are essentially adding syntax, so you can bake the guards
into your log statements.

Logging guards:
[http://www.kdgregory.com/index.php?page=java.logging](http://www.kdgregory.com/index.php?page=java.logging)

~~~
twic
Although, for what it's worth, Java programmers solved this quite some time
ago by allowing crude lazy evaluation of the string. Logging calls actually
look like:

    
    
      LOGGER.debug("last node was: {}", node);
    

Where the parameters are a template string and a varargs of objects. The
objects are stringified and substituted into the template only once the
logging library determines that the message will actually be used.

In Scala, you just make the parameter to the call a lazy or call-by-name
value, and then there's no need to write an explicit template, the language
captures the expression which produces the string and defers its execution
until it's needed.

Of course, there are many things macros can do which mechanisms like these
can't. The choice in language design is between introducing macros, or
introducing enough other mechanisms to make macros unnecessary. Introducing
macros seems more parsimonious, but given the utter confusion that macros can
be used to wreak, it's not a slam dunk.

And, of course, the Scala designers have introduced all these mechanisms and
then added macros anyway, because they're crazy.

------
sparkie
I developed a fear of macros after realizing that they're not first-class, and
you can't pass macros to functions as arguments for instance (instead you need
to wrap them up in function, which doesn't work well with variadic forms).

An even more useful concept exists in J.Shutt's Kernel programming language[1]
- that of first class operatives. Every entity becomes a first-class object,
which can be an operative (like some special forms) or an applicative (a
scheme function). operatives do not implicitly evaluate the operands, so you
can easily create macro-like behavior by defining your own evaluation model.
It's also trivial to convert between applicatives and operatives with wrap and
unwrap (which are not special forms, they're defined in terms of vau and
eval).

[1]:[http://web.cs.wpi.edu/~jshutt/kernel.html](http://web.cs.wpi.edu/~jshutt/kernel.html)

~~~
samth
It turns out that fexprs, which is what Kernel provides, are widely considered
a bad idea in the Lisp community, and were abandoned in favor of macros for a
reason. The basic problem is that you can never tell whether your function
call is really a function call or a macro application, until you run the
program, and therefore you can't reason about your code at all. Incidentally,
this also means that your compiler can't reason about your code, and thus
optimization is basically impossible.

~~~
sparkie
The fexprs provided by Kernel are not the same as the ones in traditional
lisps which were abandoned - Shutt has developed a theory for reasoning about
them in some ways (the vau-calculus, which is capable of defining the lambda
calculus too). He describes pure and impure versions of the vau calculus in
his dissertation.

Firstly, operatives are improved over the traditional ones by having lexical
scoping - where the original lisps were dynamically scoped. The other main
feature of operatives is that they gain access to the dynamic scope from where
they were called - as this environment is passed implicitly to the operative
when it is evaluated.

You're right that they do not allow us to reason inductively about code in the
same way as the lambda calculus due to this, but it's a trade off for
incredible flexibility. I think we can build alternative models to reason
about them for purposes of optimisation and such. (Or alternatively, one can
have operatives behave as compilers - in which you feed in type-tagged
expressions, and your compiler-operatives can perform full type checking -
then output an applicative which strips this type information). I see the
possibilities as endless, because the model is extremely abstract.

Kernel choses not to enforce the static separation of operatives and
applicatives for simplcity - instead, a convention based approach is used
whereby operatives are prefixed with "$", (such as $vau, $lambda, $define!,
$provide!). One can check them during runtime with the operative? and
applicative? predicates too.

However, it would be perfectly possible to create a compiler which can know
this difference, since the forms `($define! ($vau ...))` and `($define!
($lambda ...))` which distinguish them could be given special forms or special
syntax in an alternative representation of this model, although you would not
be able to tell whether an applicative uses some operatives internally -
that's the idea of encapsulation - implementation information is meant to be
hidden. (Even at the cost of performance, which often doesn't matter.)

Even if Kernel is not as widely practical as lambda-calculus based languages -
Shutt's dissertation is definitely worth a read, and shouldn't be discarded
because of a perception of bad fexprs from 40 years ago.

------
cosmez
before this article i didn't want to touch scheme macros.

he does a great job by introducing macros using basic syntax transformers,
every other tutorial failed for me by introducing define-syntax-rule before
explaining how it worked.

------
Legion
EDIT: By browsing to the Github repo, I found the link to the One Big HTML
page: [http://www.greghendershott.com/fear-of-
macros/all.html](http://www.greghendershott.com/fear-of-macros/all.html)

Original post: I wish there was a single-page version of this that I could
Pocket. Or an epub that I could put on my reading devices.

I don't tend to read multi-page things like this at my desk, not in this era
of high-resolution lightweight portable screens.

------
Ologn
I have gone from Unix shell script to Perl to PHP to C to C++ to Java to
Python. Most of them are a lot of the same Algol-descendant syntax: if/else,
for loops, while loops, variable assignment etc.

Racket Lisp is a little more of a leap. Even something like the behavior of a
for loop is conceptually different. When most of the languages you have been
bouncing around are more Algol-like, Lisp is a bigger leap into a different
programming paradigm.

~~~
dyoo1979
I'm not quite sure I agree with the for loop example specifically for Racket.

See: [http://docs.racket-lang.org/guide/for.html](http://docs.racket-
lang.org/guide/for.html)

At least conceptually, for loops work very much like that in the other
languages. You might be referring to Lisp in the general sense. If that's the
case, remove the "Racket" parts in your comment, refer specifically to the
Lisp family, and then the comment should be ok.

------
AdrianRossouw
I don't know what they authored the documentation, but I quite like what
gitbook.io is doing in this space.

~~~
greghendershott
Racket has a documentation system called Scribble.

Which is actually one of the most interesting features of Racket -- that it
lets you implement a `#lang scribble` which does not even use s-expression
syntax, but you still have the full power of Racket if you need it.

The default `#lang racket` is a wonderful modern lisp, and you can use Racket
as "only" that. But Racket is also a system for making languages.

~~~
brudgers
More on Scribble:

Racketeer Mathew Flatt's video on Scribble's concept and the rationale for
making it: [http://vimeo.com/6630691](http://vimeo.com/6630691)

~~~
pjmlp
Very interesting. Thanks for posting it.

