
A Practical Intro to Macros in Rust - jxub
https://danielkeep.github.io/practical-intro-to-macros.html
======
kibwen
Note that this dates back to Rust 1.0, and although the macro system hasn't
changed since then, plans are in the works to succeed this original system
("macros 1.0") with an improved one ("macros 2.0"), culminating in the
deprecation of the original. The two systems won't differ _that_ much, so the
documentation will remain mostly the same and the transition should be very
straightforward.

The impetus for this is that the macro system was unanimously considered the
least mature feature of pre-1.0 Rust, but there was little will to delay Rust
1.0 even further solely for the sake of macros. Instead, some expedient band-
aids were slapped over it during the run-up to 1.0, with the intent to
eventually fix the whole thing properly when time allowed.

To wit, the main problems with macros 1.0 were that macro "importing" is an
ugly hack that stands out like a sore thumb from how modules work in the rest
of the language; in addition, despite supposedly being hygenic, there are some
edge cases where hygiene does not hold. The new system will make the language
overall more consistent and reliable (and might bring some minor syntactic
improvements), with the obvious downside that making the macro system nicer to
use means that people might actually start using it. :P

~~~
FrozenVoid
Unfortunate, because metaprogramming is very important to code design. I won't
want to abandon code just because v2.0 will get deprecated and shiny v3.0 will
become the new standard next year. If Rust team cared about backward
compatibility they could preserve old code by adding something like
-std=rust1.0 which would not break code. Forcing people to rewrite any
code(esp. complex macros) will not gain Rust any goodwill longterm.

[https://github.com/rust-lang/rust/issues/39412](https://github.com/rust-
lang/rust/issues/39412)

> Eventually, I hope we can remove macro_rules!. That will take a long time,
> and would require a 2.0 version of Rust to strictly adhere to our stability
> guarantees.

~~~
scott_s
I don't think you're being charitable to the Rust team. They are quite vocal
about how important they think API stability is. But they're also vocal about
how important it is to get it right. For those reasons, they're upfront about
what APIs are guaranteed to not change, and what APIs are still experimental.
As an outsider, I think they do an excellent job of being transparent about
the process.

~~~
im_down_w_otp
Having used Rust off and on for the better part of 4 years through a lot of
tumult and change, I agree with you.

In most cases the kinds of breaking changes and immaturities that the Rust
core team communicates clearly to the ecosystem would be represented as
nothing but silence and a giant dump of breaking changes suddenly in a lot of
other ecosystems.

Sometimes the Rust team seems to pay a price for transparency, in that it
gives people visibility, and thus something to make an opinion about and/or
complain about. In an information vacuum there's nothing to opine about except
the vacuum itself.

As annoyed as I often was, and occasionally still am, about different shapes
of API stability in the language/libraries; I'm extremely happy the Rust team
has maintained a commitment to developing everything out in the open.

------
shepmaster
The author also created _The Little Book of Rust Macros_ [1], which is an
invaluable resource. They also are one of the top answerers for Rust questions
on Stack Overflow[2]. Altogether a great asset for the community!

[1]:
[https://danielkeep.github.io/tlborm/book/](https://danielkeep.github.io/tlborm/book/)

[2]:
[https://stackoverflow.com/users/42353/dk?tab=answers](https://stackoverflow.com/users/42353/dk?tab=answers)

------
panic
The concept of a "token tree" as a separate stage between ordinary
tokenization and full parsing seems interesting on its own: was it based on
another system or invented for Rust? For example, you could imagine
implementing incremental parsing by writing your parser as a transformation on
a token tree.

~~~
nine_k
Isn't it how Lisp macros work?

~~~
panic
Yep! I have also been pointed at this PhD thesis:
[https://www.cs.utah.edu/~rafkind/papers/dissertation.pdf](https://www.cs.utah.edu/~rafkind/papers/dissertation.pdf)
which defines a macro system on a uniform Algol-like syntax with intermediate
"reading" and "enforestation" phases and corresponding syntax representations
(the "read" representation being pretty much the same as the token tree
representation here).

------
surrealize
Cool! Any particular reason why you'd shuffle like that, as opposed to using a
circular buffer? With IndexOffset, you've almost got a circular buffer going
already.

------
jordigh
I still find myself most comfortable with D's compile-time function evaluation
(CTFE) and mixins. In D, one use of mixins is more or less an eval function:
take any string and evaluate it as code at the invocation site.

[http://dlang.org/mixin.html](http://dlang.org/mixin.html)

It's nice that you need very little new syntax to do compile-time tricks in D.
The same syntax that works at runtime also works at compile time. If mixins
worked with the AST instead of strings, it would be almost the same as Lisp
macros.

I'm also kind of happy to see that D seems to have inspired Rust macros a
little.

------
VermillionAzure
Hi. I'm a lead developer on the Shaka Scheme project that hopes to accomplish
implementing Scheme R7RS, which also has hygienic declarative macros.

R7RS Scheme has `syntax-rules` which is very similar in structure to this
macro system. It's quite elegant as it is, in my opinion, but I'm also
wondering how good the debugging facilities for it are. Are they good?

~~~
kibwen
Scheme's syntax-rules was the direct inspiration for Rust's macro system,
actually. As for debugging, I don't write my own Rust macros very often (and
when I do, they're not very extensive), but I'd characterize the experience as
"okay". Rust reports errors in macros as originating from the unexpanded line
of code, which is useful for macro users but less so for macro authors. But
you can ask the Rust compiler to emit the code that it sees post-expansion, if
you want to inspect the generated output directly.

~~~
Manishearth
There's a `trace_macros` macro that you can use for debugging. It's somewhat
useful.

------
solidsnack9000
The colored diagrams use to explain contexts and hygienic macros are a neat
idea.

------
Animats
The intro to macros starts with a recursive macro. Not a good approach for an
intro. Recursion in macros is usually more of a toy than a useful tool,
anyway.

~~~
Manishearth
No, it _starts with_ a non recursive macro that does less stuff, and builds up
to the recursive one.

This resource is a "learning by example" resource, and it teaches the macro
bit by bit starting from a simplified version and building up to the recursive
and complex part, teaching concepts along the way. That's a great way of
structuring it.

~~~
beefsack
I actually think it's great that it starts by showing something that looks
really impressive, it immediately made me think of list comprehensions in
Haskell. I was very keen to keep reading as soon as I saw that.

