
Writing Complex Macros in Rust: Reverse Polish Notation - jgrahamc
https://blog.cloudflare.com/writing-complex-macros-in-rust-reverse-polish-notation/
======
pleasecalllater
So it looks like people repeat how C++ templates are bad, and Rust macros are
good. Can someone tell me what's the real difference? I'm not sure I get it,
but I feel like both are good or both are bad.

~~~
devit
C++ templates are bad because they are untyped generics (e.g. C++ doesn't
check that the key type for an hashmap is hashable, and instead you get a
bunch of errors that there is no "hash" overload accepting certain arguments
when you try to call a method on the hashmap).

C++ tried to add a type system for templates with "concepts" but it was
scrapped.

Rust instead has properly typed generics instead using the trait system,
resulting in proper static checking and proper error messages (although
without higher kinded types and const generics currently, that C++ instead
supports).

Rust also has a powerful macro system, which is unrelated to templates,
although you can use them to have kinds of abstractions that the language
doesn't support directly (like higher kinded types, const generic or
abstracting over mutability), at the cost of having to explicitly perform
instantiation, not having inference and having a higher cognitive burder.

~~~
pjmlp
Concepts are on track for C++20 and they are already available on gcc to play
with.

Besides that, it is already possible to use typed templates in C++, at least
with any C++14 compiler and even better on C++17, even if it requires a bit
more of effort for the implementer of the generic code.

Basically by making use of type traits, static asserts, constexpr if and
enable_if.

Yes, it isn't as clean as Rust but as shown by Andrei Alexandrescu with D, it
opens the door for very powerful designs.

~~~
ogoffart
But the C++20 concepts is only concept lite. It does not check the
implementation of the template function that it only uses operations available
for the conceps.

suppose you have a concept Hashable that is only enabled for type with a
hash() function. But you can still in the code of your HashMap do a comparison
with the < operator, which the Hashable concept don't ensure. In that case you
woud only have an error at instentiation type for Hashable types that don't
have '<'

In Rust, the compiler make sure that your generic function only use operations
that exits in the traits declared in the function signature.

------
tomsmeding
I didn't actually try out the code, so I don't know whether it actually works
regardless, but this looks _really_ suspicious:

    
    
        = note: expanding `rpn! { @ op [ 4 , 3 + 7 , 2 ] * }`
        = note: to `rpn ! ( [ 3 + 7 * 4 , 2 ] )`
    

Is that not, like, operator precedence gone really wrong? Or is this the rust
stringifier in trace_macros messing up without anything going wrong in the
actual AST?

~~~
dbaupp
The trace_macros! stringifier isn't inserting parens when required: due to the
use of 'expr' the macro is treating 3 + 7 as a single node _X_ , and then the
multiplication with 4 is, in the AST, _X_ * 4 (i.e. correct). The thing that
is being printed there is the recursive rpn invocation in the @op branch, and
specifically the '$a $op $b', which is a "disconnected" sequence of AST
items/token trees. Once it's forced to be parsed as an expression (that is,
when matched against the recursive '$... : expr' rules), the printer handles
it correctly. For instance, the last 'to' line of the output in the example
you quote, or, if that example is changed to 'rpn!(2 3 7 + 4 * 5)', the
trace_macros output includes:

    
    
       ...
       = note: to `rpn ! ( [ 5 , (3 + 7) * 4 , 2 ] )`
       = note: expanding `rpn! { [ 5 , (3 + 7) * 4 , 2 ] }`
       ...

------
mcguire
Rust macros strike me as closely related to Scheme hygienic macros; is there a
compare-n-contrast anywhere?

~~~
kibwen
The original designers of the macro system absolutely had Scheme firmly in
mind (source: I was there). The most telling giveaway is that the identifier
for declaring macros, "macro_rules", is a reference to Scheme's "syntax-
rules". As for a comparison, Rust macros have to jump through hoops to account
for the fact that Rust has syntax (and types), and the Rust system isn't
entirely hygienic yet (work in progress).

------
zwieback
As an hp-er I'm all for RPN but that code is making my eyes bleed. I'm
assuming it's just an example and not some kind of endorsement.

~~~
RReverser
Yeah just used it as a (hopefully) simple and familiar example to explain
macro mechanics.

------
kazinator
Here is my ten minute implementation in TXR Lisp:

Trying _rpn-compile_ at the REPL:

    
    
      $ txr -i rpn.tl
      1> (rpn-compile '(3))
      (let* () 3)
      2> (rpn-compile '(3 4 +))
      (let* ((#:g0266 4)
             (#:g0267 3)
             (#:g0268 (+ #:g0266 #:g0267)))
        #:g0268)
      3> (rpn-compile '(3 4 + 5 6 + *))
      (let* ((#:g0269 4)
             (#:g0270 3)
             (#:g0271 (+ #:g0269 #:g0270))
             (#:g0272 6)
             (#:g0273 5)
             (#:g0274 (+ #:g0272 #:g0273))
             (#:g0275 (* #:g0274 #:g0271)))
        #:g0275)
      4> (rpn-compile '(3 4 + dup *))
      (let* ((#:g0276 4)
             (#:g0277 3)
             (#:g0278 (+ #:g0276 #:g0277))
             (#:g0279 (* #:g0278 #:g0278)))
        #:g0279)
    

Code in _rpn.tl_ :

    
    
      (defstruct rpn-compile-time nil
        temps
        lets
        stack)
      
      (defun allocate-temp (rct)
        (push (gensym) rct.temps)
        (first rct.temps))
      
      (defun compile-expr (rct expr)
        (if (member expr rct.temps)
          expr
          (let ((temp (allocate-temp rct)))
            (push ^(,temp ,expr) rct.lets)
            temp)))
      
      (defun compile-dup (rct)
        (let ((top (compile-expr rct (pop rct.stack))))
          (push top rct.stack)
          (push top rct.stack)))
      
      (defun compile-swap (rct)
        (swap (first rct.stack) (second rct.stack)))
      
      (defun compile-binop (op rct)
        (let* ((left (compile-expr rct (pop rct.stack)))
               (right (compile-expr rct (pop rct.stack)))
               (sum (compile-expr rct ^(,op ,left ,right))))
          (push sum rct.stack)))
      
      (defvar *compile-table*
        ^#H(() (dup ,(fun compile-dup))
               (swap ,(fun compile-swap))
               (+ ,(op compile-binop '+))
               (- ,(op compile-binop '-))
               (* ,(op compile-binop '*))
               (/ ,(op compile-binop '-))))
      
      (defun rpn-compile (exprs)
        (let ((rct (new rpn-compile-time)))
          (each ((word exprs))
            (iflet ((fun [*compile-table* word]))
              [fun rct]
              (push word rct.stack)))
          ^(let* ,(reverse rct.lets) ,(first rct.stack))))
    

Having this function, making the macro is trivial. At the REPL again:

    
    
      5> (defmacro rpn (. exprs) (rpn-compile exprs))
      rpn
      6> (rpn 2 2 +)
      4

~~~
jnordwick
Does the evaluate at compile time to a constant or does it produce that series
of let bindings?

~~~
kazinator
Nope; it makes the _let_ bindings. The job of reducing to a constant is better
left to the processing of the _let_ rather than duplicated in the macro.

If you have a compiler which can reduce (let ((x 3)) ((y 4)) (+ x y)) to 5,
why replicate that in a macro.

If you do not have one (as is the case here), then it's better to work on
getting one than compensating for it in macros.

Yet, with the following small change to _compile-expr_ , whereby we avoid
generating temps for constant expressions, we can at least get some nicer-
looking output:

    
    
      (defun compile-expr (rct expr)
        (cond
          ((member expr rct.temps) expr)
          ((constantp expr) expr)
          (t (let ((temp (allocate-temp rct)))
               (push ^(,temp ,expr) rct.lets)
               temp))))
    
      1> (rpn-compile '(3 4 + 5 6 + *))
      (let* ((#:g0266 (+ 4 3))
             (#:g0267 (+ 6 5))
             (#:g0268 (* #:g0267 #:g0266)))
        #:g0268)

------
inurigakko
Rust is probably the most advanced programming language with the worst syntax.
Prolog and Erlang are a joy after Rust.

~~~
carlmr
What is so terrible about the syntax? My biggest problem with Rust isn't the
language, but the tooling which is still not quite there yet.

I find the syntax ok, and you do get used to the snake_case.

~~~
jandrese
My impression has always been that it probably feels right at home for people
with mathematical backgrounds, and mildly line-noisy for people coming from
traditional programming backgrounds.

Stuff like this:

    
    
      fn map<T, U, F>(vector: &[T], function: F) -> Vec<U>
        where F: for<'a> Fn(&'a T) -> U {
        ...
    

It's not too hard to read once you're in the ecosystem, but there's a lot of
non-alphanumerics in there, and you know each one of them is super important
to getting the thing working.

~~~
carlmr
While I like the functional style you actually did bring up one of the cases
where I find the syntax a bit too much. Why can't we have the functional
interface information on F directly in the signature?

Why not something more akin to ML?

    
    
        fn map<T, U, 'a>(vector: &[T], function: &'a T -> U) -> Vec<U>

------
agumonkey
RISP

------
cheez
It looks like Rust doesn't have your dad's Lisp macros and more like C-style
macros on steroids. Disclaimer: I haven't read The Book.

~~~
navaati
Well C macros work at the text-level, Rust macros run at the AST level (but
not the type level, that is you cannot construct syntactically incorrect stuff
but you can construct wrong-typed stuff).

So I don't really know lisp macros but I guess it's still closer than to C.

~~~
foldr
Rust macros actually work more at the token level than at the AST level.

~~~
RReverser
They work on both - for example, if you specify $expr:expr, it will actually
parse it as an expression node and won't allow to simply pass it into another
macro as $pat:pat because of AST node type mismatch even if actual tokens are
still compatible.

~~~
foldr
That's a good point. What I was trying to get at is that Rust macros don't let
you manipulate ASTs in any straightforward way. So they are not best thought
of as AST -> AST transforms.

