http://ycombinator.com/arc/arc3.tar

If you want a single, self-contained example, here's one:

    (mac zap (op place . args)
      (with (gop    (uniq)
             gargs  (map [uniq] args)
             mix    (afn seqs
                      (if (some no seqs)
                          nil
                          (+ (map car seqs)
                             (apply self (map cdr seqs))))))
        (let (binds val setter) (setforms place)
          `(atwiths ,(+ binds (list gop op) (mix gargs args))
             (,setter (,gop ,val ,@gargs))))))

    arc> (= flag t)
    t
    arc> (zap no flag)
    nil
    arc> flag
    nil
    arc> (= foo '(1 2 3 4 5))
    (1 2 3 4 5)
    arc> (zap (fn (x) (* x 10)) 
              (foo 0))
    10
    arc> foo
    (10 2 3 4 5)

    (zap [* _ 10] foo.0)

    (zap [firstn votewindow* _] (uvar user votes))

    (= ((profs* user) 'votes)
       (firstn votewindow* ((profs* user) 'votes)))

    void zap(void *object, void* (*selector)(void *), void (*action)(void*)) {
        action(selector(object));
    }

Where object is the structure you're operating on, `selector` is a function taking an object and returning a pointer to the position [2]. The `action` is a pointer to an arbitrary function [1].

Your first example would look like this:

    int foo[5] = {1, 2, 3, 4, 5};
    void *select_first(void *x) {return x;}
    void act_times_10(void *x) {int *i = (int *)x; (*i)*=10;}
    
    zap(foo, select_first, act_times_10);

I was simply surprised that PG asked about something like that - I still think it's pretty simple and the code in C is as close to the macro as possible (with whatever object abstraction you need).

However I have to admit that I thought about other languages now and it seems that the described zap() is not possible in python... and that's interesting / worrying:

    action(object[place])

    object[place]=action(object[place])

  void zap(*action, *selector, *object) { ... }

  (mac zap (op place . args) ... )

  (zap [firstn votewindow* _] (uvar user votes))

  del uvar(user, votes)[votewindow:]

  uvar(user, votes)[votewindow:] = []

  template<typename T>
  void zap (T (*f)(T x), T &x) { x = f(x); }

  int square (int x) { return x*x; }

  int foo[5] = {1,2,3,4,5};

  zap<int>(square, foo[3]);   // foo is now {1,2,3,16,5}

The entire point of programming is automation. The question that immediately comes to mind after you learn this fact is - why not program a computer to program itself? Macros are a simple mechanism for generating code, in other words, automating programming. Unless your system includes a better mechanism for automating programming (so far, I have not seen any such mechanisms), _not_ having macros means that you basically don't understand _why_ you are writing code.

This is why it is not surprising that most software sucks - a lot of programmers only have a very shallow understanding of why they are programming. Even many hackers just hack because it's fun. So is masturbation.

This is also the reason why functional programming languages ignore macros. The people behind them are not interested in programming automation. Wadler created ML to help automate proofs. The Haskell gang is primarily interested in advancing applied type theory.

It's a bit more complicated than that. The key idea is that programs are composed of lists. But that idea is tied to several others:

Imagine this Christmas, Perl 6 is out with some killing features. Now, your language (Python? Haskell? ML? Whatever) does not have these features. What would you do?

You have 2 choices, basically: a. forget about that features, write a long blog in which you prove how your language can do just the same thing, albeit "a little" less elegant b. Switch to perl. This is not always possible, btw, since you may sacrifice some OTHER features that your language has.

How about a LISPer? Well, simple, let's implement these features. Keep in mind: All languages are Turing equivalent, meaning they can do roughly the same thing. There are two differences: 1. some do some task more elegantly (aka easier to maintain) than others. 2. Some have better support (eg. corporates, community, etc.) than others

Thus, you CAN always write the exact same features in LISP. Now, the definition of the macros WILL be messy. However, as a good practice, you spend 2 days debug that holy mess, and jam it into a file you banish somewhere with a dot precede the directory. Next time, you can use the features, pretending that you are using Perl 6!

First it finds all 'features' by traversing a 1-pixel wide representation of the text character's image. Then you describe which features you want to look for, in a macro language.

So (in layman's terms) , a 'c' is a top curve curving up, a lower curve curving down, and a stem at the left-hand side close to the left-hand of those curves. So 'c' would be about 3 rules, in this macro language.

It was fun but monstrously sluggish and I never got the 1-pixel wide algorithm (called skeletonisation) that I used, to work 100%.

One thing it did have going for it was you could identify really small font chars and even 10-pixel-high truetype fonts, as long as you normalise so each char is 60 pixels high.

  let vars
  in expr

http://common-lisp.net/project/iterate/doc/index.html

http://www.lispworks.com/documentation/HyperSpec/Body/06_a.h...

Problem: I always hated that (loop for element in list collect (my-fun element)) translates trivially to (mapcar #'my-fun list) but (loop for element in list collect (my-fun element my-constant)) doesn't. You'd either have to stick with the loop version or use something ugly and inefficient like (mapcar #'my-fun list (make-list (length list) :initial-element my-constant)) or write out the lambda: (mapcar (lambda (element) (my-fun element my-constant)) list) (my-constant could actually be an arbitrary form).

None of these solutions appealed to me. So I wrote fmask. With it, the example is simply rewritten as (mapcar (fmask #'my-fun ? (? my-constant)) list)

Another example:

  (mapcar (fmask #'list ? (? (1+ (length ?))))
  		 '(3 8 2) '(("some" "items") "A string" (nil "test")))

Here's the implementation:

  (defmacro fmask (function unknown (&rest args))
    (check-type unknown symbol)
    (macrolet ((autotraverse-let (bindings &body body)
	         (let ((gensyms (iter (repeat (length bindings))
  				    (collect (gensym "TRAVERSED")))))
  		 `(let (,@(iter (for (nil value) in bindings)
  				(for gensym in gensyms)
  				(collect `(,gensym ,value))))
  		    (symbol-macrolet (,@(iter (for (var nil) in bindings)
  					      (for gensym in gensyms)
  					      (collect `(,var (prog1 (car ,gensym)
  								(setf ,gensym (cdr ,gensym)))))))
  		      ,@body)))))
      (labels ((unknownp (arg)
  	       (etypecase arg
  		 (atom (eq arg unknown))
  		 (cons (some #'unknownp arg))))
  	     (literalp (arg)
  	       (or (typep arg '(or array keyword pathname number))
  		   (and (consp arg)
  			(or (and (eq (first arg) 'function)
  				 (cdr arg)
  				 (not (cddr arg)))
  			    (eq (first arg) 'quote))))))
        (let ((unknowns (iter (repeat (count-if #'unknownp args))
  			    (collect (gensym "UNKNOWN"))))
  	    (knowns (iter (for arg in args)
  			  (if (not (or (unknownp arg) (literalp arg)))
  			      (collect (list (gensym "KNOWN") arg)))))
  	    (known-function (and (consp function)
  				 (eq (first function) 'function)
  				 (symbolp (second function))
  				 (second function))))
  	(autotraverse-let ((an-unknown unknowns)
  			   (a-known (mapcar #'first knowns)))
  			  (let* ((unknown-function (gensym "UNKNOWN-FUNCTION"))
  				 (call-function (if known-function
  						    (list known-function)
  						    `(funcall ,unknown-function)))
  				 (main (let ((inner `(lambda (,@unknowns)
  						       (,@call-function
  							,@(iter (for arg in args)
  							     (collect (if (unknownp arg)
  									  (subst an-unknown
  										 unknown
  										 arg)
  									  (if (literalp arg)
  									      arg
  									      a-known))))))))
  					 (if knowns
  					     `(let (,@knowns)
  						,inner)
  					     inner))))
  			    (if known-function
  				main
  				`(let ((,unknown-function ,function))
  				   ,main))))))))

fmask's implementation with syntax coloring:

http://paste.lisp.org/display/81488

(mapcar (rcurry #'parse-integer :junk-allowed t) '(" 1" "2xx" "3" "foo")) (1 2 3 NIL)

... whereas your more complex example doesn't strike me as an improvement over lambda + backquote:

(mapcar (fmask #'list ? (? (1+ (length ?)))))) vs.

(mapcar (lambda (n list) `(,n (1+ (length ,list)))) numbers lists)

... which might be longer than the fmask version but has the advantage of being immediately readable to any Lisp programmer, and of allowing arguments to be reused, or used in a different order than they appear.

Doug Hoyte has a nice read macro called #`:

http://www.newartisans.com/2009/03/hello-haskell-goodbye-lis...

The rest of us just rewrite the critical stuff in whatever that's the most efficient, and the non-critical stuff in whatever that's easiest.

http://gmarceau.qc.ca/blog/2009/05/speed-size-and-dependabil...

Example, in pseudo code:

<code> defun regex_find(some_string, regex) return regex.compile().find(some_string)

def compilermacro regex_find(some_string, regex) if IS_STRING_LITERAL(some_string) and IS_STRING_LITERAL(regex) SUBSTITUTE exec(regex.compile().find(some_string)) elif IS_STRING_LITERAL(regex) SUBSTITUTE exec(regex.compile()).find(some_string) else DO_NOTHING return

</code>

regex_find(MYSTRING, '\w(.+)\b')

A language without compiler macros would compile the regex at runtime, even in those cases that it is a string literal, and could be compiled at compile time. And in the case that both the string and the regex are string literals, the result itself could be computed at compile time. Another alternative, used by lots of languages/libraries, is to maintain a cache of recently compiled regular expressions. Of course, this doesn't help when you have a loop that uses 15 regular expressions but it only caches the last 14, for example.

This is one reason why CL-PPCRE is faster than the C PCRE in benchmarks, it makes great use of compiler macros.

This is a very concrete real world thing, that isn't endlessly debatable, unlike regular macros, and it could be included in languages without any special macro syntax.

If you want to explore the highest level of code transformation, grab a nice little book called "Term-Rewriting and all That".