>why not simply define functions in the language rather than using the macro system?
I don't know. Most examples I've seen from people extolling the virtues of lisp macros and metaprogramming wind up just generating more lisp code in the same language with the same syntax and semantics, so I don't know why you couldn't just use functions in that case.
To be fair, I only have a surface understanding of one lisp variant (Arc) so it's entirely likely I just don't get it.
A nice example is `match` that is one of the macros that comes by default in Racket.
#lang racket
(define (show x)
(match x
[(vector a b c)
(display "It's a vector, total = ")
(displayln (+ a b c))]
[(list p q)
(display "I's a list, average = ")
(displayln (/ (+ p q) 2))]))
(show (list 8 2))
(show (vector 1 2 3))
The idea is that `match` can analyze the value of the first argument (`x` in this case) and then it binds the variables (a,b,c or p,q in this case).
In this case I'm matching only a vector with 3 values or a list with 2 values, but you can match more complicated and nested pattern.
The interesting part is that inside match you have the full power of the language, you are not restricted to a few special hardcoded cases (like `display`). You can send an hppt request, play a sound, save a value to disk, calculate the prime factorization, transform the text to uppercase, anything that is available in the language is available inside match.
But `match is just a normal macro. Racket is implemented in layers. The bottom layer is implemented currently in C, but it doesn't define `match`. The bottom layer is almost hidden and it is used to define some libraries and an intermediate language that it is used to define some libraries and an intermediate language, ..., and after a few steps you get the official Racket language.
But if you like you can reimplement a macro like `match`, or a variant, or your own macro, with all the magic to define local variables and use all the power of the language inside it.
Syntax is different from semantics. To give a concrete example, say you want a part of a program to use lazy constructs. In racket, you could use `#lang lazy` for that one module and then import and use that code within `#lang racket` modules.
Despite having the same syntax, `#lang lazy` and `#lang racket` behave differently:
Eager evaluation prevents many forms from being implemented at the function level. You can't for instance easily write a short-circuiting and() function in most languages, because arguments to a function are eagerly evaluated.
So given an and() function, you can't safely do "and(False, fire_missiles())", because the language will evaluate both arguments.
But an and!() macro could: macro expansion is essentially "lazy", in that it happens prior to the evaluation phase, so it can avoid evaluating any pieces of code it wants to, such that "and!(False, fire_missiles());" is perfectly safe, because our macro can stop expanding after "False" and thus fire_missiles() is never evaluated.
Incidentally, this is why at least some macro patterns are unnecessary in lazy languages like Haskell: you can write short-circuiting and() as a function there, because a function in Haskell only evaluates as much as is necessary (provided it's been written properly). Yet even there, Haskell still has support for macros and things like TemplateHaskell and so forth, because there's just some things you can't do solely with functions, like arbitrary syntax, language extensions, etc.
> why not simply define functions in the language rather than using the macro system ?
Modifying control flow, for one. A classic example is anaphoric if. If you want to implement that, you can't do it just as a straight forward function. It needs to be a function that modifies syntax, aka a macro.
Unless your goal is to be source compatible with another lisp variant, of course.