
Guile-lips: Scheme as a generic macro language - ruste
https://rbryan.github.io/posts/guile-lips-scheme-as-a-generic-macro-language.html
======
junke
For those saying that the code is too verbose, here is another version in
Common Lisp:

    
    
        (defun jim (arg &optional path)
          (etypecase arg
            (string
              (format t "import ~{~(~A~).~}~A;~%" (reverse path) arg))
            (list 
              (push (pop arg) path)
              (dolist (a arg)
                (jim a path)))))
    

Test:

    
    
        (jim
         '(java
           (util "HashMap" "HashSet" "Map" "Random" "Set" "UUID"
            (concurrent "ExecutorService" "Executors" "Callable" "Future"))
           (awt "Color")))
    

Gives:

    
    
        import java.util.HashMap;
        import java.util.HashSet;
        import java.util.Map;
        import java.util.Random;
        import java.util.Set;
        import java.util.UUID;
        import java.util.concurrent.ExecutorService;
        import java.util.concurrent.Executors;
        import java.util.concurrent.Callable;
        import java.util.concurrent.Future;
        import java.awt.Color;

~~~
ruste
If you'd prefer common lisp you should check out the project that inspired
mine: [https://github.com/zc1036/lips](https://github.com/zc1036/lips)

~~~
junke
You did a great job and I am sorry that the discussion is mostly about the
example. I saw the Scheme version and found it quite verbose, like some other
people on this thread. That's what made me want to implement a CL one. Maybe
if you had a less controversial example where you produce e.g. Markdown text,
it would not distract people from the tool.

~~~
ruste
It was verbose. I actually left a comment in the example about how it could
have been written better. Someone else further down in the comments suggested
a nicer way of writing it. I think most people got the idea though! Thanks for
the feedback. :)

------
agentgt
I think many will agree that the import statement example is fairly weak. If
you are writing import statements by hand you should just seriously consider
the benefits of using an IDE.

That being said I use code generation instead of the IDE all the time but for
harder things like generating fluent builders , copying objects, generating
visitor pattern for pattern matching etc.

What I do is write Groovy scripts and will either use Groovy GString templates
or Squares Poet to generate code.

What I do though instead of making some sort of DSL is use a subset of the
language itself to help the code generator. For example copying a list of
private fields I can quickly turn into either an immutable object with builder
or an interface or just a plain object copy etc etc.

I put my Groovy scripts in my ~/bin and then just keep my terminal running. I
then copy from Eclipse/Intellij some code and then run:

    
    
       pbpaste | SomeGroovyTransformScript.groovy | pbcopy
    

Then paste back the code into the IDE. Ideally I would use APT libraries (and
in same cases I have converted Groovy code into APT libraries) but some times
I just want something lightweight.

~~~
outworlder
> I think many will agree that the import statement example is fairly weak. If
> you are writing import statements by hand you should just seriously consider
> the benefits of using an IDE.

Agreed that the example is weak. I do agree that, for import statements,
having and IDE fill those in for you is fair game. It can figure out what you
are trying to use and offer to import those, that's fine.

The problem here is the slippery slope. You get an IDE writing import
statements. Than it will write getters and setters for you – which is already
an aberration that prevents language evolution. For instance, C# has a syntax
for default getters and setters (and for properties, in general).

Then you get IDE plugins for writing boilerplate (for, say, JEE). Then you end
up with a XML preprocessing pass that writes Java code (XDoclet).

It's amazing the amount of machinery that gets created when people do not have
access to a useful macro system.

~~~
agentgt
Yes but there is also abuse and or complexity that doesn't make it easy. One
of the classic examples for me at least is OCaml's Camlp. For Lisp like
languages it is easier to have a macro language but for typed languages I
think it becomes far more difficult both to implement and understand.

A lot of people don't realize that Java offers a macro/preprocess like system
(APT) and it is even sort of hygienic (an example project is Google's Auto
[1]).

APT is not that powerful but it gets you quite far and going back to OCaml the
new ppx language extension is actually fairly similar (ie annotation
processing instead of a complete macro/preprocessing language). .NET has
something analogous but even more powerful (IIRC as well as the whole access
to the AST aka LINQ style).

 _> Then you get IDE plugins for writing boilerplate (for, say, JEE). Then you
end up with a XML preprocessing pass that writes Java code (XDoclet)._

2001 called and wants it example back :) . Yes APT didn't exist for a while.
Tangental but XDoclet sort of reminds me of Go's very weak annotation system.
And again It takes time to get a macro/preprocessing engine right particularly
for typed languages. e.g. Rust has had some fair challenges getting this right
and even their macro system is not completely hygenic IIRC.

[1]: [https://github.com/google/auto](https://github.com/google/auto)

~~~
outworlder
> ... but for typed languages I think it becomes far more difficult both to
> implement and understand.

I don't think that types are the issue but rather that such languages were not
designed with macro systems in mind from the very beginning. Some added
complexity is unavoidable, of course.

> A lot of people don't realize that Java offers a macro/preprocess like
> system (APT) and it is even sort of hygienic (an example project is Google's
> Auto [1]).

I didn't. Will look it up.

> 2001 called and wants it example back

I had to use that abomination as far as 2005. I understand that's a dated
example, but it is the best example of that slippery slope taken to its
extremes that I could think of. It is also a good example in the sense that it
used XML instead of expressing itself in the original language. As such, it is
also a nice illustration of Greenspun's Tenth Rule.

~~~
ruste
I was also completely unaware that Java had a macro system. I'll have to check
that out.

~~~
agentgt
Calling it a macro system is probably a stretch of the word :) but if you
check out APT you should also check out the far sexier Java Agents [1] which
is code that changes other code during runtime (ie alters structure and
provides meta programming).

[1]:
[https://docs.oracle.com/javase/7/docs/api/java/lang/instrume...](https://docs.oracle.com/javase/7/docs/api/java/lang/instrument/package-
summary.html)

------
nothrabannosir
_< 20 lines of parensoup>_

 _> How easy was that!!?_

;)

The problem comes mostly from lisp's dislike for text (my personal opinion).
Lisps work best when everything is a list, always. As soon as you cross the
boundary into... Well.. Anything else , really, you need a little bit of ugly
glue.

That's why it's so ugly, in case anyone was about to make the "ermagerd if
lisp is so great explain this..." comment. :)

[edit: which, btw, is not to lisp's credit, imho.]

~~~
ruste
I completely agree. This is actually better than a lot of cases because Java
import statements are just lisp symbols delimited by periods. This is mostly
only useful if you're transforming lisp statements into code. Transforming
java to java or c to c is just a mess. Still, If I need to do some ugly code
transformation, I'd rather it be in scheme than c. :)

Also, the example could and probably should be rewritten more cleanly. Hence
the comment. That was the first thing I used the tool for and my scheme has
improved since.

------
agumonkey
I don't like the style though. I'd write an accumulative tree traversal
composed with a string formatting final step.

    
    
      ;; :: tree -> [flat-qualified-name]
      ;; :: flat-qualified-name -> string

~~~
ruste
That does sound better. Build up the structure you want then convert to a
string.

------
chriswarbo
I really like the idea of a general-purpose text-replacement pre-processor
like this. I especially like:

\- A "real" programming language (Scheme)

\- Only 1 character to avoid/escape (~)

\- Expression-oriented (as opposed to, for example, impure echo/print)

\- Completely orthogonal/independent of the code being manipulated ( _unlike_
the myriad Java/JVM examples cited by other comments)

One thing that would be nice is to distinguish between pure expressions and
impure ones. That way, pure expressions can _always_ be processed
automatically (e.g. in wrapper scripts around tools like static analysers)
since they're incapable of performing any effects.

Impure expressions would still be useful, e.g. to scan filenames in the
current directory, but can be identified as such and skipped by tools/scripts.

~~~
ruste
Hmm, I think it's a dangerous thing to be skipping portions of code in
analysis just because they're impure. I think you'd be better off just taking
the code produced after processing and running that through your static
analyzer as is. That's what's going to be running in the end anyway. Maybe I'm
not fully understanding the problem though.

~~~
chriswarbo
Well, you could choose to abort rather than skip anything.

The problem is that impure code can do dangerous things (depending on which
i/o primitives are provided). For example, a macro might create and delete a
bunch of temporary files; this might work fine in one scenario (e.g. during
compilation) but misbehave during another (e.g. static analysis).

In which case "static analysis" would no longer be static.

------
Johnny_Brahms
We have this at work, actually. I wrote a macro language for pascal in guile.
At first it was a first draft, but it worked well enough and got to stay.

It generates enormous amounts of code that would otherwise have been
unmaintainable. As it is now, it is just a couple of hundreds of lines of
scheme and pascal code that recursively generates pascal.

We had some problems making it scale (as I said, it generates a lot of code),
but the upcoming 2.2 guile solved most of our issues, and I rewrote parts of
it to make it fast enough. Got it down from 1 minute to 10 seconds on a 10+
minute build when rebuilding everything.

I might be able to opensource it. With minor tweaks, it can probably be made a
bit more language-agnostic.

------
zeveb
Here's the original lips on which this is based:
[https://github.com/zc1036/lips](https://github.com/zc1036/lips)

Maybe I'll use this to hack generics into Go …

~~~
ruste
Are you the author? I tried to find a way to get in contact with him, but
github doesn't have a messaging feature and I couldn't find an email.

~~~
zeveb
Nope, I'm not. Good luck finding him — maybe you could open an issue? Bit of
an abuse of the system, but it would work.

~~~
ruste
I considered that. I may do it yet.

------
Arelius
Very cool!

I had thrown together something very similar using s7 for a super small self-
contained preprocessor.

I'm using `@` as the symbol to consume the next-sexp and process it, otherwise
it's mostly the same.

I'll try to throw together some documentation, examples and a LICENSE in case
anyone wants to use it. But I've just been using it internally as a generator
for Ninja files for my builds.

[https://github.com/Arelius/s7pp](https://github.com/Arelius/s7pp)

------
PaulHoule
Ouch!

I never write those import statements by hand anyway, that's one of the many
reasons I have an IDE.

~~~
codemac
Meta-programming via your editor doing code-gen for you or lisp doing code-gen
for you end with about the same code.

~~~
PaulHoule
Yeah, except IntelliJ Idea is a refined product that does that and 100 other
things.

I don't know what IntelliJ has inside of it, but I know Eclipse is based on
EJC, the Eclipse Java Compiler, which can turn Java code into a Java abstract
source tree, and then turn that either back to Java code or to byte code.

It's not too different from the read function in LISP but it is a bit more
complex...

------
TheLarch
In the case of Java you can use Clojure or Armed Bear Common Lisp anyway.

