
Ante: a compile-time language - mabynogy
https://github.com/jfecher/ante
======
jfecher
Hello, I have been working on this project for a while now and would love to
take any questions anyone may have.

~~~
ivanbakel
What checking is done on code generated by the compiler extensions? Do you
manually have to build the code out of well-typed components, or does the
compiler check again after each pass?

As an aside, it's very cool to see this implemented, though it does give me
the depressing thought that none of my ideas are original anymore. Hopefully
more languages follow suit and allow for this kind of utility.

~~~
jfecher
The Ante code to generate each portion of llvm-ir is checked as normal but
there is no checking done on the llvm-ir that is generated. It is expected for
each extension producing ir to generate valid output. That being said, one of
the cool things about Ante (imo) is that if you dont like this, you can write
an extension that does check for validity.

------
couchand
Meh. Looks like a fun toy, but it seems unlikely that this adds much value
over directly modifying the LLVM IR. If you've understood enough about the
LLVM model to write these constructs you're better off using the C++ API. That
evolves pretty quickly and is very extensive, so I'm having a hard time
imagining this being nearly as powerful. If it offered a significantly more
elegant model that would be one thing, but the README example seems to add
only an extra layer of APIs to learn. And based on the test suite it looks
like the feature list is aspirational.

~~~
tom_mellior
> you're better off using the C++ API

Using it for what? If I understand this project correctly, it allows you to
put an application and corresponding application-specific language
features/optimizations into one source file. You can then always be sure that
those language features and optimizations are available.

I have a hard time seeing how this would work with LLVM's C++ API in a
comparable way. There you would have to (1) write the application in an LLVM-
supported language, (2) use some other transformation language to add features
to your surface syntax, (3) separately compile your LLVM extension pass
written in C++, (4) make sure your LLVM extension is used when compiling the
application.

~~~
couchand
It's not easy to talk specifics, since as far as I can tell the example in the
README is currently unimplemented - there's no comparable test and there
doesn't seem to be any code exporting the APIs used there.

But from what I can glean from that short example, the proposed API is
essentially LLVM's IRBuilder class, with a slightly different interface. This
seems like a strategy that's unlikely to be practical, because of all the
complexities involved in reimplementing another project's API, particularly
one that changes as frequently as LLVM's.

I'm not sure I understand your breakdown of four steps. As far as I know, it
would be very hard to make a backend pass that would have enough information
encoded purely in IR from a frontend to compile a construct like goto. This is
the sort of thing that needs to be deeply integrated into the compiler, so the
author of that feature would necessarily need to understand the data
structures and architecture of the compiler. At that point, it's probably
easier and more reliable to write it as a compiler extension.

I do like the general idea of being able to extend the operation of a compiler
at application compile time (rather than compiler compile time), but I am
skeptical that the solution involves directly mucking about with LLVM. That's
too low level, the language needs to provide a useful model of abstraction
above that to be better than just modifying the code of the compiler itself.
As others have mentioned here, Lisp is a great example of offering such a
model.

~~~
jfecher
Hello, author of the language here. You are correct that the project is an
early state, I wasn't planning on spreading the word publicly for some time
but as long as it was posted here I can answer some questions.

You can extend the compiler's functions during compile-time by doing more than
just messing with llvm-ir. You are free to implement an automatic linter, or
garbage collector for example. You can even do an optimization pass without
operating on the llvm-ir. To do this, you could walk the parse tree of a
function before it compiles and change around any common node patterns you are
looking for before it is translated to llvm-ir. Alternatively, you can muck
around in the ir and do the same thing.

The goto construct in the example provided does not need to be deeply
integrated because the existing API is setup largely separate from the
internal structure of the compiler. For example the function ctStore stores a
variable in a compile-time container separate from the scope-separated table
used internally by the compiler for other variables. The primary advantage of
Ante as a language is that these functions enable compiler extensions to be
made within the program and thus allow the swapping out of, eg. a garbage
collector, without recompiling the compiler itself. This lets each application
developer decide what features they need for their application. Instead of
changing languages for a gc or desired optimization pass, just swap out a
library.

I share your concern with the LLVM library constantly updating and the need to
update bindings with it. I plan sometime to use the clang api to make a C++ ->
Ante converter program, although I expect there to be numerous difficulties
with its implementation.

~~~
couchand
Hi! Always fun to see new language ideas and watch them grow.

I think the general idea is interesting. However, I suspect that you're
underestimating the complexity of the things you've mentioned. I don't mean to
be too negative, but have you implemented garbage collection? Getting it right
for just one language takes lots of serious effort. With the added
complication of needing to deal with other arbitrary user-designed language
constructs, it might be an intractable problem.

Brainstorming ideas for language design is great. Trying new things is
fantastic. But it's important to be humble and realistic, and to recognize
that language design requires making thoughtful tradeoffs.

~~~
jfecher
That's completely fine, negativity isn't inherently bad in the first place. I
have done some work with a garbage collector for an interpreted language I had
previously worked on before. It was very basic as far as garbage collectors
go, it just had a simple mark-sweep algorithm, nothing too complex. I'm not
denying that creating a garbage collector would be quite an ambitious process,
my goal as a language developer is just to give enough control of the language
to the user to make that possible.

That being said, by no means do I claim that all extensions will work together
perfectly, or at all. For example, a garbage collector would be completely
incompatible with a plugin that frees all variables once they go out of scope.
Compatibility is left up to the plugin's author in the case of multiple
plugins having conflicting goals.

I'm not sure where I stated I was opposed to making tradeoffs, I have done
several (albeit mostly syntactic) already. I knew the project was ambitious
when I first started it, that is part of the fun of working on it.

------
jnbiche
This is awesome, don't know how I missed it before. Kind of like a more
functional Terra. I'll have fun tonight playing around with this.

------
nwmcsween
Ideally you wouldn't directly expose the compiler to make optimizations. You
would create an IR of which the compiler could use and mung around with the IR
instead of the compiler directly.

~~~
daxfohl
Why? Are there other languages that take this approach? What is the advantage?

~~~
jonesetc
Yep, this is how anything using LLVM works:
[http://llvm.org/docs/LangRef.html](http://llvm.org/docs/LangRef.html)

~~~
daxfohl
So what is the advantage or disadvantages of either approach?

~~~
nwmcsween
The advantage is you are tied to the IR and not the compiler (if the language
designs it's own IR), the disadvantage is the same as the compiler optimizes
via passes on the IR. What would be needed is a way to 'hold' optimizations on
the IR so that you aren't playing a game of will-it-optimize (or worse will-
it-not).

~~~
daxfohl
Wait, this "optimization" stuff seems to be a tangent. So the basics: "hello
world" for Ante is a "goto" macro. Forgive me but I'm having trouble seeing
how mucking with IR yada yada produces "goto" in the target language so
succinctly. Ante's README demonstrates this marvelously. How would this be
done in the IR approach? It seems like it would have to be much more complex.

~~~
couchand
Umm... it IS that IR nonsense. This example directly manipulates LLVM IR.

------
rosstex
Could someone explain this simply?

~~~
throwaway91111
it's three things:

* a syntax, including literals

* first class support for llvm IR and primitives

* a macro system allowing you to create new ir-level primitives with aforementioned syntax.

It seems like a lisp without homoiconicity and a complex syntax (compared to
sexps). Think: greenfield compiler development, or maybe a DSP DSL.

~~~
daxfohl
Or a lisp that forsakes those things to gain the advantage of compile time
type checking etc.

~~~
xfer
You don't need to forsake any of that to get compile-time type checking.

------
vsviridov
If you are into things like that, check out Jai, an experimental language
being developed by Jonathan Blow (of Braid and The Witness fame).

[https://www.youtube.com/playlist?list=PLmV5I2fxaiCKfxMBrNsU1...](https://www.youtube.com/playlist?list=PLmV5I2fxaiCKfxMBrNsU1kgKJXD3PkyxO)

------
ajarmst
[https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule](https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule)

------
aidenn0
Remember that meta is the greek word for "after" which is why metaprograms run
before programs

~~~
sillysaurus3
Clever, but not entirely accurate. May as well be pedantic: In some Lisps, you
can set up the program so that it recompiles chunks of the program during
execution. Which is either after the program runs or before the next evolution
of the program, depending on how you look at it.

~~~
aidenn0
In most lisps, macroexpand is before eval.

