
Circle – A C++ compiler with compile-time imperative metaprogramming - orbifold
https://github.com/seanbaxter/circle
======
stephc_int13
Metaprogramming is very alluring on the surface, we've all been frustrated by
the limitations of our languages of choice at some point or another.

But, I think this trend might lead to extremely hard to read code, and there
is a good chance that this hard-to-read code will be treated as some black
box/voodoo.

It might not be a better idea than self-modifying machine code or some of the
wildest C macros...

~~~
ulrikrasmussen
Metaprogramming via reflection is used heavily in the JVM ecosystem, and I can
say with confidence that a majority of the bugs I encounter in third party
code is somehow related to reflection. I think the overuse of reflection in
Java is a symptom of the language not having adequate support for expression
the abstractions that developers need in an affordable way, and hence they
turn to reflection to work around these limitations. This leads developers
down a path where they can't seem to stop applying reflection until they reach
a point where the type system gives you almost no meaningful guarantees and
compositionality of your components is ruined.

At least compile time reflection and code generation will catch a large chunk
of bugs that would otherwise be deferred to runtime. I will take a puzzling
compile time error message over having to debug runtime reflection errors any
day.

~~~
tannhaeuser
> _I think the overuse of reflection in Java is a symptom of the language not
> having adequate support for expression the abstractions that developers need
> in an affordable way, and hence they turn to reflection to work around
> these_

Another explanation is that playing around with annotations and AST
decorations/rewriters is an excellent excuse for procrastinating through your
day without having to deal with mundane business code :). I also think there's
a psychological effect at work here where you're dellusioning yourself into
being a "tool developer" and part of an academic discourse when you throw
around annotation libs, to deflect from the harsh reality that you're working
in cost-center IT. Or maybe a kind of inner migration from an enterprise code
base which you can't identify with, and wish to put a fence against, as in
"they" (your lib users) vs "us" (elite metaprogrogrammers). As someone else
said, today's Spring-heavy Java code bases express behaviour through "anything
and everything except actual Java code". C++ developers should take a look at
Spring MVC in particular to see if that really is what they want, where the
relative sequence of method parameters has significance, and when presented
slightly different will result in service request routing 404ing, which you
find out only by examining 600-items stack traces with multiple reflection
pits.

~~~
jcelerier
> C++ developers should take a look at Spring MVC in particular to see if that
> really is what they want,

I don't understand why it would be bad to make it impossible to have Spring-
like frameworks in C++. A metric ton of useful things have been written in it,
and are only written much more painstakingly in C++.

~~~
tannhaeuser
The obvious answer would be: why don't use Java/Spring then? Does C++ have to
be everything to everybody (though that ship has probably sailed some 30 years
ago)?

~~~
jcelerier
> The obvious answer would be: why don't use Java/Spring then? Does C++ have
> to be everything to everybody (though that ship has probably sailed some 30
> years ago)?

I really prefer writing C++ code where :

\- I have the choice of the programming style for every subproblem of my
software than Java code where most of the time the only choice is new-riddled,
OOPish BS. Writing a Java visitor or observer pattern once again makes me
shiver from dread when I'm used to std::variant and Qt's signal / slots. I'll
admit that Scala mostly solves that though, if I really had to develop on the
JVM that's likely the only language that I'd happily use. No type-level
programming -> not relevant for me, given how many metric tons of bugs this
has saved me so far.

And integrating JVM code with C++ (or any kind of native) code is an exercice
in pain - I've had the displeasure to wrap one of the libraries I've developed
through JNA to make it accessible to Processing, wouldn't wish that on my
enemies.

\- Things can be made to happen deterministically and automatically with RAII,
I still have nightmares of trying to get finalizers to work in C# for instance
to release resources other than memory at deterministic times and not "some
time away in the future".

~~~
tannhaeuser
Ok I can get that, though it's not that much of a problem with "finally" code
blocks and modern idiomatic Java/try-with-resources. But (and I'm not
pretending to be an expert here) I think attempting to write generic
multithreaded service-oriented backends in a non-GCd language is going to give
you a hard time with memory fragmentation (even more so with async/evented
code), plus the performance, for all I know, isn't really all that great.

------
nickysielicki
This is where C++ is headed anyway. There's just a little bit more work that's
needed in c++23, but it's planned, and that will make it possible to use a lot
more of the STL in consteval/constexpr contexts.

I think this is a killer feature. Between execution policies and constexpr-
all-the-things, I really think C++ is set to pull away in a way that other
languages will have difficulty competing with.

~~~
closeparen
How does this differ from code generation?

~~~
ori_b
It doesn't generate artifacts that you can examine and debug.

~~~
logicchains
Circle seems to have compile-time printf, which would make it easier to debug
than normal C++ compile-time code.

------
msoloviev
This looks like a more mature/different approach to the same thing I tried to
do in a side project a while ago
([https://github.com/blackhole89/macros](https://github.com/blackhole89/macros)),
down to similar aesthetic choices. However, I pitch my project as a macro
system rather than compile-time execution.

While here the metalanguage is itself a C++-lookalike, in my project I wound
up with something that is perhaps better described as a weird TeX (which seems
to be what happens naturally when your machine model is based on binding and
substitution).

On one hand, C++ is surely more powerful/expressive and there is elegance to
having the language and metalanguage use the same idiom. On the other, I feel
like this might actually making Circle a bit more confusing to use at times,
as it becomes less clear exactly when what parts of the code are executed, and
you could even imagine a typo accidentally lifting a part of the code from
runtime to compile time, resulting in mystifying bugs. (I am myself puzzled by
the "serialize structs generically" example: does it imply that the template
is specialised before the @meta for is executed?)

------
thechao
Now add 'operator ;', 'operator if(...)', 'operator .', et al and we've got
ourselves a Lisp!

~~~
Reelin
Personally I'll just use ECL until CLASP gets its optimizations sorted out.

------
symstym
This seems similar to Zig ([https://ziglang.org/#Compile-time-reflection-and-
compile-tim...](https://ziglang.org/#Compile-time-reflection-and-compile-time-
code-execution))

~~~
pickdenis
Zig seems to have arbitrary compile-time code evaluation, but not the kind of
AST generation you see here[1]. Nim macros seem to be a closer analogue[2].

[1]:
[https://github.com/seanbaxter/circle/blob/master/gems/rpn.md...](https://github.com/seanbaxter/circle/blob/master/gems/rpn.md#lower-
the-rpn-ast)

[2]: [https://nim-lang.org/docs/macros.html](https://nim-
lang.org/docs/macros.html)

~~~
logicchains
There's ongoing work to add support for generating arbitary types at compile
time:
[https://github.com/ziglang/zig/issues/383](https://github.com/ziglang/zig/issues/383).

------
yrashk
Love the level and amount of documentation/tutorials provided by the author!

Also, wondering when and if the source code of Circle itself will be
available.

~~~
yzh
Sean is a very good engineer. Another of his work is moderngpu, a GPU
primitive library. It has the same level of doc/tutorials. I really like one
thing he wrote: Software is an asset, code a liability. On good days I'd add
200 or 300 lines to this repository. On great days I'd subtract 500.

------
beshrkayali
One of the reasons to look into Nim.

Nim has generics, templates and macors. Macros are procedural, and they make
things like generating a dsl for your software or library a simple matter.

All with a reasonably simple pythonic syntax.

------
rurban
The syntax choices are for one much better than the C++ committee ideas. Much
less eyebleed.

~~~
sesuximo
Really? I think fold statements are really nice. So often you get what you
want by just writing `...`

------
kotrunga
Regardless of how great the stuff in the repo is (I'll check it out)... this
is amazing:

"it's like c++ that went away to train with the league of shadows and came
back 15 years later and became batman"

~~~
smt88
I'm going to disagree here. It reminds me of the "Python Ninja"-type job
titles that became popular ~2012. It's so preoccupied with standing out and
breaking the formality of its medium that it actually says nothing.

The repo description literally says two things: Circle is a C++ automation
language, and Circle trained with the League of Shadows. The rest requires
clicks.

That's a terrible way to drive adoption for something that obviously was
difficult to build. Sometimes being earnest and straightforward is the best
way to break down skepticism, not by using meaningless hipster taglines.

~~~
rogerclark
there's about a zillion pages of detailed documentation on top of that

~~~
smt88
The intro text is not detailed enough to tell me whether I want to click on
any of those.

How hard is it to write a one-sentence explanation of what a project is that
doesn't involve ninjas?

~~~
asdf_snar
From the abstract,

"Circle is a compiler that extends C++17 for new introspection, reflection and
compile-time execution. An integrated interpreter allows the developer to
execute any function during source translation. Compile-time flavors of
control flow statements assist in generating software programmatically. The
configuration-oriented programming paradigm captures some of the conveniences
of dynamic languages in a statically-typed environment, helping separate a
programmer's high-level goals from the low-level code and allowing teams to
more effectively reason about their software."

While I appreciate your qualm, respectfully I think that no matter how it were
introduced, someone would've quibbled. "You should be straightforward", "you
should be more detailed", "you should have more ninjas". There's no win. The
solution is to hope the readers you care about will be adventurous enough to
make two clicks.

------
einpoklum
What I miss in C++ isn't more ability to _run_ things at compile-time, but
rather more ability to _compile_ things at run-time!

I want to finalize my optimizations with run-time-available information - and
to make run-time-definable compositions of pieces of code.

Something like this: [https://blog.quarkslab.com/easyjit-just-in-time-
compilation-...](https://blog.quarkslab.com/easyjit-just-in-time-compilation-
for-c.html)

but better still.

------
mckinney
Reminds me of Manifold[1] for Java. But Manifold integrates seamlessly with
the Java compiler as a plugin as opposed to being a separate compiler, which
makes Circle less approachable imo. Still, I like the F# type provider-ish
aspect of this.

[1]: [https://github.com/manifold-
systems/manifold](https://github.com/manifold-systems/manifold)

~~~
tpoindex
Thanks for pointing out Manifold, looks delicious.

As a historical note, PL/I had a preprocessor that itself was a subset of
PL/I. I don't recall any type safety features, but was quite useful for
creating DSLs within a program.

------
gerash
I'm all for doing things at compile time rather than run-time but both the
syntax and debugability of CPP template metaprogramming suck.

I wonder if C++ should pull a Python 3 and clean up the syntax. Given that the
transition for Py3 took more than a decade maybe they should do it in small
but frequent language upgrades instead.

~~~
adrianN
C and C++ live on backwards compatibility. There are many decades old
codebases that are still being worked on. You can't clean up the syntax if
that breaks backwards compatibility, you'd just end up creating a new language
instead.

~~~
gerash
IMO too much backwards compatibility hinders progress. It's obviously not
sustainable to be forever backwards compatible all the way to the 70s. I'd say
old code bases can simply stick with the older compiler.

As for whether it'd be considered a new language or not, it depends. I
wouldn't consider Py3 a new language even though it's not backwards
compatible.

~~~
sesuximo
General software tip: don't make a new version if you'll have to maintain the
old and the new. It gets annoying real quick. much easier to tell your users
to update and be reasonable with the changes

I actually don't see a problem keeping the old syntax alive. I do see a
problem with new programmers using obsolete constructs

------
umvi
I've just started storing metadata in JSON format. Then, I have a cmake target
that invokes a python program that reads in the JSON and generates .h/.cpp
files which can then be consumed by other targets.

The advantage of this is that I can then use the same metadata JSON to
generate documentation, or whatever I want.

~~~
reikonomusha
Have you learned about Lisp macros yet? If you’re intrigued by this idea, Lisp
macros are a few steps above in terms of functionality, integration, etc.

~~~
umvi
Are you talking about using Lisp macros to generate C/C++/documentation? I'm
not sure how Lisp macros would be better than static metadata files being
operated on by some high level scripting language.

~~~
reikonomusha
The author created a DSL which generates code. Macros are another way to do
that.

------
pizlonator
This is so wonderfully excellent. I can't wait to use it.

------
jcalabro
Reminds me of the Jai language, in development now:
[https://www.youtube.com/watch?v=TH9VCN6UkyQ&list=PLmV5I2fxai...](https://www.youtube.com/watch?v=TH9VCN6UkyQ&list=PLmV5I2fxaiCKfxMBrNsU1kgKJXD3PkyxO)

------
Zardoz84
I would keep using D ...

------
jhallenworld
Verilog and VHDL have something akin to this form of imperative meta-
programming. (See Verilog's "generate" and "genvar" keywords for example).

But there is no iterating over or building members of an interface (the
closest thing SystemVerilog has to a user defined type).

------
fredsanford
Is it named Circle because you'll be running in a circle trying to figure out
the error messages?

~~~
jzelinskie
Jokes like this are common now, but, as an outsider, I think it's crazy that
the C++ community says stuff like this as if they weren't dealing with
template errors and slowly making everything work under constexpr.

------
halayli
I am not sure if it's possible to implement this as pluggable pass to llvm,
but it would widen adoption.

~~~
jhj
The backend is LLVM. Sean wrote his own C++17 compatible frontend to provide
this functionality (thus, Circle is effectively a replacement for clang).

------
torb-xyz
There's a pretty interesting podcast interview with the author here:
[https://cppcast.com/circle-language/](https://cppcast.com/circle-language/)

I find the idea of using this as a rapid testing and exploration of potential
new C++ features quite interesting.

------
clord
The @include thing makes me wonder if a circle library could compete with
modules.

~~~
orbifold
I think the author mentioned that he plans to support C++-modules as is. He
already supports concepts.

------
dexter0
This looks amazing, but the fact that it is implemented in an entirely new
compiler (atop LLVM, sure) is a major turn-off. Would be more useful if it
were implemented as a meta compiler instead.

~~~
orbifold
There is basically no way to modify clang or GCC to support executing
arbitrary C++ code at compile time. Something like that would be a fork that
wouldn’t get accepted upstream. According to the author the compiler front end
is ~100kLoc which is an order of magnitude less than what clang is.

------
spekcular
"Your scientists were so preoccupied with whether or not they could, they
didn’t stop to think if they should."

~~~
banachtarski
Circle is honestly a breath of fresh air and much more palatable in terms of
usability/speed than the existing constexpr facilities or proposed reflection
IMO.

------
kbenson
It makes my eyes bleed, but so does C++ by itself to a slight degree. I
imagine those that like C++ (or are forced to use it) might find it very
interesting.

I wonder how this compares to hygienic macros that some languages support?

~~~
FpUser
I use C++ by choice for some projects. Still I think C++ is enough beast on
its own. No need to add another preprocessor

------
slezyr
template<typename... types_t>

struct ??? {

    
    
      @meta for(int i = 0; i < sizeof...(types_t); ++i)
    
        types_t...[i] @(i); 
    

};

Oh no.

------
amiga_500
One can only imagine the error messages from the c++ layer after a semi-
botched circle compile!

~~~
lasagnaphil
Actually the error messages are much nicer than you've thought. Here's an
example:

    
    
        $ circle taco4.cxx -I /home/sean/projects/opensource/taco/include -M /home/sean/projects/opensource/taco/Debug/lib/libtaco.so  -filetype=ll
        Creating compute_1 function
        macro expansion: taco4.cxx:428:3
        error in expansion of macro auto call_taco(const char*, const options_t&)
          code injection: taco4.cxx:420:6
          error thrown in @expression("c")
            error: @expression:1:1
            undeclared identifier 'c'
    
    

A well-defined meta language would probably have way better error messages
than an accidentally-discovered language based on insane template hacks.

