
Struct Iteration Through Abuse of the C Preprocessor - ncraun
https://natecraun.net/articles/struct-iteration-through-abuse-of-the-c-preprocessor.html
======
kevingadd
You can do some interesting things using __COUNTER__ to make code using this
less ugly. And if you're willing to use a small amount of C++ templates, you
can make it possible to resolve all the iteration/member info at compile time
to eliminate runtime overhead.

I did both of these things for Grim Fandango for vertex/uniform buffers, so
that the game could use buffers without caring about whether it was using
OpenGL or console APIs - but it's not open source :-(

Even if you don't need something like this under normal circumstances,
consider whether it could let you identify mistakes in debug builds, or avoid
errors when writing your serialization code.

~~~
Kristine1975
I'm doing something similar in C++ for vertex attributes, integration with Lua
et.al. First I define the struct:

    
    
      INTROSPECTION_DEFINE_STRUCT(S,
        (int, anInt),
        (bool, aBool),
        (float, aFloat),
        (double, aDouble),
        (std::string, aString)
      );
    

Then I create an instance of it:

    
    
      S s{ 23, true, 37.5f, 42.25, "String" };
    

And finally I can e.g. iterate over all fields and output their value using a
generic lambda:

    
    
      forEachField(s, [](auto v) {
        std::cout << v;
      });
    

There are also definitions for the field's type, offset, name etc. generated
by the macro, so it's easy to create a binding for vertex attributes, map the
struct to a table in Lua, serialize/deserialize it and so on.

I'm really hoping C++ gets introspection soon. There are some proposals like
[https://isocpp.org/files/papers/n3996.pdf](https://isocpp.org/files/papers/n3996.pdf)
and a working group at
[https://groups.google.com/a/isocpp.org/forum/#!forum/reflect...](https://groups.google.com/a/isocpp.org/forum/#!forum/reflection)
but I'm not holding my breath for 2017.

~~~
dman
Is the code for this available somewhere?

~~~
Kristine1975
Not at the moment, no. But it's not rocket science. The macro essentially
creates a struct like this one:

    
    
      struct S {
        // How many fields are in this struct?
        static constexpr unsigned int FIELD_COUNT = 5;
    
        // Full specializations are not allowed inside classes, so add a dummy parameter <T> that is equivalent to S
        template <unsigned int Index, typename T = S>
        struct FieldTraits;
    
        // Partial specialization for the first field
        template <typename T>
        struct FieldTraits<0, T> {
          using type = int;
          static constexpr std::size_t OFFSET = offsetof(S, anInt);
          static constexpr const char* name() { return "anInt"; }
          static constexpr type& value(S& self) { return self.anInt; }
        };
        // The first field itself
        int anInt;
    
        // Partial specialization for the second field
        template <typename T>
        struct FieldTraits<1, T> {
          using type = bool;
          static constexpr std::size_t OFFSET = offsetof(S, aBool);
          static constexpr const char* name() { return "aBool"; }
          static constexpr type& value(S& self) { return self.aBool; }
        };
        // The second field itself
        bool aBool;
        ...
      };
    

Now that we have the meta-information about the struct's fields, we can
process them:

    
    
      // Meta-function to process a struct's fields
      template <typename T, unsigned int FieldIndex, unsigned int FieldCount>
      struct ForEach {
         template <typename Functor>
         static void apply(T& object, Functor f)
         {
            // Get the field's value and call the functor with it
            auto value = T::FieldTraits<FieldIndex>::value(object);
            f(value);
    
            // Next field
            ForEach<T, FieldIndex + 1, FieldCount>::apply(object, f);
         };
      };
    
      // Partial specialization to end iteration
      template <typename T, unsigned int FieldCount>
      struct ForEach<T, FieldCount, FieldCount> {
         template <typename Functor>
         static void apply(T& object, Functor f)
         {
         };
      };
    

And finally a bit of syntactic sugar:

    
    
      template <typename T, typename Functor>
      void forEachField(T& object, Functor f)
      {
         ForEach<T, 0, T::FIELD_COUNT>::apply(object, f);
      }
    
    

*(omitted for the sake of clarity: dealing with const objects, using rvalue references and std::forward for the functor to make the code work with non-copyable functors)

~~~
dman
Thanks for taking the time to write this!

------
asteadman
Wow, I just finished doing something similar. The use of x-macros is
definitely a little cleaner / canonical than my approach, which instead
"recurses" through a VA_MACRO. For an excellent guide on recursion in macros,
see this article on how to abuse the C pre-processor:
[https://github.com/pfultz2/Cloak/wiki/C-Preprocessor-
tricks,...](https://github.com/pfultz2/Cloak/wiki/C-Preprocessor-
tricks,-tips,-and-idioms).

In the end I end up defining structs like this: SERIALIZER_STRUCT( structname,
int8_t, field1, int16_t, field2, ... );

Initializing them like this: structname instance = {.field1 = 1, .field2=2};

and serializing them like this: SERIALIZER_SERIALIZE( structname, &instance,
bigbuff_ptr );

As for just using a packed structure, there are definitely caveats with some
compilers (ie: compilers for embedded systems may not handle various edge-
cases for misaligned access, particularly when dereferencing pointers to
structs.)

------
evmar
Taking a step back: what you're trying to accomplish is code generation -- a
program that generates code. From that perspective, why would you choose to
write that program using the C preprocessor, when there are so many other nice
languages around?

~~~
ncraun
In actual production use, I think it could be better to write a separate
program that will generate the code for you instead of using the C
preprocessor (or use a language with this kind of introspection built in).
However, using this hack lets you depend only on the facilities of the
language, and that can be useful in simplifying the build process. Plus it was
fun to create, too!

~~~
uxcn
It can be a bit tricky if it isn't integrated with the build process.

~~~
Kristine1975
You can use e.g. cog which puts the generated code into the file itself:
[http://nedbatchelder.com/code/cog/](http://nedbatchelder.com/code/cog/) Then
all you need to do is to remember to run the preprocessor after you've made a
change to the generator code.

~~~
uxcn
Python is definitely a better syntax to write meta-programs than the pre-
processor, but you still have the issue of the additional step/tool. Anything
that depends on the generated code depends on the step which, if it's missed,
can lead to non-good things.

~~~
Kristine1975
Sure. But you _do_ test your changes, don't you? ;-) So if you forget to run
cog after making a change to the meta-program, you should notice right away.

But it still is a cludge. It would be better if C/C++ had a better
preprocessor.

~~~
uxcn
It isn't a question of testing changes. The more people working on the
codebase, the more people it impacts. Adding manual steps is tempting the
fates.

------
Kristine1975
That (define macros, include file that uses them) is fairly vanilla usage of
the C preprocessor. Abuse is something like this:
[https://github.com/rofl0r/chaos-pp](https://github.com/rofl0r/chaos-pp) and
[https://github.com/rofl0r/order-pp](https://github.com/rofl0r/order-pp)

------
comex
A bit cleaner setup, in my opinion, makes the "x macro" an argument to the
user-defined macro, rather than depending on #include. For example, to define
an enum along with a string representation function for it:

    
    
        #define DEFINE_ENUM(name, variants) \
          enum name { \
            variants(DEFINE_ENUM_VARIANT_ITSELF) \
          }; \
          const char *repr_##name(enum name e) { \
            switch (e) { \
              variants(DEFINE_ENUM_VARIANT_REPR_CASE) \
              default: return "?"; \
            } \
          }
        #define DEFINE_ENUM_VARIANT_ITSELF(variant) variant,
        #define DEFINE_ENUM_VARIANT_REPR_CASE(variant) \
          case variant: return #variant;
    
        // client code:
    
        #define foo_variants(x) \
          x(FOO_1) \
          x(FOO_2)
        DEFINE_ENUM(foo, foo_variants)

------
luckydude
While it is true that compilers pad for alignment it is pretty easy to avoid
that, at least in my experience.

So an easier approach, but more fragile, is just to define the struct, count
up how big you think it should be and then do an

assert(sizeof(struct foo) == FOO_SIZE);

If that assert doesn't pop then you can just do

write(fd, &foo, FOO_SIZE);

I know that is a lot less elegant but it works and isn't doing the translation
back and forth.

If you don't like that you might consider using Sun's RPC marshalling code
that is (used to be?) part of Linux.

~~~
luckydude
Yeah, in poking around it looks like RPC is still a thing and it's solved this
problem in an automated way.

[http://docs.freebsd.org/44doc/psd/23.rpc/paper.pdf](http://docs.freebsd.org/44doc/psd/23.rpc/paper.pdf)

[http://docs.freebsd.org/44doc/psd/22.rpcgen/paper.pdf](http://docs.freebsd.org/44doc/psd/22.rpcgen/paper.pdf)

------
Kristine1975
C preprocessor "abuse" to create reflective enums:
[http://www.codeproject.com/Articles/1002895/Clean-
Reflective...](http://www.codeproject.com/Articles/1002895/Clean-Reflective-
Enums-Enum-to-String-with-Nice-Sy)

------
Mellowcandle
why not using packed structure compiler feature?

~~~
WesternStar
I had the same question #pragma pack works just fine.

~~~
blt
Maybe wants the default alignment for performance. Plus that still leaves
endian issues

