Another interesting pattern is the ability to generate structs at compile time.
Ive ran experiments where a neural net is implemented by creating a json file from pytorch, reading it in using @embedFile, and generating the subsequent a struct with a specific “run” method.
This in theory allows the compiler to optimize the neural network directly (I havent proven a great benefit from this though). Also the whole network lived on the stack, which is means not having any dynamic allocation (not sure if this is good?).
I've done this sort of thing by writing a code generator in python instead of using comptime. I'm not confident that comptime zig is particularly fast, and I don't want to run the json parser that generates the struct all the time.
Another thing I tried as an alternative is using ZON (zig object notation) instead of json. This can natively be included directly as a source file. It involved writing a custom python exporter though (read: I gave up).
FWIW the goal for comptime Zig execution is to be at least as fast as Python. I can’t find it now but I remember Andrew saying this in one of his talks at some point.
I think if you integrated with the build system, yes, Zig can do things only when the file changed. But I'm not sure that Zig figured out incremental comptime yet. That's way harder to accomplish.
They become quite long, but it was surprisingly tolerable. I recall it vaguely but a 100MB neural network was on the order of minutes with all optimizations turned on. I guess it would be fair to say it scaled more or less linearly with the file size (from what I saw). Moreover I work in essentially a tinyml field so my neural networks are on the order of 1 to 2 MB for the most part. For me it wouldve been reasonable!
I guess in theory you could compile once into a static library and just link that into a main program. Also there will be incremental compilation in zig I believe, maybe that helps? Not sure on the details there.
Ive ran experiments where a neural net is implemented by creating a json file from pytorch, reading it in using @embedFile, and generating the subsequent a struct with a specific “run” method.
This in theory allows the compiler to optimize the neural network directly (I havent proven a great benefit from this though). Also the whole network lived on the stack, which is means not having any dynamic allocation (not sure if this is good?).