I took an undergrad course in evolutionary optimization, and became briefly excited about it, so I'm fairly familiar with the ideas in that area. I think that similarly to neural networks, they don't really have a great mathematical basis to say what will work and what won't -- it's a lot of empirical experimentation. Do they work? Yeah, kinda. There's really few other options when you have to deal with large, combinatorial spaces such as neural network architectures. I do think a lot of research in the metaheuristics area, at least a few years ago (I haven't really kept up with it) is pretty bogus -- I lampooned it in a couple of "papers" (http://oneweirdkerneltrick.com/spectral.pdf and http://oneweirdkerneltrick.com/catbasis.pdf). Yes, all the citations are real.
 Bayesian optimization is great, though I find it amusing that people who wouldn't touch a genetic or swarm algorithm are totally fine with BO when it's really not that different.
PS I have to admit, your papers made me laugh :)
Your papers are fantastic.
Runtime compared to genetic architecture search (using similar settings):
The error rate on CIFAR-10, before the final training (meaning that topologies weren't fully trained and no augmentation was used):
The error rate on CIFAR-10, before the final training compared to genetic architecture search (using similar settings):
The 2 main factors that contribute to faster search are (1) ants search for architectures progressively (meaning that early architectures can be evaluated really fast), (2) ants can reuse the weights as they are associated with the graph.
All test were done using Google Colab. Even though results might not seem that impressive, I am still really excited to see what will happen when ants will be allowed to search for more complex architectures which use multi-branching.
Also, I wanted to ask: were you using Gooogle Vision, as when I was doing the research it seemed that they do not allow you to export the model.