It's all well and good to list sensible sounding principles "for meaningful comparison of metaheuristics", but I was kind of expecting to see those fine principles applied in meaningful examples.
lots of the "popular metaheuristics" as described won't be anything approaching state of the art, and i doubt their test bed considers problem instances that are hard or of industrial relevance. so the paper sort of seems to be about a method to compare the performance of uninteresting algorithms on uninteresting problems, without doing any comparison.
tangentially, in contrast, here's something from a vendor of commercial discrete optimisation software (from 2016):
> Pasco Shikishima, a Japanese leading food company (¥165 billion, equivalently $1.6 billion, in annual revenue), has chosen LocalSolver to solve what is probably the largest real-life optimization problem in the world.
> MIP solvers needed the original problem to be decomposed into almost twenty subproblems, each one solved separately, then leading to a poor global solution
> Pasco's supply chain involves 15 factories in Japan, each one with several production lines, and more than 100 distribution centers. Pasco's catalog contains more than 1,000 products. 900,000 orders have to be executed each day in Pasco's factories. For each order, Pasco has to decide where and when to produce it. Moreover, Pasco has to decide where to source raw materials and which routes to deliver distribution centers. The goal is to minimize production and distribution costs over several days of horizon, while respecting production and distribution capacities. Here is the scale of a LocalSolver instance solved by Pasco to plan the next 3 days: 32,670,717 expressions (that is, intermediate variables) including 8,307,431 binary decisions, 991,251 constraints, 16 lexicographic-ordered objectives. This model was solved in 3 minutes of running time on a modern but standard server, including input and output processing times.
this probably would have been a pretty fun project to work on, given that there was already an incumbent optimisation system in production, so a lot of the required prep-work and IT infrastructure for being able to gather and clean the input data for planning would have been in place. a sub-percentage improvement in optimality of the solution at this scale might translate into tens of millions of dollars of net profit for the company.