This is unfortunate. It doesn't seem like any participant here is doing anything wrong in isolation-- Intel is running each core at the highest frequency it can to avoid meltdown, and the compiler is just using the instructions that are available-- but the net effect of them together is suboptimal.
I wonder if this can be solved by overprovisioning cooling for the chips. Does the dynamic frequency scaling kick in based on core temperature? Otherwise you might have to just do experiments to see how many cores to use to maximize the aggregate throughput.
It is based on the percentage of AVX instructions being executed, and kicks in when the percentage surpasses a predetermined threshold. It's different for every CPU generation I believe.
I wonder if this can be solved by overprovisioning cooling for the chips. Does the dynamic frequency scaling kick in based on core temperature? Otherwise you might have to just do experiments to see how many cores to use to maximize the aggregate throughput.