Basically, none of these agentic / MoE / etc papers have actually compared their results to the naive baseline: since these are nondeterministic programs, Randomized Algortihms 101 tells you that if the probability of success is sufficiently high, you can improve performance simply by running the algorithm multiple times and taking the majority/plurality result.
So is MoE or agents actually more effective than doing it the dumb way? AI Snake Oil says "no." Truly bizarre that dozens of researchers didn't even ask! It made me feel like I was missing something.
Basically, none of these agentic / MoE / etc papers have actually compared their results to the naive baseline: since these are nondeterministic programs, Randomized Algortihms 101 tells you that if the probability of success is sufficiently high, you can improve performance simply by running the algorithm multiple times and taking the majority/plurality result.
So is MoE or agents actually more effective than doing it the dumb way? AI Snake Oil says "no." Truly bizarre that dozens of researchers didn't even ask! It made me feel like I was missing something.