> I don’t think people understand the point sutton was making; he’s saying that general, simple systems that get better with scale tend to outperform hand engineered systems that don’t
This is your reading of Sutton. When I read his original post, I don't extract this level of nuance. The very fact that he calls it a "lesson" rather than something else, such as a "tendency", suggests Sutton may not hold the idea lightly*. In other words, it might have become more than a testable theory; it might have become a narrative.
* Holding an idea lightly is usually good thing in my book. Very few ideas are foundational.
This is your reading of Sutton. When I read his original post, I don't extract this level of nuance. The very fact that he calls it a "lesson" rather than something else, such as a "tendency", suggests Sutton may not hold the idea lightly*. In other words, it might have become more than a testable theory; it might have become a narrative.
* Holding an idea lightly is usually good thing in my book. Very few ideas are foundational.