"simplicity is a guide to truth" is not a summary of Occam's Razor. It says that _all things being equal_ the simplest explanation is generally the best. If you have an elegant solution that doesn't account for everything, there's a problem[1]. That problem is either in the theory or in the data, but until that problem is resolved, you're not helping advance much of anything.
So, for instance, we can still (I am told by credible astrophysicists) posit a geocentric cosmology if we're willing to adapt a rather complex system. But rather than think this is true, we have an alternative hypothesis that accounts for the same data without introducing a plethora of adjustments.
But of course, gravity _alone_ doesn't count for the data, hence Einstein.
Gravity is a "simpler" explanation, but it doesn't _account_ for our observations, so it get axed.
Scientists doing philosophy without training is almost always a very scary thing to witness.
[1] Re: we don't make curves that fit every data point: those curves are not -predictive- (i.e. you can perfectly account for -known- data, but you blow when you try to predict -future- data) and hence don't provide a reliable account of phenomenon. So they're rejected on that basis.
So given an infinite set of possible theories to explain a data set, which theory should be preferred? Simple accuracy over the known data is frequently not sufficient, as you point out, because of overfitting. Given noisy data (which is almost always a given), you can't just use maximal accuracy as the sole criteria for which theory to prefer. One way forward is to combine accuracy with some measure of the complexity of the theory (e.g. minimum description length), which has much of the flavor of Occam's Razor.
Interesting article and perhaps more interesting comments after it.
I wanted to add one point. The focus of the article is on the utility using Ockham's Razor for finding correct (or predictive) models of the world. A second reason to use Ockham's Razor is that simpler theories can have more utility because they are easier to use and understand. Often a simpler theory can be more useful even if it is less correct.
The reason to prefer simplicity is that fewer bits of information are required to store a simple theory. This makes simple theories easier to use correctly (since there are fewer bits which can be subject to transcription errors, for example). So among equally predictive theories, the simplest one has important advantages.
If simplicity is also aligned with some bias in the natural world, that's purely a bonus. You might expect that to be the case if simple mechanisms were more probable than complex ones, and simple mechanisms were more likely to lead to simple behavior. So maybe there's something to that idea, but maybe not -- this author seems to know a lot more about it than I do.
I always thought this was completely obvious, but most people tend to miss it. It's not about accuracy at all. It basically says unnecessary assumptions are unnecessary.
So, for instance, we can still (I am told by credible astrophysicists) posit a geocentric cosmology if we're willing to adapt a rather complex system. But rather than think this is true, we have an alternative hypothesis that accounts for the same data without introducing a plethora of adjustments.
But of course, gravity _alone_ doesn't count for the data, hence Einstein.
Gravity is a "simpler" explanation, but it doesn't _account_ for our observations, so it get axed.
Scientists doing philosophy without training is almost always a very scary thing to witness.
[1] Re: we don't make curves that fit every data point: those curves are not -predictive- (i.e. you can perfectly account for -known- data, but you blow when you try to predict -future- data) and hence don't provide a reliable account of phenomenon. So they're rejected on that basis.