To properly use your analogy, if I first saw your data, then "predicted" that "a = 1.1, b = 2.1, d = 3.99, e = 5.01, f = 3.9999" and claimed that I had "predicted" your data with 90% (or whatever) accuracy, that is substantially misrepresenting what I have done.
However, I would be convinced by a strong correlation (i.e. the model is not overfitted), coupled with a plausible model.
That is a problem. That should not be enough to convince you. It is enough to make it an avenue worth exploring.
I don't think so. I think my analogy was that given the data, if I can create a model that accurately predicts the data given, and is plausibly simple, i.e. for X=Y, Y=ordinal(X), then I can have some faith in the model.
To put it another way, if Gottman said:
Add up these 5000 variables multiplied by these factors to get the score
then I wouldn't believe it. But I think he said something much closer to (on p. 13 of the 98 paper):
We tried a couple of combinations of variables, that we had hypotheses for. One combination gives a very accurate predictor (Table 1, Husband high negativity). Two others are also accurate (Table 1, low negativity by husband or wife.)
I think where the paper went wrong, and what the Slate article is criticising is the claim made in the second column of p. 16, where they do seem to put all the variables into a model and pull the answer from nowhere. So, I'll give you that the 82% prediction is not justifiable. But negativity as a predictor for divorce seems well supported.
---
If I drop 30 monkeys out of different height trees and I measure 100 things about each monkey, each second, as it falls, and then I give you a model that purports to tell you a monkey's speed from those 100 measurements, then you won't be surprised to received an over-fitted piece of junk.
If instead I give you speed=constant * t^2, would I need to drop another 30 monkeys?
This is one of those things where the size of the result set let's you easily over fit data. Take 50 people born in 1970 and you could make a fairly simple model based on their birth day to predict if they were living or dead because you only need 50 bits of information and 10 of them can be wrong. There is even a strong bias where most people born in that year are still alive.
Edit: As a simple rule of thumb compare the complexity of the formula with the number of accurate bits in your output compared to the most simple model possible. If the formula is anywhere near as complex as your results it’s probably over fitting the data.
However, I would be convinced by a strong correlation (i.e. the model is not overfitted), coupled with a plausible model.
That is a problem. That should not be enough to convince you. It is enough to make it an avenue worth exploring.