
To Explain or to Predict? (2010) [pdf] - tacon
http://www.galitshmueli.com/system/files/Stat%20Science%20published.pdf
======
WhompingWindows
It's a blurred philosophical line between prediction and explanation. Do you
truly need explanatory/casual inference embedded in a predictive model
somewhere? What about after the predictive model is run and you attempt to
implement interventions to prevent or encourage the predicted outcomes?
Surely, some causality will creep in at some point, unless it's a purely
mechanical/mathematical slicing and no researchers/coders are attempting to
contextualize the predictions within some useful framework.

For me, there is a distinction practically in the workflow. For prediction,
you take many variables and throw them all in to a model attempting to predict
a certain outcome of choice, which you then use with domain knowledge to
affect change. In contrast, for explanation/causation, you start by carefully
constructing a model using a priori evidence and intuition, which then gives
you some idea about associations between the carefully selected variables.

------
eksemplar
I find it extremely interesting tha finance is highlighted as a place where
predictive methods are used, because almost all scientific research on
financial predictions show that they don’t actually work.

World wide 22 million people work with financial analytics, but the work they
do, empirically fails to beat a set of monkeys throwing darts at a wall.

Typically when people are presented with this they defend the industry with
things like the weather services, which is actually one of the very few fields
that are proven to do predictions right, 9 in 10 times it’ll rain on a day the
weatherman tells you.

I mention weather, because it’s strangely missing from the paper.

What value does predictive models have, when they don’t work? And why is the
paper looking at sectors, that may be using predictive models yes, but are
also sectors that provable fail at predictions?

~~~
OscarCunningham
Financial predictions absolutely do work. The price of a stock is very well
correlated with its future returns. So they're good at predicting how well
businesses will do. What they can't do is predict the future stock price
movement. But this isn't a failure of financial prediction, it's just an
inherent property of efficient markets.

If you opened a betting market on the weather then meteorologists would do no
better than monkeys at predicting its price movements.

~~~
eksemplar
There isn’t any empirical evidence that they work though. People obviously
think it has value, otherwise 22 million people wouldn’t be employed in the
business, but any scientific research done on the field has unquestionably
shown that financial predictions don’t actually work.

It’s interesting that you mention betting as a counter argument, considering
that you’re listening to financial advisors exactly to avoid betting.

Investments is still one of the few places with a positive net gain, unlike
betting, so you should absolutely invest. Just know that as far as scientific
evidence goes, you’re not better off listening to a financial advisor than you
would be by throwing darts at a wall.

Whether predictions, again, is actually the field of predictions where we’re
provable correct most of the time, compared to any other field of prediction.

So you’re actually incorrect. If it was possible to bet on the weather
forecast, it would be the surest way to make money in the world.

I mean, the fact that you actually _can’t_ bet on whether in a world where you
can bet on pretty much everything else, should really tell you everything you
need to know.

~~~
OscarCunningham
> There isn’t any empirical evidence that they work though.

What I'm saying is that traders are good at predicting the absolute rate of
return of investments but bad at predicting their return relative to their
price.

Companies that the traders have valued at $1B actually do make 1000 times as
much on average as the companies they have valued at $1M. This is the analogue
of meteorologists being good at predicting the weather. But if a single trader
says that a company will do better than the rest of the market thinks, then
that trader will do no better than chance. Similarly if a meteorologist thinks
that rain is more likely than the rest of the meteorologists, then they'll be
right no more often than chance.

Asking meteorologists to predict rain but traders to predict _relative_ price
movements is an unfair comparison. Either compare the meteorologists'
predictions of rain with the market's prediction of absolute profits, or
compare the meteorologists' predictions of how much their predictions will
change with the markets relative predictions of profits.

> If it was possible to bet on the weather forecast, it would be the surest
> way to make money in the world.

No, because the person you were betting against would only bet at odds
matching the best meteorologists' predictions.

~~~
eksemplar
You can repeat yourself, but you’re arguing against scientific evidence.

------
mannykannot
As I have though that, in science, causal explanation is the ultimate goal,
and the ability to predict is the gold standard measure of a theory's ability
to explain, I was surprised by the statement "predictive modeling is nearly
absent in many scientific fields as a tool for developing theory." To me, it
also seems odd that explanatory modeling is distinct from predictive modeling,
though I gather that the use of 'explanatory' here is a term of art that does
not necessarily mean what it does in general usage.

This is from 2010, so perhaps attitudes are changing now that concerns about
reproducibility have come to the fore.

