Hacker News new | past | comments | ask | show | jobs | submit login
To Explain or to Predict? (2010) [pdf] (galitshmueli.com)
61 points by tacon 9 months ago | hide | past | web | favorite | 13 comments

It's a blurred philosophical line between prediction and explanation. Do you truly need explanatory/casual inference embedded in a predictive model somewhere? What about after the predictive model is run and you attempt to implement interventions to prevent or encourage the predicted outcomes? Surely, some causality will creep in at some point, unless it's a purely mechanical/mathematical slicing and no researchers/coders are attempting to contextualize the predictions within some useful framework.

For me, there is a distinction practically in the workflow. For prediction, you take many variables and throw them all in to a model attempting to predict a certain outcome of choice, which you then use with domain knowledge to affect change. In contrast, for explanation/causation, you start by carefully constructing a model using a priori evidence and intuition, which then gives you some idea about associations between the carefully selected variables.

I find it extremely interesting tha finance is highlighted as a place where predictive methods are used, because almost all scientific research on financial predictions show that they don’t actually work.

World wide 22 million people work with financial analytics, but the work they do, empirically fails to beat a set of monkeys throwing darts at a wall.

Typically when people are presented with this they defend the industry with things like the weather services, which is actually one of the very few fields that are proven to do predictions right, 9 in 10 times it’ll rain on a day the weatherman tells you.

I mention weather, because it’s strangely missing from the paper.

What value does predictive models have, when they don’t work? And why is the paper looking at sectors, that may be using predictive models yes, but are also sectors that provable fail at predictions?

Financial predictions absolutely do work. The price of a stock is very well correlated with its future returns. So they're good at predicting how well businesses will do. What they can't do is predict the future stock price movement. But this isn't a failure of financial prediction, it's just an inherent property of efficient markets.

If you opened a betting market on the weather then meteorologists would do no better than monkeys at predicting its price movements.

There isn’t any empirical evidence that they work though. People obviously think it has value, otherwise 22 million people wouldn’t be employed in the business, but any scientific research done on the field has unquestionably shown that financial predictions don’t actually work.

It’s interesting that you mention betting as a counter argument, considering that you’re listening to financial advisors exactly to avoid betting.

Investments is still one of the few places with a positive net gain, unlike betting, so you should absolutely invest. Just know that as far as scientific evidence goes, you’re not better off listening to a financial advisor than you would be by throwing darts at a wall.

Whether predictions, again, is actually the field of predictions where we’re provable correct most of the time, compared to any other field of prediction.

So you’re actually incorrect. If it was possible to bet on the weather forecast, it would be the surest way to make money in the world.

I mean, the fact that you actually can’t bet on whether in a world where you can bet on pretty much everything else, should really tell you everything you need to know.

> There isn’t any empirical evidence that they work though.

What I'm saying is that traders are good at predicting the absolute rate of return of investments but bad at predicting their return relative to their price.

Companies that the traders have valued at $1B actually do make 1000 times as much on average as the companies they have valued at $1M. This is the analogue of meteorologists being good at predicting the weather. But if a single trader says that a company will do better than the rest of the market thinks, then that trader will do no better than chance. Similarly if a meteorologist thinks that rain is more likely than the rest of the meteorologists, then they'll be right no more often than chance.

Asking meteorologists to predict rain but traders to predict relative price movements is an unfair comparison. Either compare the meteorologists' predictions of rain with the market's prediction of absolute profits, or compare the meteorologists' predictions of how much their predictions will change with the markets relative predictions of profits.

> If it was possible to bet on the weather forecast, it would be the surest way to make money in the world.

No, because the person you were betting against would only bet at odds matching the best meteorologists' predictions.

You can repeat yourself, but you’re arguing against scientific evidence.

There is definitely a market for betting on weather. One of the principal components of changes in natural gas and power prices is unexpected changes in weather.

It's an adverse selection problem. There absolutely are predictive models in finance that work.

The people who make them will never publish about or describe them in detail, as they're too busy making billions of dollars on them.


The people who publish their financial predictions or models are publishing them rather than betting money on them (or are trying to fulfill the prediction through the process of publishing), which tells you something about how confident they actually are in their predictions.

A lot of the research was ordered by the companies that are making billions of dollars.

The monkey research was done by Wall Street itself, though it wasn’t actually monkeys but employees throwing the darts.

The dart throwing portfolios beat 60-67% of the other investor portfolios for years.

Seems like any dynamic system that studies itself and copies or imitates past success, roughly has this problem.

just to pick two, I think sports and politics both have this problem.

where someone finds a new method and has success, everyone tries to copy it,

only everyone trying to copy it bids up all the inputs, with higher inputs, the method has a different ROI.

In dynamic systems, there's a curve you have to stay ahead of

The carriage horse of financial industry that makes consistent gains is portfolio management. All other predictive nonsense are a bunch of experiments built upon that solid foundation.

It's obviously an exaggerated statement that "all" financial predictions don't actually work, dude.

As I have though that, in science, causal explanation is the ultimate goal, and the ability to predict is the gold standard measure of a theory's ability to explain, I was surprised by the statement "predictive modeling is nearly absent in many scientific fields as a tool for developing theory." To me, it also seems odd that explanatory modeling is distinct from predictive modeling, though I gather that the use of 'explanatory' here is a term of art that does not necessarily mean what it does in general usage.

This is from 2010, so perhaps attitudes are changing now that concerns about reproducibility have come to the fore.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact