
The power of prediction markets - Hooke
http://www.nature.com/news/the-power-of-prediction-markets-1.20820
======
RyanCavanaugh
> Prediction markets have also had some high-profile misfires, however — such
> as giving the odds of a Brexit ‘stay’ vote as 85% on the day of the
> referendum

I'm continuously annoyed by this. The prediction was 85%, not 100%. I'm
guessing if you looked at all the 85% predictions, about 15% of the time the
other outcome happened. That's what 85% means, after all.

You can't just round all predictions to the nearest integer and call it a
"misfire" when the unlikely events happen. 1 in 7 events happen all the time!
The misfire would be if 85% events _always_ happened.

~~~
wyldfire
I agree with your logic but perhaps they consider the magnitude of the actual
Brexit referendum vote to refute the prediction 'stay' likelihood of 85%. I
don't know how close the election was (IIRC it was fairly close), I'm just
stating that many people might consider a landslide in the other direction to
mean that the prediction was flawed.

OTOH maybe people misinterpreted the 'stay' likelihood itself as the ratio of
the predicted popular stay vote and that would explain it too.

~~~
HSO
It was _very_ close, something like 51:49. The predictions circulating before
the vote may have influenced the result. Apparently more Remainers would have
bothered to vote if the predictions had not been so clearly in favor of
Remain. It's a tragedy, really, that something so important would be decided
on so flippantly.

This, by the way, is another weakness (or limitation) of prediction markets.
They should probably not be applied to matters where the prediction interacts
with the "predictee" lest you enter a "Keynesian beauty contest".

~~~
hueving
Or more of the exiters would have voted if they weren't discouraged by being
told it was a big loss. That sword cuts both ways.

~~~
verbify
Biased sample here, but I spoke to multiple people with 'bregret' \- they told
me they only voted for brexit because they thought it wouldn't happen and
wanted to stick it to the man. The stupidity of these people boggles my mind,
but there you have it.

And nobody predicted a big loss - it was going to be close either way.

------
hardtke
One place I'd love to try prediction markets is with product changes
undergoing A/B tests. A product manager could submit an experiment and people
at the company could bet on what the product change will do to
engagement/revenue/growth metrics. This framework could eventually be used to
weed out product changes before they are built. It would be particularly
useful when the product change is driven by the leadership and most of the
rank and file think the direction is wrong.

~~~
n00b101
Enterprise Prediction Markets have been a thing for a long time. There are
existing enterprise software vendors [1], it is known that Google has an
internal prediction market [2], and there is a wealth of academic literature
on corporate prediction markets [3]

[1] [https://www.crowdworx.com/prediction-market-software-
consult...](https://www.crowdworx.com/prediction-market-software-consulting-
companies/)

[2] [https://www.quora.com/What-does-Google-use-prediction-
market...](https://www.quora.com/What-does-Google-use-prediction-markets-for-
internally)

[3]
[http://marginalrevolution.com/marginalrevolution/2015/10/cor...](http://marginalrevolution.com/marginalrevolution/2015/10/corporate-
prediction-markets-work-well.html)

~~~
jedc
Google _had_ an internal prediction market. It hasn't operated for many, many
years.

------
catwell
The "guess the number of jelly beans in a jar" game gave its name to one of
the startups whose failure disappointed me the most, Beansight. In 2010, it
was asking people their opinion on various topics, attributing them experience
points when they guessed right, and using this information about the expertise
of its users to improve its predictions.

~~~
n00b101
Sounds similar to [http://www.almanis.com](http://www.almanis.com)

~~~
catwell
Pretty similar yes. I didn't know that one, thank you.

------
n00b101
Here is a US election prediction market:
[http://tippie.biz.uiowa.edu/iem/](http://tippie.biz.uiowa.edu/iem/)

The "Winner Takes All" market predicts Trump vs Clinton presidency:
[https://iemweb.biz.uiowa.edu/graphs/graph_Pres16_WTA.cfm](https://iemweb.biz.uiowa.edu/graphs/graph_Pres16_WTA.cfm)

According to this, after last night's performance, Trump has an 8% probability
of becoming the 45th President of the United States of America, while Clinton
has a 92% probability.

~~~
hueving
How big is the pool of participants in that particular market? It's not that
interesting if it's only 20 or so participants.

~~~
zbyte64
Couldn't find the number of participants but over 100,000 shares have been
issued:
[https://iemweb.biz.uiowa.edu/WebEx/marketinfo_english.cfm?Ma...](https://iemweb.biz.uiowa.edu/WebEx/marketinfo_english.cfm?Market_ID=362)

------
joelthelion
One very good prediction market is Hypermind [1]:

* It's not an open panel. You can apply for an account, but you will be screened. This removes the noise from people who just want to try out prediction markets.

* Contestants don't play for real money. I believe this improves predictions by removing biases due to having real financial stakes in the outcomes.

I also like the fact that it's a real prediction market based on trading and
not one of those systems where you have to update your predictions all the
time. In a prediction market, you just trade when you see a potential benefit,
and the power of the market does the rest.

The only issue I have with it is that the site is so damn slow...

[1]
[https://hypermind.lumenogic.com/hypermind/app.html#welcome](https://hypermind.lumenogic.com/hypermind/app.html#welcome)

(Edited the link. Whoever made this site is really completely clueless about
web programming...)

~~~
chillydawg
Without real money, it's useless imo. At least with something like betfair,
one vote = £1 so you only vote big when you know something, which moves the
market and incorporates your opinion the appropriate amount.

~~~
joelthelion
Quite the contrary, actually. Real money markets give way too much importance
to clueless gamblers. Also experience shows that the prospect of losing your
rank is a sufficient deterrent to keep people from making large dubious bets.

------
rojobuffalo
I think I heard Nick Bostrom mention the idea of using prediction markets in
the legislature, and it sounded like a good idea.

~~~
panic
Wouldn't that affect how people trade? If you can spend money to influence
policy, it might be worth making the "wrong bet" and losing money.

~~~
Houshalter
If you knowingly make a wrong bet, then other people can exploit it and pump
money out of you.

But yes, I think any realistic implementation would have to be tightly
controlled. With the traders being whitelisted and audited after large bets
(to test for conflict of interests), make their bets public, and force them to
give reasons for their predictions, etc.

Personally I like the idea of not using a market at all. Have individuals bet
with play money, or just assign odds directly and skip betting. Then you can
find a set of individuals that are very good at making predictions, and
average their predictions together. This is basically what they did in the
book "superforecasters" and the CIA now uses something like this. They found
it was superior to prediction markets (which themselves were much superior to
predictions made by 'experts' with access to classified information.)

Tough players in prediction market will eventually start techniques like this,
if they work. But it shows that just the incentive of betting alone isn't
enough to get the most accurate predictions from people. You really need to
filter for the people who are surprisingly good at it.

------
dreamdu5t
It's worth pointing out that X% of people betting an event will happen is not
the same thing as an event having X% chance of happening. The former is an
estimate of occurrence and the latter is a measure of the occurrence. This is
a common conflation with statistics I hear when people talk about predictions.

~~~
drdeca
> X% of people betting an event will happen

You may be well aware of this, but in case you aren't, that isn't the value
that prediction markets give.

The value that prediction markets give is based on the odds people are willing
to bet at, not the proportion of people willing to bet in what direction.

If any risk neutral (in terms of currency) individual believes that their
predictions are better than the prediction market, and the transaction costs
and time costs are low enough, etc., it should be rational for them to
participate in it. And if they are better at predicting, the prediction market
prediction should improve.

So, yes, prediction markets might not always make the best possible
predictions, but if anyone claims to be better than them, it opens the
question of why they aren't participating.

So, if I don't have any special knowledge about the possibility of something,
and think that a substantial amount of people know better than I do, I
generally expect the market's prediction to be better than mine.

(Assuming the people better informed than me make up a reasonably large part
of the market, which I think they usually do.)

~~~
nhaliday
Interestingly enough, even idealized prediction markets don't produce accurate
probability estimates, but generally overweight bad events like collapse of
the market itself: [https://ordinaryideas.wordpress.com/2011/12/16/risk-
arbitrag...](https://ordinaryideas.wordpress.com/2011/12/16/risk-arbitrage/).

