
In 2003, the Pentagon nearly created a terrorism prediction market. - zacharyvoase
http://money.cnn.com/2003/07/29/news/terror_futures/
======
aothman
My research involves making electronic agents to mediate interactions in
markets, with one application being to prediction markets. One of the things
I've discovered is that prediction markets where prices influence events are
significantly more complicated from a theoretical perspective than a standard
market. In such markets it can be difficult to predict how rational agents
will act, even if they care only about their risk-neutral monetary payout and
not about the events in question.

For instance, one of the new trends in prediction markets has been corporate
elicitation (e.g., Inkling Markets YC W06). For instance, a market might ask a
pool of employees "When will product X ship?". Now consider the employee's
problem. He knows the product will be delayed until November at the current
pace, but that if managers see high November probabilities they'll cut back on
the project and re-focus development efforts (or, perhaps instead, shift some
staffers over to get the project done faster). How should the employee trade
in this market?

If your answer is "it's unclear" then you're exactly right. The self-
referentiality of prices (they cause actions which then define prices) makes
the problem much harder. It's actually possible to design markets which are so
self-referential that _any_ trade will be the right one (the market prices are
self-predicting prophecies) or that _any_ trade will be the wrong one (e.g.,
if there's a high price on terrorist attack Z then the government will always
spend enough money to prevent Z from happening).

In fact, the only way around the paradox is to have the prices in a prediction
market not matter at all - for the managers to take the same course of action
regardless of the prices they see in the market. But then why run a market in
the first place?

The whole thing is an interesting and seemingly fundamental perversion of the
way we normally think about prices. And again, this is without any malicious
intent on the part of participants, which will certainly only further confuse
things.

~~~
T_S_
Well said. Economic theory show that the price system is a highly efficient
way of trading allocations of _scarce_ resources under perfect information.
Funny thing about information, any scarcity is largely artificial, especially
with today's technology. Also, if it is not shared, markets tend to disappear.

~~~
jacques_chester
That's a fairly sweeping generalisation of economic theory. Hayek, for
example, explained that markets work for allocating scarce resources because
they cause actors to share their incomplete information with each other
through price signals.

Without a market, information is disjointed; everybody has some imperfect,
incomplete part of the whole picture. Any planner will make irrational plans,
he or she is dealing with a minute subset of information.

~~~
T_S_
I was describing an equilibrium model like Debreu's (very sweeping by the
way), where information is not really explicit. Hayek's opinion about the
hopelessness of central planning is more about the role of markets in
eliminating the computational burden of central planners and is very
important. But his was primarily a point about the hopelessness of central
planning.

When the market is for information _itself_ , I'm not sure if Hayek would have
much to say. Markets for such things seem like they should be far more
fragile. I think stock market volatility in the recent crisis is actually
evidence of that.

Incidentally, and ironically, all this technology at our disposal giving us
more "freedom", has made the feasibility of central planning more possible.
Dys/U-toptian novelists take note.

~~~
jacques_chester
Thanks for the followup.

I am not an economist of Hayek expert, but my understanding is that he address
"markets as information" as being markets. Even gambling markets and futures
markets, which are both classes of prediction markets generally, are actually
information markets even though they are ostensibly about goods or events.

I agree and disagree about your characterisation of technology. While yes,
computers make authoritarianism more tractable, I don't think they can
completely solve the calculation problem.

Suppose you use Leontiev input-output matrices to do your central planning.
Matrix multiplication is an O(n^3) problem (I've been told by a mathematician
that it can be done in n^2.75 with some tricks). If we take GDP growth as a
proxy for the number of matrices required, it would need to be kept below a
few percent or it would outstrip Moore's law in the long run. And that's
ignoring the problem of populating the matrices in the first place.

FWIW, I host an economics blogger in Australia, Dr Nicholas Gruen, who has
proposed to reduce market volatility by requiring blind clearances on a once-
per-minute basis.

~~~
T_S_
I like Gruen's idea. Try this one on: We could have size investors
continuously publish their postions. A 10000 investor x 10000 securities
matrix would be more than enough. Price vol would drop like a stone.

------
Symmetry
Sort of a shame how that all panned out. I was on a mailing list where Robin
Hanson was talking about the upcoming Pentagon prediction market a month or so
before the media got wind of it. I was very skeptical at first, but he did a
very good job of meeting people's objections - mostly by explaining details of
how it would be implemented.

Then the media got hold of the story and people raised all the objections I
had heard before, but the explanations couldn't fit into single sentence sound
bites so they never made it into the news.

~~~
protomyth
Sound bites beat reason quite a lot of the time. This is one of those projects
I would feel good about, but wish they had never told the public. Keep it to
security / terrorist experts and maybe some fiction writers, history profs /
writers, and cultural anthropology profs.

~~~
Herring
"A bad design with a good presentation is doomed eventually. A good design
with a bad presentation is doomed immediately."

------
dminor
Haha, amused to see this pop up on HN. I was the lead programmer on it.

Nothing quite makes you feel quite so connected to the national legislative
process as seeing your project roundly condemned on the floor of Congress ;)

~~~
jamespitts
I was working at the Hollywood Stock Exchange at the time and was completely
perplexed by the "betting parlor" hyperbole used by the opposing congressmen.
It seemed to me that the actual players of the game would be much more
interested in the status of making the right call or in creating content than
any money they might earn.

If you are able to comment on this, what were the underlying pricing mechanics
of the market? How were new securities going to be created? Heh, to think that
simulated stock market software might be classified...

~~~
dminor
We used Robin Hanson's logarithmic market scoring rule
(<http://hanson.gmu.edu/mktscore.pdf>). The combinatorial aspect allowed us to
try to answer questions about the correlation between different securities in
the market - hopefully more interesting for policy analysis than a simple
predictor of the future.

The main securities in the market were 5 statistics that we were tracking
quarterly in 8 different Mid-East countries (things like civil stability,
military activity, US aid, etc). Then there were supposed to be a handful of
one-off type events; we hadn't decided on them yet, but gave some "ripped from
the headlines" examples that were the main focus of the criticism. The values
of all these securities were to be judged by the Economist Intelligence Unit.

------
adam
The government is now running prediction markets in several different branches
(I know this because they're using our software - Inkling) and the TIA project
comes up all the time in initial discussions. The comments however are usually
that the idea was a good one and it was producing valuable information, they
were just very tone deaf to perception issues.

~~~
jedc
And for anyone reading, Inkling is an early YC company. W06, if I remember
right. (Hi, Adam!)

------
eli
Obviously it creates a problem with incentives, but I thought it was a pretty
cool idea. If you could get the volume high enough I bet it would be really
informative.

------
RyanMcGreal
I remember that. I actually put up a parody about it on my old personal web
site:

<http://www3.sympatico.ca/taylormcgreal/lottery.html>

------
eli
Decent article from James Surowiecki defending the program:
<http://www.slate.com/id/2086427/>

------
zacharyvoase
Other relevant links:

* <http://en.wikipedia.org/wiki/Policy_Analysis_Market>

* [http://en.wikipedia.org/wiki/Total_Information_Awareness#Fut...](http://en.wikipedia.org/wiki/Total_Information_Awareness#Futures_Markets_Applied_to_Prediction_.28FutureMAP.29)

~~~
jdp23
Poindexter and others pushing Total Information Awareness did a horrible job
of marketing. The logo even had an eye in a pyramid, all the better to feed
conspiracy theorists' concerns about the Illuminati! So in a rare victory for
civil libertarians that Congress wound up defunding TIA.

Many of the individual projects continued forward or have since resurfaced.
They just have better logos and acronyms now.

------
jsulak
Talk about potential moral hazard...

~~~
eli
You are correct, but for what's it worth the market was intended to be for
security experts. Not just anyone could place a bet.

~~~
JonnieCache
_> market was intended to be for security experts_

Oh. Phew. Now I know that only seasoned banking/finance professionals are
involved I feel completely safe! Those guys never do any crazy shit!

EDIT: do you mean security experts or securities experts?

~~~
T_S_
>do you mean security experts or securities experts?

Yes.

------
jmount
Even if it were informative it might me more useful for terrorists than
defenders. For example if the market correctly valued hard to fix
vulnerabilities then all the market does is help terrorists pick targets (as
the market is one big analyst leak and asymmetric warfare is by definition
easier for one side).

~~~
hexis
The same information that would help terrorists pick targets would also help
the good guys prioritize which targets to protect better/first. Public
information helps the good guys at least as much as the bad guys.

~~~
jmount
Asymmetric costs, asymmetric benefits. The market would just leak a bunch of
info as different analysts show off.

------
zzzmarcus
There's a great book on this, and related Pentagon initiatives called The
Watchers:

[http://www.amazon.com/Watchers-Rise-Americas-Surveillance-
St...](http://www.amazon.com/Watchers-Rise-Americas-Surveillance-
State/dp/1594202451)

------
JonnieCache
If we're in the market for predictions, my personal prophecy is thus:

DARPA, in an effort to offset the money wasted on this research, will pivot
and attempt to sell this idea (and its obvious potential pitfalls) to the
writers of 24 for the next series.

~~~
endtime
Given that 24 has ended, I'm going to short your prediction.

~~~
JonnieCache
Gimme your email, with this kind of blue-sky financial thinking we should
start a hedge fund.

------
xiaoma
I can only imagine how a bubble in such a market might "pop".

------
guelo
My objection to this is that terrorist attacks are extremely rare events and
there is not that much data about them so trading on this market would be very
boring and you would almost always lose the boom bet. Anyway, anti-terrorism
efforts seem like misplaced priorities, if you really want to save the lives
of American you're better off working on ways to reduce car crashes or ways to
stop people from eating cheeseburgers.

~~~
gwern
The markets were going to be about more long-term trends and political events
- the sort of things analysts usually grapple with.

This too is one of the common under-informed criticisms.

~~~
guelo
TFA: "Under the plan, traders would buy and sell futures contracts on specific
events such as terrorist attacks, based on whether they believed they would
happen. "

How do you make a bet on a long-term trend? What kind of long-term trends?
Political events?

~~~
gwern
> "In fact, however, PAM was not intended to forecast the details of terrorist
> attacks. It was instead intended to forecast aggregate measures of
> geopolitical stability in the Middle East. PAM would have used speculative
> markets to estimate economic growth, political stability, and military
> activity four times a year in each of eight nations, and how those measures
> would depend on each other and on various U.S. policy choices. PAMs
> designers thought their plan ambitious enough without also tackling the
> added complexities of predicting terrorism."

\-- <http://hanson.gmu.edu/realterf.pdf>

(Hm, how _would_ one estimate things like economic growth? That sounds really
hard and my imagination just fails to think of any way to measure a long-term
trend like that!)

~~~
gwern
Another relevant quote:

> "Our focused later narrowed to a smaller region, the Mideast, because the
> Economist Intelligence Unit charged a high price to judge after the fact
> what instability had actually occurred in each nation. The final plan was to
> cover eight nations. For each nation in each quarter of a year (over the two
> year final phase), traders would estimate five parameters: its military
> activity, political instability, economic growth, US military activity, and
> US financial involvement. In addition traders would predict US GDP, world
> trade, US military casualties, and western terrorist casualties, and a few
> miscellaneous items, to be determined by traders and the EIU. This would
> require (8 × 8 × 5 =) a hundred or so base markets. > > In addition, we
> planned to let traders predict combinations of these events, such has how
> moving US troops out of Saudi Arabia would effect political stability there,
> how that would effect stability in neighboring nations, and how all that
> might change oil prices. Similar trades could have predicted the local and
> global consequences of invading Iraq, had such markets been ready then.
> (More on this in the later section on combinatorics section.)"

------
Intrade
www.intrade.com has been listing geo-political markets such as markets on the
fall of Iran, capture of Saddam, Bin Laden and others and markets on WMD since
Y2K.

------
gwalker
The amazing free market, the duct tape of neo-liberalism.

It's kind of a cool idea but seriously?

~~~
gwern
> "An incredulous stare is not an argument."

\--philosopher David Lewis (1973)

------
Duff
You don't like this? What are you, some sort of communist?

The US Government has already sold the livelihood of most of its people (ie.
industry) under the guise of the free market. Talk to someone who grew up in
Buffalo, Cleveland or any number of places in Michigan and they'll tell you
all about it.

Letting people make money off of betting on you getting blown up is just a
more offensive way of getting rich on the backs of others.

~~~
bennylope
This is an unhelpful comment.

The Policy Analysis Market was designed to allow registered traders (it was
going to start as a market open only to select analysts, experts) to make
money-backed predictions. That is, putting your money where you mouth is. It's
a tool to filter noise and focus analysis, not fuel hedge funds.

~~~
watchandwait
Duff's ignorant anger is why the program was killed.

~~~
bennylope
In a nutshell, yes. And the earlier comment about DARPA's marketing is spot
on. Their poor response did not temper the response from Capitol Hill.

