
The stock market as a single, very big piece of multithreaded software - olefoo
http://arstechnica.com/business/news/2010/10/you-say-stock-market-i-say-ginormous-multithreaded-application.ars?comments=1#comments-bar
======
yummyfajitas
It is hard to agree with his conclusion that the market is unstable. The flash
crash was a huge exogenous shock of unknown (at the time) origin and the
market self corrected in a few minutes. The market did such a good job that
the author did not even notice the crash until a specialist pointed it out.

If only all multithreaded programs behaved that well.

[edit: added parenthetical "(at the time)", thanks chr15.]

~~~
chr15
If people are actually interested in what caused the flash crash, it was a
sale of $4.1 billion of E-Mini contracts by a single fund.

Report of SEC finding [PDF]:
<http://www.sec.gov/news/studies/2010/marketevents-report.pdf>

~~~
nomurrcy
This really isn't what caused the flash crash. The SEC either doesn't have a
complete understanding of what happened, or is omitting swaths of data from
their report. Nanex did a much better job writing this all up, and weeks
before the SEC published their report.

<http://www.nanex.net/FlashCrashFinal/FlashCrashSummary.html>

~~~
nomurrcy
To further clarify:

75,000 e-mini contracts is not an astronomical amount. Orders of this size are
not uncommon.

To really understand what happened one must answer the question: what are the
market conditions we have created such that a normal sized order can cause
such a dislocation?

A lot of this has to do with the practices of the HFT 'liquidity providers.'
This is a complex topic, but the flash crash demonstrated what a lot of people
have been saying for years: volume != liquidity, the HFT shops generate
volume, not liquidity and engage in some practices (quote-stuffing, sub-
pennying etc) which serve to systemically destabilize the markets. (and
overload the market infrastructure)

~~~
vecter
Just a minor point, but I've never seen a 75,000 lot rip through ES in one
order.

~~~
nomurrcy
No body just dumps all 75,000 in one transaction. The order is filled over a
period of time. W/R didn't dump either, actually further analysis shows that
they sold the majority of their contracts _after_ the crash

[http://www.zerohedge.com/article/refuting-secs-lies-core-
fla...](http://www.zerohedge.com/article/refuting-secs-lies-core-flash-crash-
analysis)

The blog here is zerohedge, but the data is from nanex.

------
contextfree
I do tend to think of financial markets as a giant computer whose function is
to allocate capital in the economy, and which just happens to use a really
weird architecture/computational model that incorporates human brains as
computational nodes, sort of like The Matrix if they hadn't gone with the
goofy energy-extraction motivation instead.

It's probably an analogy with limited applicability, but I do think it might
be interesting to examine the computational properties of financial markets,
try to see to what extent arbitrary programs can be encoded in them, etc.

~~~
yummyfajitas
It's actually an analogy with great applicability. Quite a bit of work has
been done on this topic - see the work of Pennock at Yahoo. One result:
completely efficient prediction market is a general purpose computer. Such an
object also requires a ridiculous number of market participants, since it can
solve NP complete problems.

<http://dpennock.com/publications.html>

One very popular result (major media attention) along these lines is this
paper, which proves that pricing a certain type of CDO is an NP-complete
problem:

<http://www.cs.princeton.edu/~rongge/derivative.pdf>

(Side note: this paper really pisses me off because they did not cite ANY of
the prediction market literature. It's hard to believe this was not on
purpose.)

------
rgarcia
_So when the HFTs got hold of the futures contracts that the Waddell & Reed
algorithm was selling, they would've first looked for legit buyers to flip
them too. And, not finding any legit buyers, they just started rapidly buying
and selling amongst themselves to at least pick up some rebate money._

A couple of things:

1\. E-mini futures don't give rebates

2\. He seems to suggest that different algos from different companies were
knowingly trying to collude with each other to get these nonexistent rebates.
This seems pretty far-fetched.

~~~
gxti
And even if there were rebates (as there are on stocks) you can't make a
profit just passing them back and forth because taking liquidity costs more
than is rebated to the offerer. If liquidity hot potato were profitable then
that would be the sole purpose of the stock market until the exchanges ran out
of money, because it'd be coming straight out of their pockets.

~~~
ScottBurson
This must be true. But then, what _did_ happen? Why were there 27,000 trades
in 14 seconds (IIRC)?

~~~
gxti
I'm not saying it didn't happen, just that it's not something that a
"rational" algorithm would do, because the one that's taking liquidity is
losing money. Unless they're profiting for reasons other than rebates, that
is, which is another small but nonzero possibility.

My favorite theory is a purely technical one. I don't have much supporting
evidence beyond the linked site though:

"""In summary, quotes from NYSE began to queue, but because they were time
stamped after exiting the queue, the delay was undetectable to systems
processing those quotes. The delay was small enough to cause the NYSE bid to
be just slightly higher than the lowest offer price from competing exchanges.
This caused sell order flow to route to NYSE -- thus removing any buying power
that existed on other exchanges. When these sell orders arrived at NYSE, the
actual bid price was lower because new lower quotes were still waiting to exit
a queue for dissemination."""

Original report: <http://www.nanex.net/20100506/FlashCrashAnalysis_Intro.html>
More news from the same company:
<http://www.nanex.net/FlashCrash/FlashCrashAnalysis.html>

In addition to the crash itself, they also have interesting charts from
various algorithms behaving strangely.

------
sz
Humanity as a single, very big piece of multithreaded software:

Minds are processes, reality is shared memory. Crashes happen under suitably
bad conditions (e.g. WWII).

I don't see how this per se offers any real insight. You can apply the analogy
to almost anything.

~~~
dpapathanasiou
Well, Douglas Adams thought of it, too.

In HHGG, the Earth is just a massive computer for calculating the question to
the answer of life the universe and everything.

All the biological components are merely instances and sub-processes spawned
by the main program.

------
davidw
I think a bunch of message-passing Erlang processes seems like a more apt
description. Any one of them can crash, and others will take over, if things
are set up properly. Indeed, after a half an hour, things were mostly sorted
out.

------
thraxil
Were you actually intending to link directly to the comments section rather
than to the top of the article? Am I the only one that finds that annoying?

------
gte910h
With this idea as "stock market as multithreaded program", I find it ironic
then that many derivatives now have to specify their function in python...

------
grandalf
Multithreaded? Maybe, but then exchanges are mutexes.

------
fleitz
Not to be pedantic but it's more like a piece of functional message passing
software with many agents each with their own immutable state (the final
record is the ticker tape and it's generally immutable).

