
A veteran programmer explains how the stock market became “rigged” - xmpir
http://www.washingtonpost.com/blogs/wonkblog/wp/2014/04/04/a-veteran-programmer-explains-how-the-stock-market-became-rigged/
======
nanexllc
Here's an animation showing the SIP vs Direct Feed issue:
[http://www.nanex.net/aqck2/4599.html](http://www.nanex.net/aqck2/4599.html)

Much of the games being played no would no longer work if the SIP was the
defacto source of "real-time" prices (as specified in Reg NMS). Sure, there
would be new games, but we have rules specifying the SIP as the key to Reg
NMS. At least there would be a decent audit trail if rules were being
followed. People would be able to detect delays (currently obscured by the
changing time stamp) and many games would be exposed.

It's helpful to actually read Reg NMS - a link is provided on that animation
page.

P.S. I'm the veteran programmer in that article.

~~~
hft_throwaway
Can you describe in detail what "games" this enables and how often they
actually happen in the data? You claim SIP slowness enables a fictional trade
called "latency arb" where HFTs can trade with orders at stale prices, for
example:

-A user is pegged to mid at a pool where they think NBBO is bid: 20.01 ask: 20.02

-An HFT with direct feeds sees the market go to 20.02 bid 20.03 ask

-The HFT buys the midpoint order on the dark pool at 20.015

-User regrets trading since they could have sold 20.02

How often does this really happen and how is it really any different from the
following scenario:

-A user is pegged to mid at a pool where they think NBBO is bid: 20.01 ask: 20.02

-An HFT with direct feeds sees the market as 20.01 bid for 100000 shares and 20.02 ask for 1 share.

-The HFT predicts the price will move up with near certainty and buys the order for 20.015

-User regrets trading since they could have joined small 20.02 ask and traded quickly

If the user or broker is pegging their order to the midpoint and not adjusting
it based on market conditions, they're going to trade at disadvantageous times
no matter what. I'm not sure how pricing off the SIP makes a big difference.
It's not HFT's fault that the broker sucks and can't trade well.

------
yummyfajitas
This is not a case of the markets being rigged. This is a case of regulators
(actually congress) failing to understand the CAP theorem.

But for obvious reasons, an article on WAPO can't use words like "partition
tolerance".

The problem is that the SIP must be available and it must be consistent. As a
result it sacrifices partition tolerance - high latency (which is equivalent
to a network partition) causes things to break sometimes.

So for all the conspiracy theorists out there, here is the fundamental
question: which two should be provided for the markets not to be "rigged"?

~~~
crdoconnor
>So for all the conspiracy theorists out there, here is the fundamental
question: which two should be provided for the markets not to be "rigged"?

There is no need to make any complicated tradeoff. For it to count as non-
rigged, you simply need a system where:

* You cannot pay to gain privileged access.

* Abuse of the system (e.g. quote stuffing) gets you punished.

I find it amusing that you call us 'conspiracy theorists'. These aren't
allegations of a government cover-up of alien UFOs - it's mundane criminal
behavior with a very minor twist.

~~~
yummyfajitas
So everyone needs to pay several thousand/month for direct access? I don't
know why it's bad that people who don't need low latency are allowed to not
pay for it.

"Quote Stuffing" is not the nefarious activity that Nanex portrays it as. It's
generally the result of your algorithm screwing up. It provides no benefit to
the trader and exposes him to risk - specifically, an HFT who is quote
stuffing will always be at the _bottom_ of the book. As a result, he won't get
any retail fills, he'll only trade with informed traders who are buying up the
whole book. It also lowers your fill rate, which can result in a trading halt
or financial penalties.

As a result, most HFTs put effort into reducing quotes and making their
algorithms more stable.

I shouldn't have used the term "conspiracy theorists". I just find this topic
frustrating since there are a large number of comments that display a complete
misunderstanding of basic market mechanics and a complete unwillingness to be
specific. No shortage of moral posturing, however.

~~~
dvogel
One of the core complaints about HFT regulation is that the regulators refuse
to update their definitions to include HFT activities that follow the letter
of the regulation but clearly violate the spirit of the regulation. Therefore
arguing that HFT detractors are not relying on precise definitions does more
to show that you're not really listening to their argument but rather just
finding something to nitpick. It looks like an avoidance strategy, not an
honest defense.

~~~
yummyfajitas
First off, my quibbles about the definition of front running vs predatory
trading are unrelated to my defense. The defense I gave was about price
discrimination. It doesn't matter what label you apply to the definition - if
you want to argue against me then say why price discrimination is bad. Or say
I'm wrong about price discrimination.

Second, the "spirit of the regulation" about front running is about allowing
people to trust their agents. Other regulations in the same spirit are lawyer-
client confidentiality and "financial advisers can't take gifts from companies
they give advice on" regulations.

There are no "you can't kick people's ass in the stock market" regulations, in
letter or spirit. If you play the stock market everyone is trying to take your
money. You are trying to take theirs.

------
cpwright
The exchanges do send the information to the SIP at the same time that they
publish it on their own direct feed. NYSE actually got in trouble for not
doing this properly. This isn't actually an easy problem, because there is a
TCP interconnect with the SIP, and if it gets bogged down, then the SIP
packets can be backed up.

The real issue is that since the SIP (CQS/UQDF) are cheap; and
big/sophisticated players don't actually use them, no one cares about them.
Beyond that the performance is necessarily slower than a direct feed, because
if I wanted to learn the price of a NYSE listed-security on NASDAQ the path
that the SIP would take is far less direct. Specifically, if I have a machine
sitting in Carteret (Nasdaq), listening to the proprietary Nasdaq feed, the
data from Nasdaq will be sent to me and the SIP at the same time. But to get
it from the SIP it has to go to Mahwah and then back, instead of staying local
to the data center.

Nasdaq makes its feed available to everyone, for a cost. Someone who is
providing execution services, or market making, needs the direct feeds so they
can provide good execution and know when to get out of the way. A small
investor looking at a screen doesn't care about the SIP or a direct feed. They
are going to submit their order, and their broker is responsible for executing
it.

~~~
jzwinck
What do you think would happen if direct feeds were prohibited and
participants had to listen to the consolidated feeds only? This could even
include getting one's own fills, if there were a consolidated feed with order
IDs included (i.e. you would not know your fills any earlier than anyone
else).

~~~
hft_throwaway
Then HFTs would still win but pricing would widen out since some liquidity is
made available because of ability to hedge in another product. Think about if
I'm quoting SPY because I can buy/sell the basket better than my quotes. If
the feeds are delayed, I'm less confident in where the basket price really is,
so I'm going to demand more compensation in bid-ask spread to put up a price.
This wouldn't be a big deal in slow markets but when things really started
moving, most derivative liquidity would be pulled entirely.

In single names HFTs mostly use these feeds to protect themselves from other
high-speed traders and they want the granularity of information for their
models. They're already faster than retail guys and banks and always will be.
It's not an "unfair advantage" like people think. They just have to get on it
if any competitor gets on it, similar to all the fast lines. If 20 HFT shops
get on a new fast line, none of them are making more money from it.

------
fleitz
I think there should be a new kind of logical fallacy: appeal to grandpa.

Today's world is not the same as your grandfathers world, we use computers to
do many that they are better at than humans, executing trading stragies in
submicrosecond latencies is one of them.

Submitting an order to a market causes the price to change, submitting a large
order means that you are likely trading against an informed trader, by making
a bid they are disseminating information about price to the market, if you
don't adapt to the new information (by raising your price) you are an missing
a huge opportunity.

In an ideal market to trade a large block you'd pay a risk premium less than
the price of disseminating that information over a longer period and possibly
paying more as supply of the stock at a given price evaporated.

Everyone can read the order book to see the market depth for a trade of a
given size, if you showing your hand all at once is the same as showing it
over a prolonged time as you eat through the depth and the market becomes
shallow.

Essentially what the anti-HFT people are saying is that for a public company
it's rational to expect to buy the entire stock are the current market value,
as anyone who has seen a takeover go down, the purchaser must offer a premium
over the current market cap in order for a bid to be successful.

Simply making an offer usually causes the share price of the underlying
company rise to a very similar to the offer discounted for likelihood of
regulatory approval and the time value of money. HFT is expected market
behaviour simply occurring over a time period of nanoseconds rather than
minutes and hours.

~~~
ubernostrum
The issue seems to be that, when you want to buy a somewhat-large amount of
stock, regulations exist which:

1\. _Force_ you to split up the purchase among multiple exchanges, and

2\. To compensate for this disadvantage to the buyer, are _supposed_ to
prevent news of the order causing a price change at the other exchanges by
spreading faster than filling of the order, so that you don't have to deal
with an artificially-inflated price on top of artificially-imposed split of
your order.

But while (1) is enforced, (2) is not. Which means buyers and most sellers
have to play by one set of rules while people who want to inject themselves as
middlemen get to play by a different set of rules.

At least, that's what seems to be being described here, and I certainly can
see problems with it.

(not to mention that that "liquidity" and "efficiency" arguments fall flat --
the types of HFT schemes being described here work only by creating artificial
_illiquidity_ , and I don't know of any rigorous argument that creating more
middlemen improves efficiency)

~~~
yummyfajitas
There is no regulation which forces you to split up your order. People split
their order because one venue does not have enough liquidity.

 _To compensate for this disadvantage to the buyer, are supposed to prevent
news of the order causing a price change at the other exchanges by spreading
faster than filling of the order,_

This is simply incorrect. There is a regulation which requires exchanges to
transmit news of price changes to the other exchanges as rapidly as possible.
The purpose is to ensure that all fills happen at the NBBO.

In concrete terms, if BATS has a BUY@$20.00 and ARCA has a BUY@$20.01, BATS is
forbidden from filling your order.

[http://en.wikipedia.org/wiki/National_best_bid_and_offer](http://en.wikipedia.org/wiki/National_best_bid_and_offer)

~~~
ubernostrum
If you believe the description is wrong, your issue is with the article.

Direct quote from article:

 _With Reg NMS, it essentially took x amount of available stock at one place,
and made it one tenth of that x at ten different places._

If that isn't a regulation forcing large purchases to split up, then it's on
you to explain why.

~~~
yummyfajitas
The most charitable interpretation of that quote is that after RegNMS,
multiple trading venues became legalized. Liquidity providers then _chose_ to
spread liquidity out because it was now legal.

Here is RegNMS. Find me a place where it says liquidity must be divided
between all possible exchanges:
[https://www.sec.gov/rules/final/34-51808.pdf](https://www.sec.gov/rules/final/34-51808.pdf)

~~~
ubernostrum
If you believe the description in the article is wrong, then as I said _your
problem is with the article_.

------
spingsprong
A veteran geologist explains how the medical industry became "rigged"

------
ThomPete
Can anyone explain to me what wouldn't be possible without HFT? Is HFT
allowing for some useful types of investments/capital?

~~~
yummyfajitas
Buying a lot of shares at once. I wrote a blog post explaining how to not
trade with HFTs if you don't want to:

[http://www.chrisstucchio.com/blog/2014/how_to_not_get_ripped...](http://www.chrisstucchio.com/blog/2014/how_to_not_get_ripped_off_by_hft.html)

Quoth a trader: "Let me tell you this as trader with 15 years direct bank and
hedge fund execution experience. Your suggestion of using such algorithms
would result in me never getting filled even remotely close to where I wanted
to get filled."

tl;dr; He wants to buy 10,000 shares right now and HFTs are the only ones
offering that.

~~~
ThomPete
So its good for the trader who wants to do that. Is there any benefits to the
market overall that he wants to do that?

Edit: Can someone explain to me why I am being down voted from asking a
question? It seems like every time I get into a discussion with yummyfajitas I
am being down voted?

~~~
yummyfajitas
Allowing a trader to make large trades makes price discovery happen faster and
more accurately. I've written a more detailed explanation on my blog - read
HFT apology 1 and 2.

[http://www.chrisstucchio.com/blog/tags/high%20frequency%20tr...](http://www.chrisstucchio.com/blog/tags/high%20frequency%20trading.html)

As for the downvotes, I lead a small cult of traders, rationalists and scala
programmers - tptacek is my #2 sycophant.

~~~
ThomPete
So again the trader not the market/society.

Whats the benefit outside the stock market to society in general? I.e. how is
HFT a net profit to other than those involved in HFT?

You are asking people to be specific, can you you?

~~~
yummyfajitas
The tutorials I linked to address your questions in detail.

~~~
ThomPete
You must be able to give examples.

~~~
yummyfajitas
An example is provided in part 2 of the tutorial. Are you demanding I
copy&paste it here?

~~~
ThomPete
Yes please

------
kasey_junk
" they will signal up to their other machine to say, “Buy everything that’s
available.”"

This sentence is very problematic as it implies that HFT are injecting
themselves between 2 intermediaries that don't need the HFT in the first
place. What is actually happening is the HFT is not buying or selling
anything. They are changing the prices they themselves were offering
originally. There is no intermediary involved.

------
runeks
Here's an interesting counterargument:
[http://streetwiseprofessor.com/?p=8333](http://streetwiseprofessor.com/?p=8333)

------
minimax
_A direct feed from one exchange could be $10,000 to $60,000 a month._

That's way off the mark. It's more in the $1k - $3k range.

[http://cdn.batstrading.com/resources/regulation/BATS_US_Mark...](http://cdn.batstrading.com/resources/regulation/BATS_US_Market_Data_Price_List.pdf)

[http://www.nyxdata.com/page/1084](http://www.nyxdata.com/page/1084)

[https://www.nasdaqtrader.com/Trader.aspx?id=PriceListTrading...](https://www.nasdaqtrader.com/Trader.aspx?id=PriceListTrading2#itch)

~~~
nanexllc
This is the SEC filing with proposed fees
[http://www.sec.gov/rules/sro/nysemkt/2013/34-70285.pdf](http://www.sec.gov/rules/sro/nysemkt/2013/34-70285.pdf)

~~~
Idocrase
That's for a full colocation - not just a direct feed. The originally posted
links are the correct prices for direct feeds.

~~~
nanexllc
You are referring to direct feed content, I was explaining (in the interview)
direct feed speed and content. Who buys a direct feed for a speed advantage
and then doesn't colocate?!

------
hawkharris
"The market is a pendulum that forever swings between unsustainable optimism
and unjustifiable pessimism." \- Benjamin Graham

------
tempodox
Those Washington Post articles are a pain to read on mobile. I will avoid them
like the plague.

