
A detailed data-centric exposé on how the market is rigged (2014) - chollida1
http://www.nanex.net/aqck2/4661.html
======
bd_at_rivenhill
(not so) Detailed Analysis of a Trade: lots of detail is lost here, you can
see the problems start right at the beginning with this quote: "The 2nd column
is the timestamp of the event - this is from CQS/CTA, the consolidated quote
and trade SIP (Securities Information Processor)". Looks like they are
determined to show only timestamps from the consolidated tape, which has been
pretty much useless as a source of information for about 10 years or more.
They omit the firm's receive timestamp for each message and also details of
the trade execution reports received from Goldman Sigma X, which they are
claiming as the source of information leakage that causes this order to be
faded. Regarding Phantom Liquidity, they ask the pertinent question directly
in the article: "what kind of liquidity"? but they get the wrong answer. The
liquidity where you can dump a 20,000 share order directly on a market with
tight spreads and not get faded does not exist now and never existed in the
past; spreads were wider back in the day where you could have a hope of doing
this. If you want to move large size with low slippage, you need to ship it to
someone with good algorithms or spend the money to develop those algorithms
yourself (along with the required technical infrastructure such as direct
feeds, etc); those are really the only options available.

------
stickfigure
I don't understand why stock markets function this way. Why don't they work
like a Diplomacy game, in turns?

Everyone puts their instructions in, then the turn 'executes' and an algorithm
(known to all parties) resolves the actions. Run one turn every second. If you
want to buy all 20k shares on offer and nobody else is making the same bid,
you get them. Seems reasonably straightforward - what am I missing?

~~~
camkego
These type systems has been offered for a number of years, it is a good
question why they are not more prevalent in the US.
[http://goo.gl/xXd8So](http://goo.gl/xXd8So)

~~~
harmegido
See my question above for an answer on why they are not more prevalent.

If you break ties based on FIFO, you've just created a clunky FIFO match,
which is used everywhere.

If you break ties randomly, you've just created a clunky pro-rata match, which
is used everywhere.

------
sambe
This is a fairly typical Nanex piece: start with loaded language before you've
made your case, be "data-centric" yet leave out important details,
intentionally exclude obvious details. By now this is all explained publicly
many times over and Nanex are definitely aware of it.

In the follow-up they admit that by not being stupid you get the fill you
want. Hence not rigged or stolen, just that naive execution is a poor choice:
same as in unfragmented markets, same as before HFT.

A good reference is [http://www.amazon.com/Flash-Boys-Insiders-Perspective-
High-F...](http://www.amazon.com/Flash-Boys-Insiders-Perspective-High-
Frequency-ebook/dp/B00P0QI2M2).

~~~
RachelF
Why can you cancel orders in a few milliseconds? It looks very much like a
great opportunity to rig the marker.

~~~
fleitz
Because you may no longer wish to sell at that price. Similar to the way that
if you have a house on the market and someone makes an offer you don't have to
accept it.

~~~
greghatch
It's not reasonable to assume that a human changed their mind that quickly.
The idea of putting out an offer with the intention of changing your mind and
letting a computer handle that process cements the point that its not a change
of mind here, more likely an intentional misrepresentation of buying pressure
- otherwise what's the point?

This is not what changing someone's mind looks like.

~~~
fleitz
The point is to sell your shares at the highest price possible. Computers help
you do that by reacting quickly to market information.

If someone sends a buy order to the exchange that quote is listed on for the
same amount of shares, it will fill the entire order. What is happening here
is that there are multiple exchanges and when the algo sees an order come in
on one exchange it cancels orders on other exchanges quickly disseminating
price information as liquidity evaporates at the current price.

The point of price is to disseminate information about the supply and demand
of a good. When demand goes up price increases. The buyer is buying because he
thinks the price will be higher in the future.

Both are motivated by profit, at a certain point the buyer will no longer be
interested in buying at that price. The buyer obviously has a piece of
information the algo doesn't and is equally trying to capitalize on the algos
lack of knowledge of that information.

In the case in point the order was ~60% filled. The price the order was
submitted at reflects the 'current' demand, not the demand after the order is
submitted. It's pretty good to get a 60% fill when you are buying 83% of the
currently available stock.

------
fleitz
Imagine you live in a city and there are currently 1000 houses on the market,
you take out an ad in the news paper telling everyone that there are only 1000
and you want to buy 900 of them.

Someone wakes up at 6 AM reads the paper and phones every real estate agent in
town telling them they will pay the asking price and a little bit more, most
agents sell to him and he makes a nice tidy profit off of you who now have to
buy from him.

Is he front running you or are you just dumb? Especially given that you could
take news paper ads out all week long and then let the guy who bought 800 of
the houses take the loss on them. Or you could execute a bunch of small 50
trade orders and buy all the houses you want at the asking price.

~~~
webXL
So the solution to this problem is to break up large trades then? Isn't this
decreasing liquidity and if so, how big of a problem is that? Seems like
trading a large amount of anything has always been challenging.

~~~
fleitz
The market is already illiquid at this demand / timescale which is why the
order is being partially filled, if you spread it out over 5 minutes there'd
be no problem finding that much liquidity.

------
bluejekyll
If this is legit, what amazes me is the time! Network hops can account for
approx 100microseconds, which means that these HFT trades would not only have
to be in the same datacenter, but within no more than 2-3 hops of the exchange
device. Then on top of that, it's making its trade decision within 500
microseconds.

I need to sit down to calculate this, but that is guaranteed to mean that they
are reading from RAM and not disk. Could the entire stock market fit in a
single hashmap in memory?

That's just some awesome hardware and super efficient software.

Edit:typos

~~~
gruez
>Could the entire stock market fit in a single hashmap in memory?

I don't see why not. You also don't have to put the entire market onto one
machine - you can shard the market across multiple servers, with one server
handling each symbol.

~~~
bluejekyll
> shard the market across multiple servers

Sure, but now you've added a router, consistent hash or the like, to the
picture, which is either another hop or a custom physical rack instead of a
traditional ToR. It's not impossible at all, it's just impressive that they do
all of this in 1ms!

~~~
gruez
>Sure, but now you've added a router, consistent hash or the like, to the
picture

not necessarily. If the HFT companies are using some sort of secret internal
API (very possible as they like to shave milliseconds off their execution
time), it's possible that they are talking directly to relevant server,
bypassing any loadbalancers or routers.

~~~
omgtehlion
On a regulated exchages you cannot bypass official gateway. Though you do not
need to. The only thing you want is to be faster than other traders.

------
gd1
I think where Eric Hunsader (and Michael Lewis) go wrong is that they have a
big flaw in their mental model of the market. They make the assumption that if
say 500 shares are offered at exchange A, and 500 shares at exchange B, then
they are entitled to sum those two numbers together and assume that there are
1000 shares on offer. This is wrong. This is the basic mistake that they just
can't wrap their heads around. I will try to explain as simply as I can from
the point of view of a market maker.

Imagine you are a market maker trying to make the spread and provide liquidity
back in the old days. There is only a single stock exchange. Your firm
obviously has a limit on how much risk it can take. How much capital/money it
has available. So let's say you are allowed to post 1000 shares on the offer
and 1000 shares on the bid. You are not allowed to hold a position of more
than 1000 shares long or short. Life is simple.

Then things get complicated. A competing exchange is now set up and begins to
thrive. Yay for competition! Yay for the free market! But you still have the
same limits on your capital and risk. What do you do? Do you now post 500
shares on the first exchange and 500 shares on the second? You could do that.
Then you are guaranteed that you will never be long or short greater than 1000
shares. You'll never break your limit. But what happens when there are 10
different stock exchanges? Do you post only 100 shares at each one? Should the
available liquidity in the market be fragmented into 10 pieces? Of course not.

So what you do is post your bids and your offers for your full amount on each
exchange. And then if you get filled on one exchange, you cancel your orders
on the other ones. This isn't crazy, this isn't radical, this isn't some
nefarious plot. It happens every day in all sorts of markets. I might put my
house for sale on three different real estate websites. When it sells on one,
I cancel the listing on the other two. No big deal. Or if I am a fisherman in
India, I might send my sons to three different markets to sell my catch. When
one finds a buyer, he phones the other two and tells them.
([http://qje.oxfordjournals.org/content/122/3/879.abstract](http://qje.oxfordjournals.org/content/122/3/879.abstract)).
This is simple stuff.

In more technical terms, the offers and bids are all _bona fide ab initio_.
They are all bona fide when initiated. When posted, you do actually want to
sell or buy at that price. When you get filled at one exchange, circumstances
(your inventory) have now changed, and you are entitled to cancel.

Note in my explanation, the market maker isn't front-running, or even back-
running, they are just responding to their own personal changes in inventory
level (position) that means they can no longer offer those shares at the other
exchanges. Electronic market makers may well have sophisticated models that
predicts the short term price move causing them to pull orders at other
exchanges, but we need not even consider those models to explain this
behaviour. Changes in inventory is ample explanation.

Of course, by posting their full available inventory on each exchange, market
makers expose themselves to a new risk. Someone might come along at the exact
same instant and fill them on every exchange! Then they will be over their
limit and massively exposed. Luckily, they have very fast systems and low
latency lines to help protect them from this. If they didn't, they would be
forced to offer 1/n times their current liquidity, where n is the the number
of exchanges. That would suck for all liquidity consumers.

Which brings me to the most amusing part of this whole story, is that when the
'hero' of Flash Boys Brad Katsuyama figured out he could precisely fire off
orders so that they hit all exchanges at the same instant in time with his
'thor' program, the SEC came knocking. Because behaviour like that, ambushing
market makers on multiple exchanges, could in fact force market makers to
offer much less liquidity. I don't have a problem with it, but it amuses me
that the supposed hero was the one probably endangering market structure.

~~~
MagnumOpus
You are obfuscating and you know it.

>"ambushing market makers on multiple exchanges, could in fact force market
makers to offer much less liquidity"

The article (as well as your first paragraphs) demonstrate that this
"liquidity" isn't real in the first place, and gets withdrawn if anyone tries
to trade on it. So if the HFT are not offering it, nothing of value was lost!
(Indeed it is better from a macroeconomic and social standpoint, since the
wasted brainpower of HFT quants goes to some other economically useful
endeavour.)

> So what you do is post your bids and your offers for your full amount on
> each exchange

No, financial markets order books don't work like this. High-latency markets
where your counterparty is a spiv who goes back on their word work like that
(OTC securities, real estate or indeed dealing with people at Indian fish
markets). But in an order book, orders are supposed to be firm and executable.
If they aren't, the market is rigged. As it very clearly is.

~~~
gd1
>(as well as your first paragraphs) demonstrate that this "liquidity" isn't
real in the first place, and gets withdrawn if anyone tries to trade on it.

Where? I have explained that it is real. The confusion stems from thinking you
can sum up the orders at each exchange. It would be like seeing the same house
on three different real estate websites and then thinking you can buy three
houses. They are the same house. Listed three times. The liquidity listed at
the 13 different US equity exchanges is listed by the same market makers, so
you can't sum it. Do you understand?

>But in an order book, orders are supposed to be firm and executable. If they
aren't, the market is rigged. As it very clearly is.

How aren't they executable? No one can see your order coming. If you send a
single order to NASDAQ or any other exchange it will execute. They are all
executable and firm at the exchange level. Nanex seems to think you should be
able to execute everything visible at all exchanges at once. You can't.

~~~
Aardappel
> It would be like seeing the same house on three different real estate
> websites and then thinking you can buy three houses.

No it's not, since with the houses it would be easy to verify that there's no
3 houses available. The trader in the Nanex article had no way to know how
many of the 24800 shares were actually available. He's trading at an
information disadvantage.

To your analogy, it would be similar to a real estate agent flooding the
market with multiple listings of each house, but with different pictures and
different addresses, giving the impression of a buyers market to a buyer
(phantom liquidity).

~~~
gd1
>No it's not, since with the houses it would be easy to verify that there's no
3 houses available. The trader in the Nanex article had no way to know how
many of the 24800 shares were actually available. He's trading at an
information disadvantage.

You are correct, public markets are anonymous by design, so there is no way to
_tell_ that the same liquidity is being offered in more than one place. That
doesn't change the reality that it is. As I explained, it must be, or the
transition from 1 exchange to 13 exchanges would have seen offered liquidity
by market makers cut to 1/13th.

So like I said, anyone with a working mental model of the market knows this.
You can't just aggregate displayed liquidity at each venue and think that
number is valid. It doesn't matter what your 'screen' is telling you if the
screen is doing something that makes no sense. No one ever pretended that you
can magically execute on different exchanges as if they were one exchange. No
one ever made that guarantee. The only people who think that are wingnuts like
Eric.

And since we are talking about an issue that only effects people sending very
large orders to multiple exchanges, the likes of Katsuyama have no excuses for
their ignorance. They are paid to be on top of these details. You can't have a
'trader' in the modern age who doesn't know what a millisecond is, or isn't
prepared or capable to dig below one abstraction layer. "dUh, my screen say
24800 shares are there and i pressed buy" is not what you pay an execution
trader for. It is his fucking job to know what happens when his order goes
down the wire.

>To your analogy, it would be similar to a real estate agent flooding the
market with multiple listings of each house, but with different pictures and
different addresses, giving the impression of a buyers market to a buyer
(phantom liquidity).

They are not "flooding the market" or trying to present "phantom liquidity".
They are doing exactly what you do when you put the house on 3 different
websites. Trying to increase your chances of finding a customer. They are
forced to put bids and offers on 13 different exchanges, because that is US
equity market structure. It's no fun. Life would be a lot simpler if there
were just one exchange. As I explained, it's like playing russian roulette for
them... they are now forced to take the chance that they may get filled
simultaneously on multiple exchanges and take on more risk than they are
comfortable holding. But they are forced to do that since their customers are
now spread across 13 exchanges. Market makers would like nothing better than
to have 1 exchange. Arbitrageurs are the ones who love having all this
fragmentation.

