
Flash Boys a Year Later - ganeumann
http://www.vanityfair.com/news/2015/03/michael-lewis-flash-boys-one-year-later
======
lordnacho
I read the book a few weeks ago, and I was somewhat disappointed.

He focusses way too much on the IEX gang, and in a way that is puts the team
members on a pedestal. Something about it made it sound like he spent a lot of
time with this gang and not so much with others. I'm sure they're smart guys,
but I just don't like the style and it seemed overdone to me. This is having
worked in finance for 12 years and met loads of smart people with credentials
and interesting stories.

Anyway, aside from the style I thought there was a lack of real investigation.

There was one interesting strategy mentioned, the BATS front-run.
Superficially, it sounds like it might work. You see a trade on BATS, you pull
and reverse. Okay. But what exactly is wrong with that? There's a number of
market makers who collectively make the market. A big fish comes in, scares
away the minnows. Is that wrong? If there's a big buyer, shouldn't the price
be higher? That discussion didn't seem to be there.

Another interesting story is the Spread Networks line. Now, what exactly is
being arbed between Chicago and New York? The S&P Future vs ETFs in NYC? Vs
the basket? Can that really support a dozen or so firms paying tens of
millions each? Maybe it can, let us see the data. A recent conversation with
an HFT friend did not reveal the answer. And what happened when the microwave
towers came in? Is the line useless now? At the end, I wasn't satisfied I knew
WHY the speed was necessary. All I can think of is that there's some very
obvious trade that can't lose money if you're the fastest. What is it, and why
is it a no-brainer? Is is anything untoward happening?

And then there was the funny order types. How about giving us a concrete
example? Use such-and-such an order, cause the market to cross, end up at the
front of the queue, profit. There was some mention of the multitude of order
types, but the case isn't made until you bring something concrete. Someone
like Nanex might have data showing these shenanigans. I picked up another book
by Tabb, who was mentioned, that has some more detail.

~~~
infecto
I really enjoyed Dark Pools. Of course it also focused on a side but it was
more enjoyable to see how things came to be, the creation of Island and how
seemingly good intentions created the market we have today.

~~~
ethbro
_> how seemingly good intentions created the market we have today._

It's been a bit since I read Flash Boys _[edit: didn 't realize you were
talking about a different book]_, but from memory dark pools were never
created with good intentions. At best, they were created with the intention of
enabling cheaper trades by not having to route orders to the broader public
market (just like a colo CDN)?

And dark pool owners had all the information needed to gather real-time
reporting on the prices predatory dark pools were trading normal investors'
orders at vs. the broader market price.

Which is to say, they absolutely knew they were screwing someone.

~~~
tptacek
It's tough to discuss dark pools for a couple reasons. First, they have a
sinister name. Second, they're generally run by giant financial firms, who
nobody likes right now. Third, they're a fuzzy concept. In reverse order:

A dark pool is really just a private exchange, or really just what you'd call
anything that functioned in any way like a private exchange. They're a tool,
deployed to solve problems for (primarily) large investors. When you start
talking about market structure and facilities, it can get pretty weird to talk
about "intentions". Major exchanges are also routinely used for shady
transactions!

The best-known purpose (or "intention") of a dark pool is to allow huge buy-
side institutional investor to move large blocks of securities. If you don't
spend much time reading about market microstructure, you're probably unaware
that this is one of the fundamental problems in trading.† Market "physics"
dictate that there is no instantaneous price available for the sale of a large
block of securities. A huge order will gobble up many orders off the order
book, and, as it executes over time, it _moves prices_ \--- the intent of a
large investor to dump a lot of securities on the market is extremely relevant
to everyone else in the market, and to the correct price of that security.

The best known dark pools tend to be owned by giant investment banks and
similar firms. This should be unsurprising, because one of the key services
those firms offer their clients is "efficiently shopping or buying large
blocks of securities". To deliver that service on open exchanges, they have to
compete with other firms that are aggressively trying to detect big moves and
profit from quickly making prices reflect those moves, often while the move is
happening. The participants in a dark pool are cooperating to "more
efficiently" move large blocks in private without impacting prices (as much)
while the move is happening.††

(Dark pools also end up executing some retail order flow as well; as "private
exchanges", they're also a useful service for their operators to be able to
sell to retail brokerages. They take on the task of trying to ensure orders
are executed quickly, and their other trading clients get exposure to retail
order flow, which is demographically likely to be uninformed and profitable to
trade against.)

Finally: the name. It sounds Death Star-ish. But really it's just a comparison
to normal "lit" exchanges, where order books are visible to participants.
"Lit" and "dark".

It might seem like lack of transparency and visibility into prices could be a
license for all sorts of shady stuff. And some shady stuff happens (apparently
most of which has to do with the promises made to dark pool clients when they
agree to execute orders there). But most of the "shenanigans" that drive
concerns about HFT require order visibility (I like to suggest that they are
direct consequences of the CAP theorem, applied to electronic markets).

† _Bafflingly, a huge and apparently valid complaint about _Flash Boys_ is
that Lewis --- a former bond salesman --- doesn 't seem to understand this
fact._

†† _I suspect we 'll find that when we start ascribing motives and intentions
and morality to market activities, we'll quickly find out that this stuff is
all murky, and that almost any activity can be "shady" or "virtuous" viewed
through the right lens. For instance: some dark pool controversy involves big
pools persuading big investors to execute trades on their pools, whereupon
they're "victimized" by HFTs and other aggressive traders. Through the lens of
"promises should be kept" and "truth in advertising", this is bad. But through
the lens of "giant financial firms probably shouldn't have a special sneaky
execution lane for dumping a zillion shares on the market without alerting
other investors that they are moving the market", things look a bit
different._

~~~
minimax
_The best-known purpose (or "intention") of a dark pool is to allow huge buy-
side institutional investor to move large blocks of securities._

That is the raison d'être for only a couple of dark pools (mostly the older
ones like liquidnet). The reason most big i-banks have their own dark pools is
because it's cheaper to trade on their own ATS than it is on most of the
public exchanges. The "take fee" paid by anyone crossing the spread on the 5
biggest exchanges (by volume) is usually in the range of $0.0025 - $0.0030 per
share, which doesn't seem like a huge number, but it can add up when you are
trading tens or hundreds of millions of shares daily. If banks can trade
customer and internal flow on their own ATS, they don't have to pay those
fees.

You can look at the average trade sizes broken down by symbol and ATS. The
data is here: [https://ats.finra.org](https://ats.finra.org). The average
trade size for most ATSs is somewhere between 100-200 shares. The whole "dark
pools are for crossing large blocks" thing is sort of anachronistic these
days.

~~~
tptacek
I was hoping someone would come and provide this additional color. Doesn't
this fact _reduce_ the moral implications of dark pools, though?

~~~
minimax
There are a couple of complaints I have seen.

1) That orders that are "pegged" to the NBBO at dark pools may execute at
inferior prices relative to the true NBBO. If the market moves from 20.23 x
20.24 to 20.24 x 20.25, you can in theory try to quickly buy in the dark pool
at 20.235 and lay off by selling at 20.24 in the lit market. The theoretical
per-share profit would be 0.005 before fees and probably zero after. This is
the whole point of the IEX 40 (or whatever) miles of fiber in your order entry
path. It gives IEX the opportunity to update their view of the NBBO while your
order spins around in the coil. I think the issue is partly intrinsic to
pegged orders. The pegged order is effectively getting a free ride re pricing
based on the the price discovery happening in the lit markets. Maybe that is a
natural trade-off? I actually read a really good blog post recently where a
guy tried to work out what would happen if more exchanges tried these sorts of
delay gimmicks. It's worth a read if you're interested.
[https://mechanicalmarkets.wordpress.com/2015/03/03/should-
ie...](https://mechanicalmarkets.wordpress.com/2015/03/03/should-iex-become-a-
full-exchange-incentives-for-exchanges-to-compete-by-increasing-delays/)

2) The Barclay's thing where the claim is that Barclay's could prevent you
from trading with certain counterparties or types of counterparties but maybe
they didn't? It's been in the news but I don't think anything has really been
settled. One interesting thing is that the court case is between the NY
attorney general and Barclay's, not between Barclay's and any of Barclay's
(presumably legally sophisticated and deep pocketed) customers.

3) Unsophisticated or low-cost order routers that sequentially try to trade at
various dark pools signal to the market makers who are quoting on those dark
pools that someone was potentially trying to sweep. If you think market makers
should be able to use this kind of information to avoid being adversely
selected against on their other orders, then it probably doesn't bother you,
but it bothers people like Michael Lewis. A crappy order router can let out a
lot more signal than a good one. It seems like the kind of thing you could A/B
test but I don't work on the buy-side so I'm not sure what they do.

------
encoderer
HFT is not bad. Technology is democratized now. The best trading tech is
available to consumers. The professional edge lies in their unlimited, low-
cost leverage not in their technology. As a consumer, HFT is not my
competition it is my order facilitator. A huge share of equity trades get
their price IMPROVED by the market makers in order to capture the trade flow.

I love Michael Lewis. The Big Short was a very enjoyable read. I've read all
of his stuff. I've run into the guy at the grocery store. But Flash Boys was
just silly. Even if you believe it, the guy getting "disrupted" was not a
retail trader but a broker crying that HFT was making it hard for him to dump
orders on the market in 10,000 share lots.

Edit: I won't respond to every critic, I respect your views but I've given it
thought and have reached a different conclusion. You can quantify that orders
are filled today more quickly, at better prices, with far lower commissions,
and we have tighter bid-ask spreads, more penny-wide markets than ever before,
and an explosion of ETFs that give retail traders access to something that
they otherwise would need a futures contract ($100k in notional value) to
trade. And of course in the world of algorithmic trading, there is technology
not available to consumers. But a consumer today can have a setup at home that
is as sophisticated as a professional trader's setup. That never used to be
the case. But the software is commoditized now. The trading platforms
available to consumers, like Thinkorswim and Interactive Brokers, are top-
shelf.

~~~
tdaltonc
It's absolutely not democratized. The opaqueness of the trades and the fact
that the vast majority of trade volume passes through pools with centralized
(and again opaque) rule-writing means that the those centralized authorities
can and do write the rules to privilege insiders.

~~~
tptacek
What "opaqueness" are you talking about? What kinds of "trades"? What are you
referring to when you talk about "trade volume"? What "pools" are you talking
about? By the technical definition of the term --- and the context of your
subsequent words --- you could (validly) be referring to the major exchanges
as "pools"; they are, after all, centralized and responsible for their own
rule-writing.

------
dreamweapon
"Shook to its core" is more than a bit overstated.

~~~
Lazare
I was going to say the same thing.

My feeling is that "Wall Street" mostly ignored the book, which they expected
to have no lasting impact. And even that turned out to be an overestimate. Not
only has it had no lasting impact, I wouldn't say it had a fleeting impact
either.

In fact, I can't think of a single measurable impact the book has had on Wall
Street. At most it gave people who already distrusted the financial industry a
few more reasons to do it, but it led to no new laws, no new regulations, no
prosecutions, no _anything_ really.

And if you ignore the headline and look at the article, Lewis's main argument
seems to be that he made some very rich people "angry". Which is 1) a far cry
from "shaking wall street to its core" 2) something he offers zero
evidence—even anecdotally—to support and 3) also not true, as far as I'm
aware.

Don't get me wrong; I've read several of Lewis's books, I like his writing,
and I'm sure he's a great guy. But I kind of doubt even Lewis believes this
spin; it's just marketing.

Edit: It's also worth remembering that the losers in the rise of HFT are
overwhelming the big institutional players in the market. People have this
weird idea that HFT is all about big banks victimizing small investors, but in
reality the big banks are the losers, and the small investors are the winners.
(Yes, really.) Which makes Lewis's claim that he was making the richest people
in Wall Street angry so ironic. They were already angry at HFT, and they tried
to use Lewis as ammunition in their ongoing fight against HFT. It's a bit like
making a movie about how fracking is terrible, then priding yourself about how
you must have made the big oil majors very bad...not even realising that they
were the ones funding you all along, because fracking is a threat to them, not
an opportunity.

(And yes, that happened. Matt Damon's film _Promised Land_ turned out to be
heavily funded by the UAE, who has a vested interested in discouraging
domestic fossil fuel production in the US. I'm sure Damon thought he was
standing up to Big Oil, but reality is more complex.)

------
gd1
"between high-frequency traders, who trade with computer algorithms at nearly
light speed"

Seriously, why the do we have to put up with this shit? "Google, which returns
the results of your search queries over a network at _nearly light speed_ ".
"Your remote control, which changes the channel on your television at _nearly
light speed_ ". It is too stupid for words. Old school luddism.

~~~
devingoldfish
The book explains that one company was running their own fiber optic cable
between Chicago and New York to shave a handful of milliseconds off the
latency. HFT firms also built their trading desks as close as possible to
where the fiber terminated in New Jersey. This isn't the speed of your mom's
google query, they were literally running up against latency caused by the
speed that light travels down a cable.

~~~
Animats
Right. If you're totally unfamiliar with HFT, you don't realize how far they
go to get latency down. Computers are too slow for HFT. There are trading
algorithms written in VHDL and loaded into FPGAs which are looking at packets
as they come in over gigabit Ethernet.[1] (That description is four years old
and out of date.)

All this is really to achieve front-running, executing an order after another
order has been submitted but before the first order is executed. This is
betting on a sure thing. It's also illegal.

[1] [http://www.wallstreetfpga.com/resources/fix-on-an-
fpga/](http://www.wallstreetfpga.com/resources/fix-on-an-fpga/)

~~~
harryh
Your understanding of what defines front running is incorrect.

Front running is when your stock broker gets an order from you but then turns
around and executes an order on his own behalf before he executes yours. This
is illegal because your broker has a fiduciary duty to you, his client.

It's not front running when I see an order on one exchange and then, very
quickly, go make an order on a different exchange. It's not illegal because I
have no fiduciary responsibility to any of the other people involved.

You have no right to execute multiple orders on different exchanges
atomically.

~~~
Animats
If someone paid for order flow or fast access so they could do that, it's
illegal.

~~~
tptacek
If that's true, you should be able to cite a statute, an SEC/FINRA/CFTC rule,
a court ruling, or an exchange rule to explain how (allowed and prohibited
behavior on markets being defined by all four of those kinds of sources,
frustratingly enough).

I suspect you won't be able to find any such source. Malfeasance by trading
firms makes career cases for prosecutors.

That's not to say, normatively, that that's how things _should_ be: obviously,
prosecutors are not making much of a dent in the trustworthiness of big
financial firms.

------
aswanson
The best argument I have heard in favor of HFT was from Jim Simons of
Renaissance Technologies. He said that the so-called market-makers would run
at the first sign of trouble (as they did in the 1987 crash) and caused
prolonged downturns, whereas the algorithms for HFTs were truly liquid market-
makers and kicked back in the buying faster once prices got irrationally low.

~~~
thrownawayhft
I worked at a market maker (owned by a bank) that handled around 8-10% of US
stock trades. When things like the flash crash happened, the strategy was to
stop anything automated, because the simple strategies had no way of knowing
the cause. Trading on normal market-microstructure signals, the trading that
provides the bulk of liquidity, would just get you creamed.

To meet our market maker requirements, which allowed us to do things normal
market participants can't do, like naked shorting, we would do things like
leave a 1 penny bid and ten-thousand dollar ask and not change them for the
entire day. Exchange regulators tried to address this and made market makers
quote within some percentage of the bid/ask. Nasdaq literally just created an
order type that would automatically reprice itself to always be exactly that
percentage away from the bid/ask, so that it could never execute--just like
the previous 1-penny bids--but would comply with the regulation. The order
type only existed to bypass that regulation and was literally created in
response to demand after its passage.

Even without that order type we'd just do (and did do, it took them a while
after the regulation to add the order type) the same thing in the models'
trading logic.

~~~
thejaredhooper
Do you have any links/resources for the subject matter you're covering in this
comment? I'd like to learn more about what you're saying, and I can't seem to
wrap my head around what it is you're trying to express with penny bids.

~~~
harryh
What he's trying to say is that his firm was contractually required to have a
bid (an offer to buy) and an ask (an offer to sell) for some set of stocks.

But under certain circumstances they didn't really like this requirement so
they kind of cheated by changing their bid to 1 penny and their ask to
$10,000. For a stock that trades around, say, $100 this means that while they
de-facto had a bid and ask in place it wasn't really a real bid or ask as no
one else would ever want to trade at those prices.

~~~
thrownawayhft
And under the regulation we might have been required to ask at minimum $120
for that stock when it was trading at $100. If the stock had a big day and
rose to $200, Nasdaq would just raise our ask to $240, or whatever exact
amount the regulation required. The only risk compared to the old way of doing
things was that someone would hit the stock with a huge enough order to blow
through all liquidity in one shot on the way up to our ask.

------
itschaffey
I read the book the week it came out and it felt like watching a gripping
thriller. I think the reaction/ debate to his book was a (welcomed)
distraction for the Wall Street bankers. His book left much unanswered,
inevitably, but I fail to see any progress/ change since he published it. His
debate on Bloomberg with William O'Brien (BATS Global Markets Presidents) was
legendary and great viewing. But ultimately Brad Katsuyama came out as the
hero of the book and it would be interesting to know how the IEX Group are
doing now... I hear there are plans to make it in to a movie? Perhaps I will
be able to watch it as a gripping thriller...

------
chollida1
I've spent more than a few words bashing Flash boys so I won't try to do it
anymore.

In a way I feel kind of bad for Michael Lewis. He's had such success with
earlier books that it must be hard for him to find a target for each new book.
In my opinion this book was his first awful book.

Instead of finding data and following it through to reach a conclusion he
starts with a trendy conclusion, HFT is bad, and then really contorts and
reaches with his data to try and make his case. He was also hit by the issue
of HFT starting to wind down around 2010 and really starting to wind down
around 2012. Many HFT firms no longer concentrate on the US Equity markets
alone, they do alot more volume in the futures, options and bond markets where
there are alot more mathematical correlations, and hence more opportunities
for mispricings to correct.

If you want a sound rebuttal, and in my opinion a better book have a look at
this book:

[https://news.ycombinator.com/item?id=8577237](https://news.ycombinator.com/item?id=8577237)

I find it more balanced, better researched and it provides a much better
understanding of the HFT industry.

Having said that, credit where credit is due, he's a tremendous writer and the
book is a fun read that you can consume in 2 days. If you like it then I
recommend his entire back catalog, they are all fun and informative reads.

 __EDIT __someone asked for an example of why I didn 't like the book so....

One of Lewis' biggest issues was what he termed HFT front running. This alone
was a pain as front running already had a well understood meaning( it
originally was meant for a intermediary who took your order and executed its
own order ahead of yours, often buying the stock lower and selling it to you
at a slightly higher price).

His example was that a HFT system would see an order for say Microsoft trade
on the exchange BATS, say a buy of 1000 shares. The HFT system would then run
ahead to all the other exchanges and buy up the remaining shares on those
other exchanges, an ammount that might be 1000 shares or it might be 100,000
shares, so when the remaining part of the order to buy Microsoft got to those
exchanges it would have to buy the shares from eh faster HFT system at a
higher price.

This is down right silly for a few reasons.

1) it assumes that the order that got filled at BATS was either a market order
or a limit order that was priced for more than the fill price, because if it
was a limit for the same price as the fill then buying up the remaining shares
won't do the HFT firm any good:)

2) it assumes that the order was for more shares than what got filled on BATS,
so now the HFT firm is exposing itself by owning shares it's not even sure
anyone wants to buy at the price its willing to sell.

3) it assumes that the HFT system can both buy the remaining shares at the
original price and get to the top of sell side of the order book to sell those
shares back. The HFt firm now has to take the risk that it won't be at the top
of the order book and even if it was right about steps one and two, a big if,
it might not be able to capitalize on it as it can't jump ahead of anyone who
had previous sell limit orders at the price it now wants to sell at.

There are just so many unknown factors there that there is almost no way this
type of trading system could be profitable.

Remember HFT firms worship at the alter of the law of large numbers, they make
fractions of a penny per share traded but only do it if they have an edge ie
90% upside to 10% downside. There just isn't any edge to be had in the above
scenario, its just a heaping pile of risk.

~~~
devingoldfish
Ah so in your view, why was the order flow of retail brokerages worth hundreds
of millions of dollars per year to these HFT firms?

Michael Lewis is not the only person accusing HFT firms of front-running
trades. Joseph Stiglitz, a nobel laureate in economics, has made similar
accusations. Does he too just not understand the market?

~~~
patio11
_why was the order flow of retail brokerages worth hundreds of millions of
dollars per year to these HFT firms?_

Order flow from retail brokerages can be assumed to be "non-directional",
which means that it comes from people who are buying or selling for some
reason other than "I have better-than-market knowledge of information which
will shortly be relevant to this stock."

One _extremely important_ example of directional order flow is when you have
the knowledge "100k shares of this stock are shortly going to be shopped on
various exchanges" because _you 're doing the selling._ This will typically
cause price impact (i.e. the price of the stock declines), meaning that market
makers who take your first few hundred/thousand shares are going to get
shellacked. They generally don't love this.

In the (virtually guaranteed) absence of directional order flow, market making
is a _license to print money._ Both sides pay you the spread and you don't
accumulate much inventory risk. (i.e. You buy, you sell, you sell, you buy,
and you're rarely left with a meaningfully sized position in either direction
which would expose you to the stock at issue.)

That's why you pay for non-directional order flow. It's like leasing a toll
bridge.

~~~
harryh
It does seem like the fee being paid for order flow should be able to be
captured by those making the orders in the form of further reduced spreads
rather than by brokerage firms selling the flow. Though perhaps the issue is
that with stocks regulated to trade in penny increments that's hard to do?

I am uncertain.

~~~
kasey_junk
The spreads can't reduce much more than they are now due to the regulation
issue.

The order makers do see part of that rebate in the lowering of the fees they
explicitly pay to trade.

Robinhood is a particularly good example of this. They seem to be financing a
no-fee model based purely on selling the order flow.

------
stillsut
Just finished the book, was not shaken to my core.

When 99% of trading became computerized,very few "trading desks" knew anything
about the foundation of what they did. They were still trying to be an 80's
guy.

Not surprising that learning deeply about the electronic system at the basis
of trading was lucrative for the few who did,

------
mrdrozdov
More of a question than a comment, but how do Trading Fees [1] effect HFT and
the market? I suspect that high volume trades over time could be seriously
effected by the trends in these fees.

[1] e.g. NYSE Trading Fees, [https://www.nyse.com/markets/nyse/trading-
info#trading-fees](https://www.nyse.com/markets/nyse/trading-info#trading-
fees)

~~~
inspectahdeck
Most HFT players are market makers, not market takers. If you take a look at
the fees in that link, the fees for liquidity taking are positive, i.e.
$0.0027, whereas the fees for liquidity providing are negative, i.e.
$(0.00150).

Exchanges offer rebates to market makers to incentivize market activity on
their platform.

