
A High Frequency Trader's Apology, Pt 2 - uggedal
http://www.chrisstucchio.com/blog/2012/hft_apology2.html
======
jellicle
> Of course, it’s highly illegal to use trojans to rob retail investors and
> game the stock market, so this story is not particularly realistic.

But in fact that entire scenario he just described is perfectly legal. The
industry calls them "flash orders", and due to industry pushback against an
SEC initiative to ban them in 2009, they are still legal.

Nutshell description:

\- you put in an order to buy AAPL

\- your trusted exchange "flashes" the order to a company that has paid for
the information, so they know you want to buy AAPL. They now have exactly .5
second (SEC rule) to act on that information before your order hits the wider
market.

\- said company goes and buys up the AAPL on the market, ahead of you

\- your order to buy AAPL hits, executing at a slightly higher price than you
expected

\- said company sells AAPL at the new higher price (perhaps to you)

You've acquired AAPL at a slightly inflated price, and another company has
pocketed the extra money you paid. They could buy low and sell high with a
GUARANTEE of success because they knew your order was coming to the market. We
used to call this front-running and call it illegal, but not in the modern
U.S. stock market.

This is completely legal in the United States. It's happening now. It happened
hundreds of times while you read this sentence.

~~~
grandalf
This is done in order to avoid intra-ETN transaction costs. There is nothing
malicious or underhanded going on.

This has always happened. Suppose you're in a room with your investor club and
you want to buy 500 shares of AAPL. The evil "flash order" is akin to
mentioning to your local group "hey guys I want to buy 500 shares of AAPL at
$x, in case any of you want the other side of the transaction".

If none of your local group (with whom a trade would incur no transaction
cost) wants it, then it goes through to the wider market where it may or may
not find a counterparty.

The 0.5 second time limit is arbitrary, chosen to accomodate a range of
latencies for the various local would-be-counterparties.

Nobody is stealing anything. It's just a way around some of the fees that one
willingly pays for harder to fill orders. If an entity on your local ETN has a
market making strategy where he/she is willing to fill some orders, then a
flash order rule can improve efficiency by removing transaction cost.

If you think this is a bad thing then you have a profound misunderstanding of
its mechanics. Ironically, like most of the anti-HFT claims, your position
benefits the old school establishment exchanges. ETNs are the little guys
trying to compete against the big guys by offering better technology and lower
transaction costs. Flash orders are part of their service to fight against the
competition-stifling fees charged by the big guys to route electronic orders.

~~~
jellicle
>There is nothing malicious or underhanded going on.

And yet most market participants think banning them is a no-brainer.

<http://blog.themistrading.com/when-exchanges-attack/>

~~~
grandalf
You've provided data for my assertion that the big, establishment players
oppose flash orders. Of course the NYSE opposes flash orders. The NYSE is the
big, old, fat cat. The ETNs where flash ordering is common are small upstarts
that offer lower transaction fees and better technology.

The NYSE's members are the big market making firms who have the most to lose
if there is more competition -- IE more small firms using market making
strategies.

------
SomeCallMeTim
OK, answer me this: How does an HFT, _in practice_ , add liquidity to a
market? None of the examples given showed liquidity being added, because the
people involved would have traded with each other directly without an HFT
being there.

Further, I submit that any HFT will only place a buy/sell spread in the case
where the volume is high enough that they can complete their purchases within
seconds or at most minutes.

At best an HFT will cut a few minutes (more often seconds) off of the time of
a trade. Anyone who is worried about waiting a few minutes for a trade to
complete is Doing It Wrong. Therefore, the "liquidity" HFTs provide is _only_
in the cases where it's not truly needed because the volume is high enough to
provide it anyway.

I've traded in a lot of stocks where there was very little liquidity, and
fewer than a few dozen trades would happen per day. I'd sometimes wait hours
for a trade to complete. Where are all the HFTs providing me "liquidity"? They
weren't anywhere to be found, because in the cases where the volume is too low
HFTs can't make a sure bet. And when the volume is high, they aren't needed.
So what good are they again? At any point that an HFT is willing to buy stock,
it's because there's a _high_ likelihood that someone will show up to pay more
after a few seconds or minutes.

IMO the popular media actually has it right. HFTs really are just stealing
pennies on every transaction, because they only enter a stock and "add
liquidity" if and when the stock doesn't need it.

The onus isn't on me to prove they aren't providing liquidity, but on the HFTs
to prove they are, and the OA doesn't even come close. In _every example_ OP
gives, the people actually buying and/or selling stock would have been better
off without the HFTs in the mix (at best their trades were a few minutes
faster).

If we just taxed or put a fee on a "short term hold" of a stock, say less than
one day, then high frequency trading would vanish. I see HFTs as a parasite on
a broken system, nothing more, and I'm shocked that so many HNers idolize
them.

~~~
yummyfajitas
_None of the examples given showed liquidity being added, because the people
involved would have traded with each other directly without an HFT being
there....In every example OP gives..._

You clearly ignored one of my examples.

Consider the example of Fry and Zoidberg. Fry places his sell order at 12:01.
Leela and Bender are absent, so Fry's order goes into the book. Zoidberg
places his buy order at $9.50 at 12:05. The bid/ask spread is now $0.50 and
Fry has not traded.

In this case Fry is better off with Leela in the market.

 _At any point that an HFT is willing to buy stock, it's because there's a
high likelihood that someone will show up to pay more after a few seconds or
minutes._

If you want to take that risk, place an ALO order at the bid or ask price. You
won't cross the spread, though your order might go unfilled.

The fact that most people don't place ALO orders suggests they don't want to
take the risk.

~~~
wtvanhest
I don't think that SomeCallMeTim understands that 0.10/share is a lot of money
($100,000) when a mutual fund is trading 1 million shares over 10 days.

Now mulitply that by 60 holdings which are turned over 100% per year and you
can see why pensions would want those mutual funds to save $6,000,000/year in
liquidity costs.

(Obviously mutual funds don't just do program trades, they also negotiate
directly with each other etc. but the example still stands).

~~~
SomeCallMeTim
I get that $0.10/share is significant, or I wouldn't have bothered posting.
Why be annoyed at a process that isn't costing anyone anything significant?

I also know that HFTs are taking money out of the equation, not adding money
into the equation. If they weren't, they wouldn't be doing it -- and if the
only value they're adding is reducing the time that a trade takes place by
minutes or seconds, then I _still_ submit that the value added to the stock
market isn't worth the ACTUAL dollar cost.

The OP pointed at the example with Fry and Zoidberg. In that case, Leela
either makes $0.05/share on the transaction (which otherwise would have gone
to Fry and/or Zoidberg), or takes a hit of $0.50/share. In the latter case Fry
(say that's you're mutual fund) makes $0.50/share more than they would have
otherwise, but that money doesn't come out of thin air -- it's lost by Leela.

But in order for HFTs to do well, they have to make more money than they lose,
so for every case like the above where Fry does better than he would have
otherwise there is more money lost by people who would have done better
without the interference of HFTs.

So for every $100,000 saved by a mutual fund, much more is being siphoned off
by HFTs, for no real added value.

~~~
wtvanhest
_I also know that HFTs are taking money out of the equation, not adding money
into the equation. If they weren't, they wouldn't be doing it_

This assumption is questionable. There are a vast number of market
participants who do loose money.

The best start up example is the bias toward reporting companies who just got
funding and not reporting all the companies that hit the dead pool.

We don't really have a good idea on the net of HFT strategies and it is just
as safe to assume they net zero.

Lets just say HFT firms did net to zero, would you still have the same
argument?

~~~
sirclueless
That seems massively unrealistic to me. It's safe to assume that HFT makes its
participants a lot of money, for example because we know that they tend to
spend a lot of money on hardware.

~~~
wtvanhest
Do VCs all make money?

Mutual funds beat the bench?

Hegde funds?

Airlines? They spend a ton of money on fixed costs.

Some make money some do not.

~~~
danenania
They all have revenues. Whether they make profits is another question, but
that's not relevant to the point. If HFTs didn't at least have positive
revenues from their trades, then no, they wouldn't be buying expensive servers
and hiring expensive talent. Money extracted from the market is the issue
here, not whether the amount extracted is sufficient to cover costs.

~~~
wtvanhest
I understand where you are coming from, but investments are not made the way
you are assuming.

An HFT investment likely involves someone deciding they want to create an HFT
fund, going out to institutional investors and selling them on the idea that
THEIR fund will be profitable.

That happens over and over and investors in these HFTs (as well as hedge
funds, PE etc. all think they are picking winners who can make it work.
Whether they can or not will depend on execution.

Investment in a sector, industry, or asset class does not mean there is net
winning from that sector.

At the end of the day, HFT is still competitive since they are bidding against
each other (hence the arms race for faster equipment). Even if the market is
growing for HFT now, it does not mean it will be forever. At some point it
will become mature and all that will be left is net negative HFT profitability
with all the benefit going to investors in the form of liquidity.

I am not backing up HFT because I trade in HFT, I do not. I am backing it up
because I find it agitating that smart people on HN look at HFT and assume
because they are smart, and because they don’t understand it that it must be
bad. If something in the market is happening you don’t fully understand it
really pays to sit back for a second and think about it.

(I’m guilty of not doing that with a variety of topics too and I do not fully
understand HFT, but I understand enough to say that I cannot say with any
certainty that it is bad and lean more to saying it is net good.)

------
SkyMarshal
One of the biggest justifications for HFT that I see is that it increases
liquidity. However, did major markets ever really have a huge problem with
lack of liquidity, 20 or 30 years ago before HFT?

I can't help but wonder if this level of liquidity is only really useful to
HFT, if it is something that HFT is both the primary provider and beneficiary
of, and if they're sort of using their existence to justify their existence,
so to speak.

Chris uses mom and pop trader examples to explain the concepts here, but the
fact is that Algo trading now accounts for much of the volume on major
exchanges, and HFT is probably a decent portion of that.

I'd love to see Chris or any other HFT trader's take on that. He sort of does
near the end, but would be interesting to see a more in depth discussion:

 _> Although the latency competition provides little to no value to the end
consumers, all HFTs must play the game. If they don’t, a faster HFT will beat
them to market and their order flow will be reduced.

This is a very real social cost of HFT - many very smart people spend a lot of
time and effort reducing the latency of trading systems. I’m no fool, but when
I worked as an HFT I often felt like one. The field contains a lot of very
smart people, and the world would probably be better off if the stock market
had a little more latency and those smart people were building products for
the world. For example, after leaving HFT I built a useful consumer product.

But with current market mechanics, the global optimum where trading is
marginally slower and smart people build useful products is impossible to
reach. The latency arms race has us stuck in a suboptimal Nash Equilibrium
which is similar in character to signalling competitions (e.g., the education
bubble)._

~~~
harryh
> However, did major markets ever really have a huge problem with lack of
> liquidity, 20 or 30 years ago before HFT?

Yes. It used to be that humans were the market makers. Humans are (generally)
a lot more expensive than computers so providing liquidity came at a higher
cost. Bid/ask spreads used to be much higher.

HFT it just an example of computers replacing humans at a lower cost. A
phenomenon that can be observed in many industries.

~~~
tptacek
Humans are also far less transparent than algorithmic market makers. Talk to
professionals about the specialist system. It's (words chosen carefully) _hard
to take anyone seriously who pines for the good old days of 1970's and 1980's
trade execution_.

~~~
teamonkey
> hard to take anyone seriously who pines for the good old days of 1970's and
> 1980's trade execution.

I've just read through the entire thread and I don't think _anyone_ has
suggested that.

~~~
tptacek
"did major markets ever really have a huge problem with lack of liquidity, 20
or 30 years ago before HFT?"

~~~
SkyMarshal
I wasn't 'pining' for those days, I was barely even alive then. Just asking.

------
arjunnarayan
One point that is bothering me is this: how much of the existence of HFTs is
an artifact of the rules of the exchange? In particular, the rule that the
first bid gets priority in executing the trade. It strikes me that the entire
existence of HFTs seems to be taking advantage of this failure of mechanism
design.

~~~
fr0sty
There was an extensive discussion on this point in the previous HN thread:
<http://news.ycombinator.com/item?id=3855610>

The short summary is no one can come up with something better than price-time
priority for matching orders.

~~~
btilly
I still like my suggestion at <http://news.ycombinator.com/item?id=3855846>.

True, the HFT folks would still try to provide liquidity by maintaining a
bid/ask spread. But every trade that executes because someone wanted to trade
while the price was somewhere between the bid and the ask would cut the HFT
folks out of the loop to the benefit of everyone else. And scary anomalies
like the flash crash would be impossible.

After careful thought I am sure that it would not eliminate HFT. But it would
reduce their size and impact on the market.

~~~
yummyfajitas
_...every trade that executes because someone wanted to trade while the price
was somewhere between the bid and the ask would cut the HFT folks out of the
loop..._

See the first post in the series. This is exactly how the markets work. The
best price always wins.

If the bid/ask is $10.00/10.05, and I offer to buy at $10.05, I will trade
immediately. If I offer to buy at $10.03, I jump the queue and will be the
first to trade (provided someone is willing to sell at $10.03 or lower).

~~~
btilly
You either did not read my suggestion, or did not understand it, because it is
very different from how the markets work today.

Suppose that the bid/ask is $10.00/10.05. Suppose that I am willing to buy at
$10.20.

In today's market I will immediately make the trade at $10.05.

In my suggested market there is another fact to consider, the price. Suppose
that it is $10.03. Then I become an outstanding buy order, and for as long as
I am outstanding, the price will drift up. If the price reaches $10.05 without
finding a seller, then I will trade at $10.05 with the HFT folks. But if a
seller who is willing to sell at $9.90 comes along before that happens, we
will trade with each other at the current price and leave the HFT folks out of
the loop.

My suspicion is that, if this were implemented, a large fraction of trades
would actually execute in the middle ground between what the HFT traders are
willing to offer as a bid/ask. Therefore this would be even better for actual
speculators than the current system. HFT players would still be in that
market, but they would not be as important.

In short the difference with my system is that trades may have a small delay
before they execute, but should execute at the same or better price for me
than the existing market mechanism.

~~~
yummyfajitas
I definitely misunderstood what you proposed.

Your idea is definitely interesting. It certainly slows down price discovery,
though probably not enough to matter.

I'd need to think more carefully about whether it's a good idea or not, but
it's definitely the best suggested tweak to market mechanics I've heard in
this thread (or the last one).

------
haberman
Loving this series of articles!

Does any of this low-latency work make it back upstream into Linux or other
parts of the software stack? As a latency-obsessed person I would love to know
that the fruits of all this labor were available to me for my own low-latency
systems. To me, that alone would be enough to feel that the latency-race is
providing value to the world.

Also, one thing that was not clear to me is how electronic market makers make
decisions. The article clearly states that they "have no opinion or
information on whether Apple is a valuable company." So what _do_ they have
information about? The price history of the stock? The current contents of the
book? And on what basis do they make decisions? I guess answering that
question in too much detail would be giving up the "secret sauce," but I'd
love to know even in broad strokes how an agent that knows so little can so
consistently make money.

~~~
yummyfajitas
Usually the low latency stuff is kept pretty secret. However, I got good at
writing performing code, and as a result styloot.com is pretty fast.

 _The article clearly states that they "have no opinion or information on
whether Apple is a valuable company."_

This is an exaggeration. Some HFT's incorporate speculative mechanisms into
their strategy, supposedly a few people buy the twitter firehose and market
news and feed that into their strategy. But in general, HFT's run purely
technical strategies.

 _So what do they have information about? The price history of the stock? The
current contents of the book? And on what basis do they make decisions?_

All this sort of stuff. For example, if the book has 20 buy orders at $10.00,
and only one sell order at $10.03, that's an indication the price might be
going up shortly. But really, it's not all that vital to predict things.

The HFT is passively waiting with orders at $10.00 and $10.03 for someone else
to fill them - they make money when they sell to someone at $10.03, then close
their position by buying at $10.00. The trick is to avoid adverse selection -
if the market is likely to move up, don't get caught with a sell order.

------
ynniv
My understanding is that HFT has been used for frontrunning (which is
undeniably stealing pennies), and associated with unintended market
instability (flash crashes) due to unforeseen trader interactions. This post
primarily defends algo trading, which does not have the negative connotations
of HFT.

The author claims speed is mostly important for order precedence, but I find
this to be a naive view. The potential upside for a trader coming to a
conclusion and executing a trade first is minute compared to the potential for
frontrunning trades that have been executed but not yet communicated to the
market. Yes, this kind of frontrunning requires (sometimes[1]) automated
illegal eavesdropping on secure financial transactions, and is highly immoral
and illegal. But you don't make $100M a day without compromising some
morals[2].

Algo I'm okay with, but HFT is a fancy toy for messing with the markets.

[1 | [http://zerohedge.blogspot.com/2009/07/is-goldman-legally-
fro...](http://zerohedge.blogspot.com/2009/07/is-goldman-legally-frontrunning-
its.html)]

[2 |
[http://www.bloomberg.com/apps/news?pid=newsarchive&sid=a...](http://www.bloomberg.com/apps/news?pid=newsarchive&sid=a7HGVAn8w73Y)
]

~~~
azmenthe
Your association of a dollar amount to morality is unnerving. If Happy Unicorn
Widget Inc. upgrades it's production line speed, it experiences some increase
in revenue and can thus even make a $100M a day _gasp_.

I work at a HFT firm and any serious suggestion of frontrunning would probably
get you fired. The truth is the majority of HFT is liquidity providing or some
form of arbitrage.

~~~
ynniv
_Your association of a dollar amount to morality is unnerving_

That's a silly assertion. What if it were $100B a day? That's $35T/year, or
half the GDP of the world. Do you think that a company could receive half of
the productivity of the entire world morally?

 _I work at a HFT firm and any serious suggestion of frontrunning would
probably get you fired._

Another silly statement. Of course people don't talk about it openly - who
plans illicit activities openly? I would expect that anyone involved in such a
system would have no knowledge of it being used immorally/illegally - there is
too much risk in that.

HFT is an assault rifle. People who make them talk about how they keep the
peace, but that isn't the use that caused them to be outlawed.

------
jmn
One market structure alternative that is gaining some traction in US equities
is PDQ: <http://www.pdqenterprises.com/faq.html>

Basically, liquidity providers upload their market making algo to the matching
engine. Every time an order comes in, there is a 20ms 'mini auction' in which
market making algos compete on intelligence and not on speed.

------
rdlowrey
As a programmer with an economics degree, I'm thrilled to see articles like
these on HN ... popular media outlets do a terrible job of explaining how
these systems work and usually devolve into generalizations like "HFT is evil
and you should hate it" because they don't understand basic economic
principles.

~~~
MarkPNeyer
maybe they think basic economic principles such as "everyone is an equally
informed sociopath acting at all times to consume as much as possible without
regard to relationships" are absurd.

economics is a bullshit field.

~~~
rdlowrey
But math isn't. Everyone is entitled to their own opinion; mine just happened
to result from the mathematical proofs I was forced to study and write for the
postulations of that "bullshit field." I would also submit that your argument
ignores the law of leaky abstractions
([http://www.joelonsoftware.com/articles/LeakyAbstractions.htm...](http://www.joelonsoftware.com/articles/LeakyAbstractions.html))
... All non-trivial abstractions, to some degree, are leaky. Finally,
economics doesn't assume people "[act] at all times to consume as much as
possible without regard to relationships." It assumes that individual actors
make decisions that maximize their utility. Utility can result from completely
selfless acts -- it's whatever makes you happy. "Selfishness" does not rule
out actions that benefit others or society at large. This argument exemplifies
the misunderstanding of "basic economic principles" referenced in my original
comment.

~~~
nasmorn
That is funny because the proofs had the opposite effect on me. It dawned on
me that the math of e.g walrasian price setting is surely correct but the
story that this is supposedly how our economy works was much less believable
after each supposedly logical assumption (free disposal, no money pump) could
be traced back not to psychology or physical realities but conditions for the
equations to remain solve able.

Looks to me like a cryptologist predicting what passwords people would choose
based on the needed computing power to crack them. I wonder if 'password' made
the list.

~~~
rdlowrey
I tend to agree with your position. The fundamentals of individual
microeconomic decision-making are much more sound _IMHO_ than their
extrapolation to large-scale macroeconomic systems. At the macro scale I
believe the system is far more complex than we can correctly model with
anything but rough approximation. Otherwise, we'd already know exactly
when/why/how future recessions and booms would occur. The High Frequency
Trading question, though, exists _entirely_ within the micro sphere where the
math is rock-solid. As such, I've yet to encounter a logical refutation for
its use.

~~~
nasmorn
This is one of my favorite papers explaining possible problems with arbitrage.
<http://www.math.mcmaster.ca/~grasselli/ShleiferVishny97.pdf> It is not
directly applicable to HFT though.

------
scotty79
What's the social and economical value of precisely discovering the price of
casino tokens?

Stock market is economically important only when company issues new stock.
Apart from that it's just a huge casino where people like to play with their
or other peoples money.

~~~
chrisaycock
The secondary market provides liquidity to investors who participated in the
initial company stock issuance. Without a robust means for selling their
shares at some point in the future, investors would be loathe to provide
capital in the first place.

~~~
MarkPNeyer
is there any evidence for this argument? would investment just stop
alltogether, or would it change to support lots of smaller ventures instead of
a few big ones?

~~~
wtvanhest
Answer this question and you will need no evidence:

What would a stock be worth the day of the IPO if there was no way to sell it
the day after the IPO?

------
tedunangst
At the very end, Farnsworth buys from, not sells to, Bender. I think it's
clear, but had to read it twice.

~~~
yummyfajitas
Thanks. Fixed it.

~~~
daltonlp
One more: "Fry - all he knows is that he bought 100 shares @ $10.00 from
anonymous counterparty."

Didn't Fry _sell_ 100 shares to an anonymous counterparty?

~~~
yummyfajitas
Yes. Thanks. Arrggh, so many people reading my bad writing.

------
kylemaxwell
I have little to add on the substance other than to note that I find these
posts (and the discussions here) incredibly informative. They shed light on an
area that turns out to be far more interesting than I would have imagined.

However, the last example may be more realistic than you suppose. DEF CON last
year contained a presentation[1] on security considerations in HFT networks,
which I attended. It's thought-provoking and something I hope gets a little
more attention.

[1]: <https://www.youtube.com/watch?v=kjIdzBtTBnI>

~~~
tptacek
What did you learn from this? I found the slides and read them, and it seems
like he's just saying "the people who build these systems believe that
firewalls and TLS add too much latency to be deployed".

(I have a lot of opinions on this subject but I am mercifully restricted from
sharing most of them owing to professional obligation; we do a lot of work in
this field).

~~~
kylemaxwell
Mostly, insight into the technical underpinnings. I had absolutely no concept
of the existence of this world before that talk. In terms of take-home value,
not much more than I got from your talk at BH last year, though largely
because of the fire-hose effect there. :)

~~~
tptacek
If you're looking for good talks on the security or technical implications of
trading markets, look for microstructure details. Any talk that has a diagram
of "orders" going to a "trading engine" is addressing itself at a higher level
than you're interested in. Just the order entry side of a real firm OMS is too
complicated to get one bubble in a diagram.

Again: strongly recommend _Trading & Exchanges_ by Harris. The TCP/IP
Illustrated of markets. Supremely readable.

~~~
RickHull
Funny, I just bought this on Kindle a week ago. While not exactly a page
turner, it is indeed supremely readable. I'm about halfway through and remain
fascinated.

------
jfager
This is a great series so far, but I think some people are taking it too
seriously. This is roughly the equivalent of giving you a good tutorial about
how a short map-reduce implementation works and then asking you to agree that
Hadoop is awesome. It may or may not be, but the toy example isn't sufficient
to make the determination. It's just background information for the
uninitiated.

Some relevant questions that are completely glossed over:

1\. What compels an HFT to actually trade? Is there anything forcing them to
keep supplying liquidity even if the market's moving against them? How is an
HFT different than an actual market maker?

2\. How does an HFT decide that it has a better-than-even shot at turning a
profit on a trade? Most of the objections to HFTs revolve around the answers
to this question (i.e. pseudo-front-running by trying to detect large
buys/sells that get split over lots of orders) and their implications (i.e.
'real' investors leaving the exchanges).

3\. The "market-maker strategy" HFTs you describe are indisputably compensated
for providing liquidity and taking on risk, but is the return on HFTs actually
equivalent to the return on other investments with equivalent risk? If not,
and they earn a premium, why isn't that evidence that something's broken?

~~~
yummyfajitas
_It's just background information for the uninitiated._

That's all it's intended to be.

 _Is there anything forcing them to keep supplying liquidity even if the
market's moving against them?_

No.

 _If not, and they earn a premium, why isn't that evidence that something's
broken?_

Wait for part 3. I do believe HFTs are capturing fractions of a penny in rent
and I have an idea of how to fix this.

------
mykolasmith
Thanks for following up on this. Nothing irks me more than "Pt. 1" posts,
never to hear from the OP again. As an undergraduate scheduled to be doing HFT
systems development at a major bank this summer, I find these posts very
useful!

------
harryh
Additional evidence that HFTers are not stealing from speculative investors:

Anyone can start a market. If HFTers were stealing pennies someone would have
come along and started a market that banned HFTers (or changed the rules to
otherwise get rid of them). Then all of the speculative investors would use
this market instead since they could be assured that no one was siphoning off
pennies on every trade. Eventually markets with HFTers would shut down due to
a lack of customers.

The fact that this hasn't happened has to be considered evidence that the
HFTers are, in fact, providing value to speculative investors not the other
way around.

~~~
sailfrog
I'm not sure reg NMS allows this. Registered exchanges publish protected
quotes that other venues are not allowed to trade through.

~~~
boxy_brown
> I'm not sure reg NMS allows this.

This is correct. It would be extremely, unequivocally forbidden under Reg NMS
to create a quote-disseminating market center with the proposed property.

------
tolos
What makes me uneasy about HFT is the speed in which things happen.

For example, I believe the BATS IPO that happened last month began trading
around $15, and was below $1 in about 900ms before trading was halted.
(Nanex.net has news postings about these kinds of things, and I wouldn't mind
hearing other people's opinion about the site)

I guess my question is, is it possible for traders to make money in the short
term (hours/minutes) or is that forever in the land of robots now?

~~~
sailfrog
Insofar as Nanex is concerned remember they are attempting to sell you a
product, so be aware of their motivations before swallowing any
unsubstantiated assumptions. With that said from what I have seen their charts
are accurate but the interpretation of what they mean is sometimes a bit
dramatic :)

I can't speak directly to the BATS IPO debacle but I can say that when the
NBBO in a symbol is locked or crossed (market speak for the bid and ask being
the same or being inverted) matching engines can ignore the NBBO. This results
in trades being executing at prices all over the map and can explain how a
stock can go from $15 to $1 almost immediately.

------
wtvanhest
_In the Hacker News comments on part 1 of this series, there were several
comments suggesting that HFT’s somehow steal pennies from ordinary investors.
I don’t agree with this claim, but I’m very interested in hearing well
reasoned disagreement._

I would go past this and say that anyone who did come up with a "logical"
argument would be ignoring time value of money.

------
Anm
After reading both articles, it seems to me much of the race for faster
trading speed is fueled by the first come, first serve order matching.
However, I don't see how this is actually important to either price discovery
or liquidity. In other words, this artificial restriction does not actually
contribute to greater good of the stock market.

I presume it was designed to ensure fairness, but is it any more 'fair' than
random selection (true random, from a physical source that cannot be
influenced) from all available matching bids.

Would anyone care to present an argument on how FIFO matching, after best
price, is beneficial in the broader sense?

~~~
harryh
Let's say I want to sell 100 shares of MomCorp at $10. Bob also wants to sell
100 shares of MomCorp at $10.

If, instead of employing first come first serve you use a random selection
then both of us have an incentive to tell the market that we actually want to
sell _more_ than 100 shares to increase the odds that we'll actually get to
sell what we want. Various bad/unpredictable/unstable things start to happen
in this sort of situation.

FIFO matching leads to a stable market.

------
hartror
I don't have any more or less dislike for HFT over more traditional trading.
My biggest problem with finance is the brain drain it creates, in HFT's case a
lot of great maths and computer science people get sucked into the finance
black hole which saddens me greatly. Certainly having liquidity in the market
is important but what level of liquidity is enough and beyond that no more
value is provided to society and so the brain drain in fact holds back
society?

------
albertsun
Going off the last example, what difference does it make if there is a trojan
or not? Bender might very well have chosen to buy at $10.10 and sell at $10.15
without any evil foreknowledge to narrow the spread and it would have caused
the some effect on Prof. Farnsworth.

Yes, it does mean that whoever had a sell order at $10.10 gets their order
filled faster. But is that trade off worth it?

~~~
yummyfajitas
_...what difference does it make if there is a trojan or not?_

Absent the Trojan, Bender has no idea if Farnsworth will show up or not. If
Farnsworth doesn't show up, this might happen:

BUY(bender, $10.10, 100, 12:00:00.000)

SELL(bender, $10.15, 100, 12.00:00.100)

...crickets chirping...

SELL(hermes, $10.05, 100, 12.00.10.000)

SELL(zapp, $10.00, 100, 12.00.15.000)

BUY(amy, $10.00, 100, 12.00.16.000) (Amy trades with Zapp)

...MomCorp continues it's downward slide.

I.e., without the trojan Bender is just a guy who thinks MomCorp is going up.

------
benthumb
My 2 cents:

1\. Productive is Destructive ... literally (that's why we're in the position
now of having figure out a way to mine asteroids -- industrial productivity
under the current model is environmentally unsustainable)

2\. Markets are (viewed in aggregate) evolving to be something akin to a
global autonomic nervous system ... money like transduction makes the world go
'round.

------
ChristianMarks
Liquidity has gone _down_ with the advent of HFT:
[http://opusminimax.wordpress.com/2012/04/20/ardent-
support-f...](http://opusminimax.wordpress.com/2012/04/20/ardent-support-for-
hft-on-hacker-news/)

~~~
anonymoushn
You don't seem to provide any evidence of this claim.

~~~
yummyfajitas
Indeed - the author seems to confuse trade volume with liquidity.

~~~
ChristianMarks
The author gave you the benefit of the doubt by not spelling out the essential
relation between liquidity and volume over time. Just what exactly do you
think liquidity is, if it has nothing to do with volume over time?

~~~
anonymoushn
It is the ability to transact, rather than the fact of having transacted.

~~~
ChristianMarks
This is imprecise. OK, let's say that liquidity has gone up in the long run,
thanks to automation (and to some extent, regulation). This reference
<http://www.tau.ac.il/~azibenr/Liquidity_BKW.pdf> shows that more liquidity
has not helped: "...the effect of each unit of liquidity on returns has
declined over the years." So one cannot flatly assert that HFT has led to
improved liquidity and that this benefits everyone. And still one cannot
discount the effect of decreased volume: this indicates that retail investors
have left.

Here's what that incontrovertible reference, the Wikipedia, states:

 _Another elegant definition of liquidity is the probability that the next
trade is executed at a price equal to the last one._

The Wikipedia also "confuses" liquidity with volume: _The liquidity of a
product can be measured as how often it is bought and sold; this is known as
volume._ Source: <http://en.wikipedia.org/wiki/Market_liquidity>

------
Schwolop

      To illustrate with arbitrary numbers, if HFT’s currently focus on 50% strategy (i.e., price competition and liquidity improvements) and 50% latency, a more optimal scenario would be 50% fewer HFT’s focusing 100% of their efforts on strategy.
    

Nitpicking - "more optimal" is non-sensical. Optimality is binary. You're
either optimal or you're not. Of the sub-optimal cases there are better and
worse options, but neither is "more optimal" than any other.

------
zopf
Some concrete examples of ways that HFTs inhibit, disrupt, or defraud market
participants:

[http://www.chrisstucchio.com/blog/2012/hft_apology2.html#com...](http://www.chrisstucchio.com/blog/2012/hft_apology2.html#comment-510202854)

------
rscale
I have no objection to the provision of liquidity. That said, the flash crash
seems to me to be a perfect example of a danger created when liquidity is
provided largely by algorithms.

We ran into a situation where the market was already volatile, and a bad trade
exacerbated the issue by causing a number of HFTs to take unexpected losses
and withdraw from their markets, consuming further liquidity while driving
prices down, which created more losses for the remaining market-makers, who
had to close their positions, consuming further liquidity, driving prices
further down; all of those also negatively affecting long-term investors.

This also seemed to me an example of the opportunism of HFT, where the HFT
shaves the spread by a penny or two during calm markets, but withdraws (and
exacerbates issues) during volatile and troubled markets, which seems to me
the point in time at which liquidity provision is most valuable.

I'm not suggesting that HFT should be outlawed, nor that HFT firms should be
forced to register, act, and be regulated as official market-makers, with the
associated duties.

But I do note the benefits seem to come with costs.

Long-term, I doubt it matters. It seems inevitable that the provision of
liquidity will become commoditized, and that the days of concerns about flash
crashes will eventually disappear into the past along with $50 retail trades,
and $0.50 bid/ask spreads.

~~~
yummyfajitas
Flash Crashes are not a phenomenon caused by algorithms. We have actually had
two flash crashes - the first was in 1962.

[http://online.wsj.com/article/SB1000142405274870395760457527...](http://online.wsj.com/article/SB10001424052748703957604575272791511469272.html)

Also, the main reason many HFTs pulled out of the market is the risk of broken
trades (regulatory risk [1]). Staying in the market would have been a big
moneymaker absent that risk - spreads were often huge.

But broken trades were dangerous. If you buy accenture at $1.00, and sell at
$30.00, you've helped fix the flash crash. You also just lost $10.00 - your
$1.00 trade was broken, and you now have a short position you bought at $30.00
(Accenture recovered to $40.00, so you lost $10.00/share).

[1] Regulation by the exchanges, not the SEC. Maybe there is an SEC regulation
mandating they do this, but I have no knowledge on that point.

~~~
rscale
Your described scenario with Accenture doesn't describe a market-making HFT
strategy. You're describing something akin to a mean-reversion algorithm that
would be MFT or slower, and is not a market-making strategy. It demands that
you buy and hold inventory to profit. It doesn't provide liquidity.

Of course you can make a huge profit when a crash occurs, whether that crash
is due to a vicious circle of algorithms, or a vicious circle of human
psychology. There are algorithms out there that look to do just that, trying
to profit from exploitable market anomalies, and it's great that those people
have found a way to get paid for fixing some problems. But they're not HFT
market-makers, they're a different group of quant/algo traders.

Personally I care little about HFT. The flaws in the technology will get
ironed out; the competition for the low-hanging fruit will continue to
intensify, and eventually many of the functions will become commoditized as
they mature.

In the meantime, I think it makes sense for HFT market participants to be
sensitive to the fact that many individual market participants have trouble
identifying the value they've received because of HFT participation, but can
clearly remember fears that have been induced by HFT driven events.

~~~
yummyfajitas
_Your described scenario with Accenture doesn't describe a market-making HFT
strategy._

The HFT could have placed a passive buy order at $1.00 and a sell at $30.00
(or at $2.00, which he revised upwards as the price corrected).

------
DannoHung
Why not explain what's going on here and how it's not defrauding markets:
[http://www.zerohedge.com/news/step-right-its-hft-whack-
mole-...](http://www.zerohedge.com/news/step-right-its-hft-whack-mole-time)

~~~
yummyfajitas
I discussed this phenomenon before - it's basically just poorly written
algorithms behaving oddly.

<http://news.ycombinator.com/item?id=1564445>

Note that your article provides no explanation as to who is being defrauded
("markets" isn't a person) or how, so I don't know what you want me to
explain.

~~~
tptacek
What is your take on Zero Hedge? It drives me sort of bananas, but I'm not a
pro.

~~~
yummyfajitas
It's like a conspiracy theory site but without the theories.

------
briholt
Fantastic series, please keep them coming.

------
leoh
How is this an apology? Have I missed something? It sounds like this guy is
giving tips for how to do this shit yourself.

~~~
tedunangst
Apparently you missed the entire second paragraph. Or you suck at reading.

~~~
leoh
No, you're a moron. I'm not getting into a debate with someone online, but I
think that a good apology would actually apologize. Then, if so inclined, the
apologizer might go to lengths to prevent the egregious act from being
committed again. Instead, this writer enables others by giving insider
knowledge of how to commit unethical acts. Thanks for reading between the
lines, though, asshole.

------
AznHisoka
Why apologize? Noone's robbing anyone here. If it's legal, it's legal.

~~~
rauljara
I can't speak for everyone, but you are probably being downvoted for two
reasons:

1) Apologia does not meet "I'm sorry". Read the first few sentences of the
article for an explanation.

2) I shudder to think of a world where everyone felt the only criteria for
whether it was all right to do something was legality. We should all be
grateful that most people don't go through life with that mindset.

~~~
AznHisoka
I knew what he meant by apology - it was an explanation on why HFT is not
unethical. However, I felt writing such an essay showed the author is insecure
and has doubts about his profession.

~~~
kylemaxwell
It's an intellectual, not ethical, defense, as he explains in the second
paragraph. His defense has nothing to do with remorse and everything to do
with helping people understand the concepts so they can participate in an
informed debate.

