
Watch High-Speed Trading Bots Go Berserk - cpeterso
http://www.technologyreview.com/view/428756/watch-high-speed-trading-bots-go-berserk/?ref=rss
======
zacharyvoase
When I was in high school, my math teacher instilled me with the value of
always labeling axes when plotting graphs. This GIF provides us with exactly
no information on anything, but it sure _seems_ scary enough. The line is
getting higher and more erratic! This can’t be good!

Let’s piece apart some of the language here:

* “High-Speed Trading Bots Go Berserk” — doesn’t mean anything, they’re processes, they’re just following their algorithm.

* “…unnerving the financial markets.” — doesn’t mean anything, ‘markets’ don’t have nerves. In fact, I once wrote about the dangers of reification: <http://blog.zacharyvoase.com/2012/03/31/reification/>

* “activity becomes much, much more frenetic and erratic” — to a human observer. These algorithms don’t have concepts of ‘frenetic’ or ‘erratic’; neither does the universe.

* “…so complex that its potential systemic repercussions are literally unknowable” — suggests that systemic repercussions of _anything_ were ever ‘knowable’. If they were, we wouldn’t have markets. c.f.: <http://mises.org/daily/2608/>

Don’t just read. Think. Thanks.

~~~
mvzink
Yeah, but you have to admit that this is pretty skilled sensationalist
journalism :P

How convenient Nanex didn't include labels in the graphics they provided! [1]
In fact, they specifically say that high frequency trading doesn't concern
them, and are talking about quote spamming. Not. The. Same. The animated graph
used in the article combines metrics of both, as far as I can tell, but Nanex
makes it clear that High Frequency Trading is stalled or declining while High
Frequency Quoting is on the rise.

Furthermore, the bit saying "activity becomes much, much more frenetic and
erratic" is just... totally silly. Partly because of what you mentioned, but
also because... well I don't know what else to say, it's just so utterly,
embarrassingly silly. I'd wager the graph's movement isn't even nearly as
'frenetic' as the stock market itself.

In any case, it's quite clear that the author didn't even peek at the page he
got the graph from.

Ugh. This kind of 'journalism' makes me sick.

[1] <http://www.nanex.net/aqck/2804.HTML> linked all-too-subtly in the article

------
yummyfajitas
Lets stop and think for a moment about what we are discussing. Knight gave
other people about $400M by making some bad trades. Also, some graphs wiggled.

The net impact on the market was negligible. If you buy on the timescale of
days to weeks, you can completely ignore HFT. Here is a blog post with graphs
demonstrating the market impact of various HFT events:

[http://www.chrisstucchio.com/blog/2012/flash_crash_flash_in_...](http://www.chrisstucchio.com/blog/2012/flash_crash_flash_in_the_pan.html)

~~~
MarkPNeyer
the technology used by HFT firms to make markets in microseconds makes the web
development world look like children.

pubsub is the hot new thing here in the valley - welcome to wall street, circa
ten years ago.

asynchronous is the new synchronous? what is that - a new album by smash
mouth?

if you completely ignore HFT, you'd think there would be way more programmers
and mathematicians available for hire.

~~~
Aloisius
As someone who has designed, built and ran systems that operated at a scale
that made the software that HFT firms look like toys and built several web
companies, I can honestly say that you have absolutely no idea what you are
talking about.

Web development and low-level systems engineering are entirely different and
neither one is particularly harder than the other. If anything, both are
simply tedious.

And pubsub was old by the time wall street got a hold of it.

~~~
MarkPNeyer
yes, pubsub is an old pattern.

what HFT experience do you have that makes you think of those systems as toys?

a webpage that responds to requests in 10 milliseconds is fast. a trading
engine that responds to requests in 10 milliseconds is a joke.

~~~
true_religion
It's not as if many webpages couldn't be made to respond to faster.

Not to brag, but I've built trading engines in the past year that can process
20 years worth of data in less than a ms.

Now I'm a founder at your usually derrided photos-sharing start up. Sure we
could make most requests come out of the server in less than 2ms, but what of
it? It's still going to take the user upwards of 1.5 seconds to load all of
the content, and at least 100ms spent on the wire. Working at a consumer level
startup, there's no point in optimizing the server yet when network costs are
so high, and there's no point in improving the network when there's no ROI in
it.

\----

Answering to his question: > what HFT experience do you have that makes you
think of those systems as toys?

I'd imagine that massive distributed systems are more complex to think through
that HFT systems because the later are more monolithic due to speed
constraints.

~~~
MarkPNeyer
HFT systems _are_ massive distributed systems.

where i worked - a small shop with 10 engineers - we had 350 machines running
in 5 different data centers in chicago, new jersey, and korea. each machine
ran around 100 different processes, which were coordinated using a combination
of several different pubsub systems:

\- in house reliable tcp-based broker mediated (we called it pubsub3 )

\- in house unrelaiable ip-multicast system (i named it blue ray because i
thought that sounded cool)

\- 3rd party implementation of a PGM derivative that we licensed for god knows
how much

the engines subscribed to various topics on the system, processed market ticks
(sent by marketdata publishers over blue ray) and fired orders at whatever
instrument they were handling

the risk engines modeled risk scenarios inresponse to trading shifts, powering
through the risk models our quant trader wrote in c. we had a 'calcparser'
system that constructed a DAG rerpresenting the AST of the c code and
distributed the DAG across several machines.

shit was nuts. i miss it.

~~~
photorized
>HFT systems _are_ massive distributed systems

The complexity is also in having to deal with limited resources, and factors
outside of your control. Have you seen many bootstrapped HFT implementations?
The budgets are just... different.

When you spend 10x or 100x hardware, get massively redundant backbone
connectivity (connecting to a tier 1 core router directly, and not through an
oversubscribed shared switch port that a typical bootstrapped startup would
get), you end up with a much better controlled and somewhat predictable
environment.

With a typical "web" (non-HFT) distributed system, you've got millions of
"subscribers" aka users - and I don't think people outside of IT fully realize
just how bad a typical user's [Windows] machine is messed up - and that was
before they installed a couple of toolbars, adware, firewall, and AV
products... And all of these subs are tying up your publishers in their own
unique screwed up way, keeping sessions open, dropping packets, blocking
ports...

I've designed global CDNs, and operated one for a decade - and the kind of
randomness our distributed systems have to deal with are not always technical,
even though we don't typically count microseconds.

------
curiousDog
Any idea how much Knight ended up losing? I remember interviewing with this
company, Allston Trading in college. They were the only ones who showed up at
the Career fair with the salary printed out on brochures. $100k for a fresh
java developer. During the interview in their plush offices in the Chicago
stock exchange, they were casually discussing how they ended up losing
$5million in forex last night. All of them were young, brash and looked like
they had little regard for investors money. One trading guy was telling us how
he flies to vegas just for the weekend to get with hookers (during a lunch
interview!).

The managers seemed like your typical alpha-male..bullish, dominating and
little regard for morals. Although some computer scientists that they had
seemed brilliant and were genuinely interested in building safer, better
algos. So i wouldn't be surprised if it's these wall st. types that are
pushing the limits further even if deemed unsafe.

~~~
theorique
Consensus number seems to be $440M in losses on the bulk liquidation of $4.5B
in unwanted positions to Goldman Sachs. This represents about a 10% loss.

GS does bulk liquidations like this and typically takes a haircut of 2-5%,
presumably based on a risk and liquidity assessment.

------
danso
I know HN's frontpage has had a few posts dealing with what financial industry
coders do and the validity of it. The ones who claim to be HFT-bot coders
argue that they're highly-skilled coders...but it was not always clear if
their skill is in algorithm-implementation, or in engineering. That is, they
may be great at working with the traders in coming up with a highly performant
bot, but they have less ability in exception-handling and design, which may be
things that their bosses don't really think about. (I honestly don't know, was
just wondering if someone could competently recap the relevant discussions.)

~~~
radikalus
Most everybody I know (in the field) is in disbelief concerning the length of
time the Knight disaster unfolded over. I've had systems "loop" in BAD ways,
but never for more than 1-3 seconds.

Safeties are paramount in the field. There's many layers of protections to
avoid *ing yourself. Most of these are deployed in ways that are not terribly
expensive on the 'critical path' -- that said, there are a LOT of people doing
REALLY stupid things in the field. (As there are in any field, it's just more
'scandalous' when an error in a financial application occurs)

Algorithm implementation is largely trivial, and the high pricetags paid
developers in the industry are usually for specific engineering skills along
with the semi-requirement that developers be 'all-hands-on-deck' when the
situation calls for it. (There's a bug in the new strategy...well you need to
fix it before tomorrow's open etc) At least, that's how I see it.

------
hexagonal
Blogspam. OP is [http://blogs.reuters.com/felix-salmon/2012/08/06/chart-of-
th...](http://blogs.reuters.com/felix-salmon/2012/08/06/chart-of-the-day-hft-
edition/)

------
TomGullen
If the chart is showing frequency of trades it's just showing that there are
more trades right?

I find it a little misleading that the graph is presented as somehow being
connected to market instability. If they are correlated, they haven't
demonstrated the relationship clearly to me. Am I misreading this?

Also I fail to see how someone else's algorithm going nuts could impact
regular Joe traders as the article also suggests. If a stock price starts
plummeting because of a bug in a trading algo goes haywire, a savvy trader
would look to their news sources and realise nothing has happened. Or if they
are passive investors probably by the time they mobilise to panic sell it's
already corrected itself.

------
rgarcia
Anyone know what the y-axis measures? The original Nanex source doesn't label
it either[1].

[1] <http://www.nanex.net/aqck/2804.HTML>

~~~
lt
Looks to me like the time of the day.

~~~
rgarcia
That's the x-axis.

------
kerryiob
Most of comments here are missing the big picture.

Most of the general public and individual investors do not have clear insight
into how HFTs work, or if they are beneficial or not. Most are inclined to be
scared of crashes and erratic behavior traced back to HFTs. Trading off
liquidity for a loss of trust in capital markets is a bad deal.

When investors lose trust in the system, it all falls apart.

------
powertower
Do HFT firms have contracts with larger banks to take their positions with a
discount in situations like this, so the stocks are not imidiatly dumped on
the market causing havok? Or to insure them against loss?

Sounds like we could segment, rate, and pacakge these contracts up into
insured "credit-default-swap" type-of vehicles and make money
making/selling/buying/trading them.

------
retrogradeorbit
Tech journalist discovers zerohedge. They ran this story back in February.

[http://www.zerohedge.com/news/presenting-rise-hft-machine-
vi...](http://www.zerohedge.com/news/presenting-rise-hft-machine-visual-
confirmation-how-skynet-broke-stock-market-us-downgrade-day)

------
joering2
This should be illegal long time ago -- think Captcha for buying/selling
stocks.

The real issue here is that -- whoever was in charge of those stock exchanges
hasn't properly addressed this issue as we all can see now its been growing
slowly but constantly since 2007.

~~~
pavel_lishin
> think Captcha for buying/selling stocks.

I like that the side-effect would be a drastic improvement in OCR technology.

~~~
radikalus
This would be awesome. (Time to bust out your DBNs)

------
ShabbyDoo
I'd like to see a graph of average spreads between 2007 and now.

------
tectonic
Does HFT create value in any way? It strikes me as purely driven by greed and
so utterly irresponsible for society as a whole.

~~~
slapshot
As has been pointed out a few times in connection with this incident, Knight
is largely a market-maker (not an HFT firm): Knight keeps open quotes on both
buy and sell sides of transactions at all times and provides liquidity. Their
strategy to make money is the "spread" -- they price the buy a couple of cents
lower than the sell and on average they end up buying a couple cents lower
than selling.

They are part of the infrastructure of the major markets; a company listing on
NASDAQ or NYSE is required by the market to choose at least one market-maker
who will be designated to always keep open buy and sell orders on the books.

As it happens, Knight does its market-making through computers; this replaces
sweaty men who would stand in the "pit" of the trading floor and trade by
waving fingers at each other (including, occasionally, the middle one). It
looks like HFT in that computers trade quickly, but it is intended to be a
liquidity-providing service to the market: there is always somebody willing to
buy or sell anytime the market is open.

(In contrast, HFT does not require always-open buy and sell quotes.)

~~~
white_devil
Yeah, "liquidity" is the standard bullshit distraction trotted out every time
HFT is discussed here. Strangely enough, we've got "yummyfajitas" defending it
again, which maybe, just maybe, _might_ have something to do with the fact
that he _works in HFT_.

As for liquidity, since the bots only make trades with the goal of shaving off
tiny little slivers of profit on millions of trades, why and how would one of
those sweaty guys ever buy anything from them? Even if one wanted to, the
opportunity to buy might pass in a couple of microseconds.

All in all, working in HFT or even the financial "industry" at large, is
complete bullshit. No self-respecting, decent human being should do it. No
matter how you look at it, your mission there is to help world-raping scumbag
bankers and the like make more money. They certainly don't need it.

~~~
yummyfajitas
I no longer work in HFT.

As for your other questions, I suggest you go read my blog posts on the topic.
They answer all your factual questions.

~~~
white_devil
> I no longer work in HFT.

Finance still, though? I bet it's difficult to let go of the fat paychecks.

> As for your other questions, I suggest you go read my blog posts on the
> topic. They answer all your factual questions.

Ah yes. I was just _giddy_ with delight when you used Futurama character names
in those examples. How could I _not_ love everything HFT represents after
that?

~~~
yummyfajitas
_Finance still, though? I bet it's difficult to let go of the fat paychecks._

Your guesses as to my personality type and employment situation are just as
wrong as your speculation about the nature of HFT.

~~~
white_devil
> Your guesses as to my personality type and employment situation are just as
> wrong as your speculation about the nature of HFT.

That's easy for you to say. We're both human though. Humans are above all
selfish, and have a hard time letting go of easy money.

So who knows, if I were some kind of math wizard living in the US, I might
have ended up working for a bunch of sociopaths too. It's possible I'd be here
on HN, rationalizing and defending HFT and blowing smoke up people's asses
with cutesy link-bait-titled posts about it.

I didn't actually say anything about your _personality type_ though.

We shouldn't even bother, but perhaps you'd care to point ouf my mistakes in
describing "the nature" of HFT?

------
sprobertson
If computers are running the markets, and the markets have such a large
influence over society, does that mean the computers have already quietly
started their takeover?

~~~
angersock
what does that even mean go away

------
ck2
I don't know stocks but this strikes me to be the same mindset as domain
tasting and should be ended entirely, not regulated, just killed, regardless
of any claims of industries being built around it - it's just wrong.

------
samstave
HFT should b illegal. Period.

What value is there in any system where trades over "pennies millions of times
per second" is provided at all to real world conditions/humans.

This is simply a tool by a few to make money in an invisible, unregulated and
uncontrollable space.

This is simply not sane.

~~~
scrumper
That's rather simplistic, not to mention completely wrong. US markets are
highly visible (you just saw a free visualization of HFT activity since 2007),
very tightly regulated and well controlled. Things go wrong, as they do in all
complex systems, but rarely do they go wrong to the extent that ordinary
investors are harmed.

Read this, before jerking your knee any further. Think of it as turning on the
bedroom light, showing the monster in the corner to be nothing more than a
pile of clothes:

<http://www.chrisstucchio.com/blog/2012/hft_apology.html>
<http://www.chrisstucchio.com/blog/2012/hft_apology2.html>

Financial markets have been moving towards zero latency trading since their
inception. First, men gathered under trees or in coffee houses, within
shouting distance of each other. They fought over the best spots. Then,
traders became some of the earliest adopters of telegraph technology, running
private lines from remote cities to the exchanges. Ticker tapes appeared in
distant offices. Computerized price dissemination followed, together with
electronic trading connections to the exchanges themselves. All in the name of
getting information and acting on it before the next guy. It's the reason you,
as an individual, can trade global markets cheaply and instantly. It helps you
take control over your retirement and savings, rather than paying through the
nose for some managed scheme where only a privileged few can access the
markets.

Where is your arbitrary line where progress stops?

~~~
samstave
I think you are arguing something different than I. I am open to being wrong,
but I think I am making a different point.

You are saying that by having a zero latency trading capability, financial
markets are the sign of progress and it is an inevitability that, not only is
desirable, but preferred.

That's fine, but not what I am am taking issue with is fully automated system
of extremely high volume, low margin, trades that are conducted by bots and
have no human interaction other than those that are profiting from them.

I admit I am not savvy enough in this area to argue about whether or not HFT
provides more equilibrium to the price of a stock, but I don't see that as a
wholly convincing argument as to why HFT is important and adds value.

I am skeptical of that claim, mostly due to my ignorance, so please educate me
on the following: The claim that HFT rarely goes wrong such that "ordinary
investors" are harmed; who then is the non-ordinary investor who is benefiting
from the HFT, and what value are they providing in the markets, other than
profiting from HFT.

It appears to be making the claim that HFT is a meta market that the ordinary
investor shouldn't even be concerned with except in cases of extreme rarity
where a flaw causes them some financial harm/risk.

Again, if this is true - what true concrete value to the world does this meta
system provide? I simply cannot see it, and again, I claim awareness of my
ignorance, so please explain like I am five.

Thanks!

~~~
jamii
> ...and what value are they providing in the markets...

It used to be that if you wanted to buy a book you would have to wander around
town until you found a place that had chosen to stock it. If it was an obscure
or specialist book you might have to drive to a different town. You might just
be shit out of luck. Nowadays you search for the title on Amazon and it shows
up at your door the next morning. When you are done you can sell it again on
Amazon with minimal hassle. Amazon improved the efficiency and liquidity of
the book market which is why they are rich, even though they are charging less
overhead than the bookstores.

Your local corner store doesn't create anything. All they do is buy from
manufacturers and sell to you at a markup. But you don't care about spending
an extra few pennies when its 6am and you're out of milk. You just want
breakfast now.

Liquidity providers make it easier to buy and sell financial instruments
whenever you want. HFT enables firms to provide liquidity more efficiently and
with less risk. Just like Amazon, they get rich by saving you money.

