
A summary of what quantitative trading firms do - hendzen
https://blog.headlandstech.com/2017/08/03/quantitative-trading-summary/
======
Dowwie
A vital enabler of these firms is leverage through margin financing. There
wouldn't be quant trading firms without leverage, extended by a prime broker.
Larger firms deploy fairly interesting, complex arrangements across multiple
prime brokers. You don't want a pb to know your entire trading strategy. The
PB, on the other hand, wants self-funding accounts, featuring hard-to-borrows
and other types of valuable securities as collateral that can be
rehypothecated and used by its securities lending businesses. Finding a viable
leverage policy across multiple prime brokers is a nice, juicy challenge for
the problem solvers.

~~~
aldanor
> There wouldn't be quant trading firms without leverage, extended by a prime
> broker.

Please allow me to disagree :) Most larger pure-algo HFT prop shops are self-
clearing, they are exchange members (possibly with more favorable fees due to
market making) and don't use pbs, instead relying on their own co-located
execution infrastructure and funding.

~~~
turingcompeteme
This is true for, as you said, large hft shops. But there are thousands of
quant trading firms that are not on this scale and will not be an exchange
member. You can see all the NYSE exhchange members here [0], and it's a very
small portion of all quant funds.

Managing margin and leverage between prime brokers is an important job for the
backoffice at the fund I work for. These numbers are watched very closely, and
we have strong, multiyear relationships with the people who work at these PBs.

[0] [https://www.nyse.com/markets/nyse-
arca/membership#directory](https://www.nyse.com/markets/nyse-
arca/membership#directory)

------
zbentley
This is substantially higher information density than anything I've read in
the last week. Learning about triangular arbitrage programmatic trading alone
looks like it's going to occupy most of the next few days for me.

~~~
seanp2k2
Btw, people are doing lots of arbitrage with crypto currencies as well if you
weren’t already aware. It could be a fun way to play with it without spending
much if you’re interested.

~~~
S3raph
the problem with cryptocurrency arbitrage is that the profit depends on how
fast/reliable the various exchanges process your transactions and how much the
transaction fees are and not how sophisticated your algorithm is.

nice read: [https://steemit.com/arbitrage/@kesor/the-math-behind-
cross-e...](https://steemit.com/arbitrage/@kesor/the-math-behind-cross-
exchange-arbitrage-trading)

~~~
alphydan
Maybe nit-picking ... but shouldn't a very sophisticated algorithm precisely
incorporate the reliability/speed of the various exchanges (maybe based on
past and current latency or downtime which weighs the risk of a transaction
not clearing at a given exchange). The smartest algorithm should be able to
win such a battle on average.

------
frankc
This is a good write-up but it's also missing huge other sections of quant
finance like statistical arbitrate and factor investing. I guess you could put
this under the 'market taking' section except everything he describes is still
in the mode of single stock thinking while these strategies are more about
portfolios. You typically estimate some factor model of the market and use a
portfolio optimizer to create your trade baskets.

In these types of strategies you care about latency but on the order of
milliseconds not microseconds. The challenge is in building multi-day
predictive alpha models with low correlation to each other, getting good
executions though brokers can do a reasonable job these days, and especially
combining alphas into one book which requires sophisticated mathematical
programming (conic programming etc)

------
human803658052
Many companies make a profit by doing something that benefits society in at
least some way, society pays them for their service. E.g. selling electronics:
the company makes a profit, but society benefits by being able to buy and use
those electronics.

What about quantitative trading firms. They make a profit themselves. But how
do they benefit society?

~~~
lordnacho
Here's a very simple service that everyone would understand.

Suppose you are a customer with a connection to an exchange. You can't afford
to connect to every exchange, because that would cost you a small fortune.

If you stick your order into this exchange, you might have to wait around for
someone to meet your price. In fact, there might be someone willing to trade
with you at your price, but you don't know it because you're on different
exchanges. (Let's not get into NMS.)

An HFT with a view of the whole market would help you distribute your order to
all exchanges. They do this by leaning on your order while posting everywhere
else. So if someone came in and met the HFT's order in some market, the HFT
would turn around and fill you immediately for a small difference, and they'd
pull all the other orders when this happened.

That's a service, and you pay for it.

A lot of HFT strategies provide a service similar to this in some way.

~~~
dna_polymerase
Oh come on, if order books were generally accessible I could just see where
stuff is traded right now and sell there by myself. It's just that market data
for stocks is so heavily gated that HFT guys already have an edge by just
buying data from all the excahnges for huge piles of cash.

If this was such a great service why is IEX becoming so big?

HFT is a nightmare from a CSR view. Those companies provide no real value to
society (not by any standard) and even worse: By hiring all those talented
young people with degrees in various fields they actually hurt society. Give
those people real problems to work on, instead of wasting them on optimizing
those HFT systems.

~~~
lordnacho
> Oh come on, if order books were generally accessible I could just see where
> stuff is traded right now and sell there by myself.

You'd have to write a feed handler for every exchange, and a real-time
composite orderbook so you could see where the best order was at every point
in time. For every stock. You'd need to rent the fastest network as well.

This is no different from "I could tile my own roof, what do roofers do that
is so useful to society?"

> Those companies provide no real value to society

A lot of businesses provide no real value to society. The only way to do that
is to interact with a large portion of society. So restaurants for instance
only serve the local market, no real value. I mean who benefits, other than
the people who eat there? I could go on with a whole load of other businesses.

~~~
dna_polymerase
> You'd have to write a feed handler for every exchange, and a real-time
> composite orderbook so you could see where the best order was at every point
> in time. For every stock. You'd need to rent the fastest network as well.

Why would I need to rent a fast network for every stock? I didn't say abolish
the exchange, I said make the data open, generally available so I can see for
myself whats going on. Also smart order routers are there already, so no I
don't have to write them again, every other broker already offers them. Now
just cut out the HFT frontrunners and I can actually the price I see on the
terminal.

> A lot of businesses provide no real value to society. The only way to do
> that is to interact with a large portion of society. So restaurants for
> instance only serve the local market, no real value. I mean who benefits,
> other than the people who eat there? I could go on with a whole load of
> other businesses.

That may be right, but that doesn't make HFT any better.

~~~
dsacco
_> Now just cut out the HFT frontrunners and I can actually the price I see on
the terminal._

High frequency trading is not front-running. High frequency traders 1) do not
have a fiduciary duty to other traders in the market, which is a hard (and
definitional) requirement for front-running; and 2) cannot see a retail
investor’s order before it is executed on an exchange. They can only alter
their prices in reaction to orders that have already executed.

 _> I didn't say abolish the exchange, I said make the data open, generally
available so I can see for myself whats going on. Also smart order routers are
there already, so no I don't have to write them again, every other broker
already offers them._

1\. This data is already available for purchase. There’s nothing intrinsically
preventing you from acquiring it as an individual. I’ve personally purchased
this data as an individual.

2\. Even if you can “see for yourself what’s happening”, without the extremely
fast market making provided by HFT you’ll be waiting significantly longer just
to find a trading counterparty. Your proposal sacrifices the liquidity and
price efficiency provided by HFT in exchange for freely available market data,
which the vast majority of retail investors will not be able to use (let alone
want to).

3\. Smart order routing is possible because there is sufficient inter-exchange
liquidity. In fact, it is directly facilited by HFT.

I’m not often this blunt on Hacker News, but in this case I think it’s
warranted: you do not appear to have even basic familiarity with what you’re
criticizing.

------
stuxnet79
The author of this article has also summarized most of the basics in a mini-
book called "Max Dama On Automated Trading" which can be found online. Highly
recommended.

~~~
godelmachine
Does it cover anything about FPGAs in HFT?

------
godelmachine
The author says that he is giving a high level overview of the work done in
quantitative trading firms, and then mentions that - "things like co-location,
direct connection to the exchange without going through an API, using a high-
performance language like C++ for production (never Python, R, Matlab, etc),
Linux configuration (processor affinity, NUMA, etc), clock synchronization,
etc are taken for granted."

I'm surprised to see that engineers are required to know hard core VLSI stuff
like processors and physical affinity to cores, Non Uniform Memory Access,
clock synchronization et al. Thought only Integrated Circuit Digital Design/
Chip design engineers are required to know that.

Would anyone kindly shed more light on this subject? Has anyone used VLSI /
Digital Design topics as part of his Quant trading / HFT work?

~~~
lordnacho
I never thought of processor affinity as a VLSI topic. I mean you can muck
about with it just using your OS, right? Also I did EE in college and it
seemed a lot more low-level than that. Semiconductor principles, digital
filters, JK Flip-flops, that kind of thing. I guess you end up meeting NUMA
and affinity somewhere along the way.

The point the guy is making is that when you're coding this type of thing you
need to understand how the computer actually calculates things, rather than
just have a vague idea of a machine like you might if you code python. In
python and those types of languages a lot of the inner workings are abstracted
away. You can sort of just imagine an Oompa Loompa (a thread) reading the
code, seeing the instructions (x = a + 2 if y==3 else a - 2 ) and going out to
find the value of a in a box and putting in the value of x in another box. No
idea where a and x boxes are, how close they are to each other. No idea about
how that branching instruction works either, he just checks at the time and
decides.

By contrast when you're writing for performance you want stuff to be in cache,
so you need to have an idea of a machine that includes cache. Specifically the
idea that the further away you are from the registers, the longer things take,
and that some cache is shared while some isn't. You also want to think about a
machine that can speculate and do branch prediction, so you'll need to think
about how to write the code so the branch predictor mostly gets it right. I
think the top stackoverflow answer is about that.

Affinity and NUMA are talking to this kind of machine (which is still abstract
of course) which has a bit more detail than the Oompa Loompa.

~~~
godelmachine
Alright. I got your point. Knowledge of Computer Architecture is required (
Reference book - CA: A Quantitiative Approach by Hennessy Paterson).

I was thinking more in terms of Digital Electronics.

~~~
tcbawo
You are probably better off learning more about what Martin Thompson calls
'mechanical sympathy'. Check out the LMAX/Disruptor talks. There are some blog
around high performance/low latency computing.

~~~
godelmachine
Thanks for this suggestion :)

Will definitely go through it.

------
melling
Here are some useful quantitative trading links:

[https://github.com/melling/MathAndScienceNotes/blob/master/q...](https://github.com/melling/MathAndScienceNotes/blob/master/quant_trading.md)

Some include previous HN posts.

~~~
afeezaziz
Thanks for this.

------
S3raph
he published also a now kinda old, but still great trading
introduction/summary:

[https://www.quantopian.com/posts/max-dama-on-automated-
tradi...](https://www.quantopian.com/posts/max-dama-on-automated-trading-pdf)

The pdf link seems broken, but through Google you should easily find the pdf
(maxdama.pdf)

~~~
KngFant
here it is:
[https://web.archive.org/web/20160515034451/http://www.decal....](https://web.archive.org/web/20160515034451/http://www.decal.org/download/2582)

~~~
dogruck
Thank you. I’m having trouble downloading the pdf from that link. Tips?

~~~
liftingdutchman
google : inurl:maxdama.pdf

------
RyanShook
Was aware of arbitrage and market making as a quant strategy but had never
heard of market taking as an outsider. Thanks for sharing, helps give a little
better picture of how the market works.

------
anonu
Having been in this space, I agree with most of the article. Good summary.

The one thing I would say is that it's very hard to make money doing this. The
profits accrue to the select few who have cornered the market in terms of
technology and speed.

Which is why firms who make money doing this usually have an edge beyond the
technology. In one word: flow.

They are able to interact with a stream of orders... Not theirs... That is low
in alpha and they are able effectively to get "first look".

~~~
mkalygin
Does it mean that trying to make some cash doing automated trading in 1-2
people is nearly impossible?

~~~
anonu
I'd say your probability of success is going to be low. Success is not solely
defined by PnL. If you're making money then you will certainly have to deal
with increased compliance and oversight from whoever is sponsoring your access
to the markets - this comes with a cost.

Furthermore you can never just "sit" on a strategy. Eventually your edge gets
arbed out and you need to be creative enough to data-mine the heck out of the
market to find some new edge.

Some thoughts on figuring out your edge: 1\. You're not the only smart person
looking at something. There are armies of PhDs who have been there and done
that. 2\. Study regulation and regulatory changes. Keep abreast of new &
upcoming changes. 3\. Look for markets where barrier to entry is higher or not
as studied. eg: Electricity markets are a _bit_ less understood or traded. 4\.
Always think in pairs or more... Longing or shorting something outright is
almost never the answer. But trading the spread between two (3,4,many) related
products is more difficult to calculate and backtest. 5\. Always do the more
difficult thing - but don't make it complicated. Simple math is usually what
works.

~~~
mkalygin
Thanks for your explanation, I don't have illusions that it's going to be
easy. I rather think the opposite. For now I'm trying to get a basic
understanding of trading and finance. This is just a hobby along with my
programming job.

From your words I understand that I have got to be very creative in order to
find an edge. Especially your advice on thinking in pairs or more is valuable.

------
godelmachine
Never knew FPGA programming was so easy in quantitative trading. Would anyone
kindly shed more light on use of FPGAs in HFT & Quantitative Trading? Any web
links or PDFs?

~~~
seanp2k2
Not sure how helpful this is but [http://algo-logic.com/node/1](http://algo-
logic.com/node/1) Is a vendor of NICs that have FPGAs built in to do
programmable algo trading VERY quickly. A friend used to work there.

~~~
kgwgk
Another vendor: [http://www.novasparks.com](http://www.novasparks.com)

------
infinii
Is there an error in this? "In this case book pressure is simply (99.00 _10 +
98.75_ 5)/(10+5) = 98.9167." in the Market Structure signals section. I can't
see how that equation relates to the trade data given just prior.

~~~
arnioxux
I don't really get it either so here's an attempt to justify it with gambler's
ruin:
[http://mathworld.wolfram.com/GamblersRuin.html](http://mathworld.wolfram.com/GamblersRuin.html)

> Let two players each have a finite number of pennies (say, n_1 for player
> one and n_2 for player two). Now, flip one of the pennies (from either
> player), with each player having 50% probability of winning, and transfer a
> penny from the loser to the winner. Now repeat the process until one player
> has all the pennies.

> The chances P_1 and P_2 that players one and two, respectively, will be
> rendered penniless are

> P_1 = n_2 / (n_1 + n_2)

> P_2 = n_1 / (n_1 + n_2)

So if the current order book looks like:

\- Sell 5 for $99.00 (best offer)

\- Buy 10 for $98.75 (best bid)

And you chomp up orders randomly, flipping a completed order to the opposite
position:

\- the probability that the price is above $99.00 (all 5 sell got filled
first) is 10 / (5 + 10)

\- the probability that the price is below $98.75 (all 10 buy got filled
first) is 5 / (5 + 10)

So expected value is (99.00 * 10 + 98.75 * 5) / (10 + 5) = 98.9167.

I am not happy with this explanation. It seems like after you execute an buy
for $98.75 you need to immediately put a sell for it at slightly above $98.75
(and vice versa) to fit the random walk model described above. And then I took
the expected value using the price at two different points in time. Overall I
am still confused myself.

------
gravypod
How do you get started with pulling in this data? I've always wondered how you
could break into this market at all.

The only impossible part is finding tick-by-tick market data

~~~
jzwinck
Finding tick by tick data is easy. Every major lit exchange sells it for a few
hundred to a few thousand dollars per year, either directly or indirectly
(through a data vendor). Just search for "market data price list" and the name
of your preferred exchange.

~~~
gravypod
That's still quite a bit of money. If you want to train some good models
you're going to need a lot more than 1 year and you're going to need many more
than 1 market.

~~~
tcbawo
Some exchanges post sales data available for free. If you are relying on the
full market data stream, you are talking about a considerable amount of data.
In some cases, terabytes per week. It may be less costly to buy the data than
connect and record directly, especially for a proof of concept.

~~~
gravypod
Which exchanges? This is more of a hobby. I can throw together stuff that can
handle that load.

------
SirLJ
Very interesting article, but heavily focused on high frequency trading...

I am really involved in low frequency trading with as I call them stock
trading robots and have been doing this for many years with double digits
return on average and all is done with python and few VPS around the world for
redundancy e.g. very low brier of entry...

~~~
chirau
Hi, can you elaborate on low frequency trading? Is that day trading or it's
holding positions for much longer? Any resources?

~~~
SirLJ
Nope, it is not day trading... I think you cannot win against the HFT shops,
unless you spend millions on co-location and the whole arms race... No
resources, everything is based on my own experience...

~~~
MarkMc2412
Would love to hear more about this, you should do an article/post!

~~~
SirLJ
Frankly I won't... I am not selling or promoting anything, I don't need
clients or investors, I am not promoting myself and I am not looking for a new
job, quite the opposite, I am actually looking for an early retirement, (but I
just can't leave the team I am leading behind, so I am postponing very
year...)

------
lifeisstillgood
One of the things that interests me is just how much, if anything, the
development of these systems can be applied elsewhere outside of finance.

Now I get some attempts to financialise other industries (usually having large
social costs) but just wondered if we are seeing this creep out elsewhere
(online advert auctions perhaps?)

------
whoisninja
This alludes that all quantitative trading is high frequency trading. I would
think all high frequency trading is quantitative trading but converse is not
always true.

~~~
whoisninja
Because then one of the most successful quant funds like twosigma and rentec
are not doing quant trading since hft is only small portion of their business.
You cannot manage 50B in an hft strategy single handedly. The author lives in
a different dimension, but my dimension could be wrong!

~~~
georgeek
HFT is best defined as predicting movements under a tick; everything above a
tick is out of HFT territory. Latency requirements could still be pretty
serious though, as many models at many firms can act simultaneously for longer
frequencies as well.

------
ComodoHacker
Is quantitative traders' role in the economy similar to that of cryptocurrency
miners: they form consensus on the state of the market?

~~~
rocqua
That would be the case for Market Taking and Arbitrage. They help with
determining an accurate price (Market Taking) and with keeping prices equal on
all exchanges (Arbitrage).

For Market Making the goal is not to form consensus on the state of the
market. Instead, Market Making is about creating a liquid market. That is,
making sure that when someone wants to buy or sell, there is a willing
counterparty at a non-stupid price. Alternatively, you could describe them as
'keeping the spread low'. That is, keeping the difference between the lowest
buy order and highest sell order low.

------
known
Just port your trading strategy to kernel space.

------
mlevental
the linked article (500k in one year...) is really good too.

------
wind85
this is awesome.

