
Introduction to Algotrading (2011) [pdf] - steventhedev
http://isomorphisms.sdf.org/maxdama.pdf
======
maxfdama
Thanks for posting. I wrote this back in college and looking back a lot of it
is outdated (Getco doesn't exist anymore), and some parts even cringeworthy...

After writing that I joined Headlands, where I've been working for the past 8
years as we grew from a startup to a relatively large firm (trading is
actually a very small industry). If you want a more up to date description of
low-latency trading in particular, please see this more recent post:
[https://blog.headlandstech.com/2017/08/03/quantitative-
tradi...](https://blog.headlandstech.com/2017/08/03/quantitative-trading-
summary/)

Feel free to email me, my email address in the pdf is still valid.

~~~
fnord123
>some parts even cringeworthy...

Indeed, "work shall set you free" is not a great reference to include.

~~~
JHonaker
I just looked that phrase up. That’s some supremely bad taste from the author.

~~~
mjlee
To be fair, you had to look it up. It's possible if you didn't know it was in
bad taste you'd have repeated it in the future as well.

~~~
normal_man
Anyone with a basic modern education should know exactly what that phrase
references. It's like pretending "that's all, folks!" doesn't immediately
evoke Looney Tunes

~~~
detaro
> _It 's like pretending "that's all, folks!" doesn't immediately evoke Looney
> Tunes_

It doesn't for people who haven't grown up with Looney Tunes. You're on a
fairly international site here. Similarly, you can't expect everyone's
education about the Holocaust to have covered this specific thing in a way
they'd reliably remember and avoid an embarrassing mistake like that.

------
usgroup
This is full of gems ; and a great compendium I wish I had 10 years ago.

However, honest advise: if you actually really want to trade in this way,
start with a solid course in mathematical finance. It’ll teach you how to
model the space, think about arbitrage, portfolios, risk, factors and the
sheer important of theory and hypothesis in real strategy.

Building a backtester is exceptionally hard to do if you include scenario
analysis, slippage, event risk, lag, multi hypothesis testing, data snooping,
etc. Testing is a huge source of competitive advantage.

In summary, if you’re starting out, go learn the basics like the back of your
hand, and he basics are mathematical finance.

~~~
logicchains
I work in HFT, and I'll be contrary and suggest that if you want the most
value from your time, focus on learning as much maths and probability as
possible. Once you have a strong maths background, learning most financial
maths is relatively easy. Any of the financial models you find in textbooks
are probably useless because if they work, everybody else who's read the
textbook has also used them so there's unlikely to be any edge left, hence
it's the ideas underneath them that matter, and understanding and extending
these is much easier with a strong maths background. The key skill is
developing and assessing new models, and the better one's maths, the easier
this is (imagine if a market behaved like a complex aerodynamic system; only
somebody with enough mathematical background to model a complex aerodynamic
system would be able to create a very accurate model of that market).

One of the most successful hedge funds, Renaissance Technologies, also takes
this approach. Regarding its employees: "a third have PhDs, not in finance,
but in fields like physics, mathematics and statistics. Renaissance has been
called “the best physics and mathematics department in the world” and,
according to Weatherall, “avoids hiring anyone with even the slightest whiff
of Wall Street bona fides."
([https://en.wikipedia.org/wiki/Renaissance_Technologies](https://en.wikipedia.org/wiki/Renaissance_Technologies)).
Similarly some of the best HFT hires I've been have been people with a maths
or science background.

A quote sourced to Renaissance Technologies from [https://www.quora.com/What-
is-the-secret-behind-Renaissance-...](https://www.quora.com/What-is-the-
secret-behind-Renaissance-Technologies-Can-mathematics-really-be-used-to-
bring-high-returns-to-investors/answer/Vladimir-Novakovski): "We have some
Tier One mathematicians and a lot of Tier Two mathematicians. Other quants
funds have mostly Tier Three mathematicians, and worse, they don’t even know
that these tiers exist! Someone has to create the correlations in the
markets." Aim to be one of those Tier one mathematicians!

~~~
tudelo
How would you even become a 'Tier One' mathematician? What does that entail?

~~~
logicchains
I suppose work hard on learning, deeply understanding and practicing as much
math as possible, and trying to make novel discoveries / models. I'm certainly
not a Tier One mathematician, so I'm probably not the best person to ask.
Maybe it requires people to be born with some innate ability, although I'd
certainly like to hope it doesn't. Regardless of whether it's actually
achievable or not, however, it's definitely a worthy goal to strive towards if
you want to be a good quant/trader (and a very intellectually stimulating
one).

A nice post on the topic by someone who most certainly is a Tier One
mathematician: [https://terrytao.wordpress.com/career-advice/does-one-
have-t...](https://terrytao.wordpress.com/career-advice/does-one-have-to-be-a-
genius-to-do-maths/).

~~~
usgroup
Remember reading an article by an ex rentech guy who says that linear
regression is about as complex as the modelling gets; that the edge is not in
making things hard.

~~~
logicchains
I'd be surprised if that's the case, as surely somebody else would have
replicated it by now. [https://www.quora.com/What-are-the-investment-
strategies-of-...](https://www.quora.com/What-are-the-investment-strategies-
of-James-Simons-Renaissance-Technologies-I-understand-he-employs-complex-
mathematical-models-along-with-statistical-analyses-to-predict-non-
equilibrium-changes/answer/James-Baker-69) describes something that sounds
much more complex than what could be achieved by linear regression alone.

~~~
usgroup
Tbh I imagine their edge is more likely to be better data and a secret theory.

Then the applied part of it is simple but different: different analysis,
different theory, different data, extraordinary results.

Edit: just read your article ... kind of poo poos my theory if what he says is
true ; at least at the level of the mega model.

~~~
saosebastiao
It doesnt refute your theory, it's just that your theory applies very
specifically to HFT. Alpha generation in HFT is incredibly simple, and HFT
analysis typically relies on very basic math: linear regression and recursive
filters. HFT relies on execution far more than modeling to generate returns.

But HFT is inherently low capacity. You can't put a billion dollars on an
order book and expect to make the same returns as you would with a thousand.
That's the reason HFT firms are almost always proprietary trading firms...they
don't need or want more capital.

A hedge fund, like Rentech, is typically on much higher timeframes because
their size necessitates higher capacity strategies. This could mean holding
periods of minutes for the million dollar funds to days for the billion dollar
funds. As you get to higher and higher timeframes, your mathematics are going
to need to be dramatically more sophisticated in order to beat the market. The
math you are looking at in that quora link is about pairs trading which is
where pairs of instruments mean revert over time. I would expect Rentech to be
doing this type of trading over long timeframes and holding a trade for days
to weeks.

------
jmrobertson
[https://meanderful.blogspot.com/2018/01/the-accidental-
hft-f...](https://meanderful.blogspot.com/2018/01/the-accidental-hft-
firm.html)

My all time favorite article related to this topic, I always try to promote it
Guys like the author started this whole rodeo through some really ingenious
hardware/software/networking jerryrigging.

------
myaccount80
Kind of unrelated but I would like to have HN opinion: what is the actual
point in working as a trader ? At the end of the day you just made more money
or lost money. But you’re not contributing in anything for the world. I really
don’t understand why people are happy to say they are trader when at the end
it seems that the only do it for money. In 40 years when I look back I don’t
personally want to see that I worked for 20 years for a company just to make
money, rather I would like to say for example: I built this product which
helped some people

~~~
QuackingJimbo
I sleep better working in HFT than working in regular tech

HFT, at worst, adds exactly zero value to society. I don't buy into the memes
of "we provide liquidity!" or "we make markets efficient!" being some noble
mission. But HFT certainly does no harm

Regular tech ultimately just pushes ads or products on people, which I
consider poisonous to society

------
lordnacho
Algo trader here. This document is fantastic.

I think it's hard to show, but coding is of paramount importance. General
skills like keeping things in version control, writing simple code, and making
things modular. But also quite advanced specific topics like CRTP or cache
optimization. Could be a whole book in itself without the financial parts.

~~~
CoolGuySteve
Watch out, if you get too good at technology they’ll stick you in the back
office and never let you trade, or even see what the pnl is.

~~~
soVeryTired
Back office's job is trade reconciliation, pnl, and accounting.

------
Ragib_Zaman
This document is amazing. I'm interested in working in quant trading when I
graduate, and I can't believe I haven't found this before. Do you think if I
wrote my own back tester and test strategy and put it onto GitHub it would be
a positive in applications/interviews, or is that too basic to be seen as
interesting/substantial?

~~~
mruts
It depends what kind of job you want. If you want to go into quant research
you’re probably just going to be writing inefficient python code. So if you
want to do that, just go straight to something like Quantopian where the
infrastructure is already set up.

If you want to be the guy who actually knows how to program (the guy
translating the Python into C++), then I think a backtester would be a nice
project. Look at the python libraries zipline and pyfolio and then try to
create something yourself (ideally not in python).

------
hash872
Here's my question I've always wanted ask re:trading. How do EMH & profitable
trading (especially by institutions) coexist? I already understand EMH &
'active management'. Do trading profits consistently exceed a market benchmark
like the S&P 500's return for the year? (I'm assuming so). If so- doesn't this
completely disprove EMH? And if so, further- why does any institution do
active management when they could just actively trade assets for profit
instead? Where is really the line between active management and trading?

Sorry for the dumb questions- I have genuinely spent significant time Googling
and could never find the answers, thought I'd use the opportunity to ask here.
I am obviously not a market professional, just one guy with most of his assets
in index funds. I can rephrase questions if necessary

~~~
1e-9
The efficient-market hypothesis is a useful theoretical ideal that is never
perfectly achieved in reality. Good trading drives the markets towards
efficiency with the work of the traders being compensated by capturing a
portion of the inefficiency that they remove. Traders who are above average at
removing inefficiency will be compensated above average and the less capable
traders will tend to drop out due to losses. This competition allows markets
to converge towards efficiency over time. If the markets were perfectly
efficient, there would be no trading firms as they would have no way to get
compensation for providing their service. If there were no trading firms, then
the markets would be less efficient because of the erratic appearance of
buyers and sellers as well as the loss of many trading experts evaluating risk
and value. So, you see, there is a nice negative feedback loop at the heart of
the markets that drives them closer to efficiency over time as traders are
motivated to improve and advancements in technology and theory allow better
determination of pricing.

Investing in equity markets is very different from trading. Corporations pay
you dividends and capital gains in return for the capital you provide them.
That's why long term investments in passive index funds are typically the best
bet for someone who is not a finance professional. Rather than trying to beat
the professionals at pricing assets, you get to sit back and enjoy the
compensation from a mix of companies working hard to increase the value of
their stock and/or generate dividends. You just need to ensure you don't take
on excessive risk. Stay away from margin at all costs.

~~~
hash872
Thanks for the response man. Not sure if you'll see this, but- is it actually
proven that some % of traders are consistently, year over year profitable?
Because that would seem to be another blow to the strong form of EMH. I could
see a more EMH-friendly world where some traders are profitable some of the
time, but it's not consistent who's who year over year (which is what is
supposedly said about active fund managers, one good year has little
correlation to having another good year)

~~~
1e-9
I am unaware of much publicly available proof, but in my experience, there is
definitely a class of trader that is consistently profitable every year or
even much better than that. These are the traders who are meticulous about
their risk management and have found a way to improve a market. For a public
glimpse, you could have a look at Virtu Financial's S-1 filing from 2014
([https://www.sec.gov/Archives/edgar/data/1592386/000104746914...](https://www.sec.gov/Archives/edgar/data/1592386/000104746914002070/a2218589zs-1.htm)).
In it, they disclose that over the previous five year period, they were
profitable on every day except one.

------
zygy
I enjoyed this, but I'm sure the strategies have evolved a lot. Anyone know of
more recent resources like this?

~~~
anonu
Nobody is going to give you the keys to the castle. Trading strategies that
are more recent probably still have some edge in them. These things are
closely guarded secrets.

Having said that, in addition to the other recommendations in this thread,
lookup SEC settlements. The Athena trading one from a few years back regarding
their "Gravy" on close strategy is particularly memorable.

Edit: [https://www.sec.gov/news/press-
release/2014-229](https://www.sec.gov/news/press-release/2014-229) SEC order
at the bottom of the press release

~~~
HenryBemis
+1

A successful trader won't give out the thing that if replicated by 10.000
people will attract attention that may deem this inoperable any more.

Part of my 'hobby' is to find EAs and put them to test in other pairs,
timeframes, and try to optimise and get better configurations for the triad
EA-pair-timeframe. When I get one, it goes straight to MY 'parking lot', NOT a
blog :)

~~~
anonu
What is EA?

~~~
HenryBemis
"Expert Advistor", aka the softrware that stays online (running on a virtual-
private-server preferrably) and has the capability to trade 24/5.

~~~
Youden
I'll add since it isn't clear that this term is usually exclusive to Forex.

I'll also add that the commercial market for them is about as scummy as it
comes. You want to know how to game backtest metrics? Have a look at what
they're doing with EAs.

------
malhotra_chetan
This is pretty good. I had a quick glance and surprisingly did find some parts
that are generally missing from 'Intro to Algo Trading' literature especially
in the execution part. Anyway, I work at this company where we are a product
that makes production systems for trading algorithms into a PaaS. We also have
a library
LibKLoudTrader([http://docs.kloudtrader.com](http://docs.kloudtrader.com))
that incorporates pretty much every thing one needs to start with algorithmic
trading.

------
w23j
Looks great. It's always interesting to learn more about how these things
actually work.

This XKCD is always a good reminder not to get to enthusiastic:
[https://xkcd.com/1570/](https://xkcd.com/1570/)

~~~
HenryBemis
Painfully true.

Numbers is just one aspect, then we got 'the News' (and the specualator, the
other EAs that are programmed to kill 'your' EAs), and then we got politicians
twitting (I won't name names), and semi-democratic governments that have other
agendas unrelated to financial prosperity, and so on, and so forth.

Basically unless one is running algos with a target of more than 20% per
annum, they should be losing sleep.

------
shivekkhurana
> In many countries such as Russia, corruption is a better career path.

ROFL

------
jrh206
This is generally pretty great.

I was taken aback by the 'work shall set you free' quote at the top of Section
2. That seemed in very poor taste.

~~~
jack_pp
Depends on your interpretation. I found it inspiring

~~~
JHonaker
It’s a quote from the entrance of Auschwitz, and lots of other Nazi
concentration camps.

~~~
jack_pp
They did not invent the saying, i just did a little googling and it seems they
didn't even use it on all the camps. It was preserved from previous wars and
it became a horrible sign in the context of the holocaust.

I could see how that specific saying is tainted in German but I do not think
it should be tainted for all of history and in all translations because of
some unfortunate coincidence. I believe it has wisdom to share

~~~
JHonaker
I mean they didn’t invent it, but the author of the book that did was a member
of the German Nationalist movement that eventually (~75 years later) evolved
into the Nazi party.

It’s not like it was completely disassociated with Nazi history and then they
decided to put it on their camps because it sounded cool. They literally
wanted to work people to death, i.e. freedom.

------
2038AD
Minor point but the following isn't a Korean proverb:

>When you laugh, the world laughs with you. When you cry, you cry alone.

It was in the film Oldboy but it's actually from an English-language poem.
Ella Wheeler's Solitude:

    
    
      Laugh, and the world laughs with you;
      Weep, and you weep alone.
      For the sad old earth must borrow its mirth
      But has trouble enough of its own

------
quickben
I used to work for one of the HFT companies listed there out of university.

What are the salaries now-days for a BackOffice Technology (C++ and everything
else related)?

------
deehouie
I cringe when I see so much goes to building the IT infrastructure and so
little to modeling and validating algorithms. Someone points out that most
quant funds fail. Key reason is their ML models are badly overfitted. By
fitting enough models, you'd always find one with a good looking Sharpe Ratio.
This is the classic multiple hypothesis testing problem.

[https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3274354](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3274354)

------
webmascon
It is not about algotrading. It is about quantitative automated trading

~~~
FabHK
The distinction between algo trading vs automated trading is referred to as
New York vs Chicago "style" algo trading in the document:

> The phrase “algorithmic trading” has a different meaning depending on if you
> are in Chicago or New York. In Chicago it will mean using a computer to
> place trades to try to make a profit. In New York it means to use a computer
> to work client orders to try to minimize impact.

~~~
webmascon
Algorithmic trading means the same in chicago new york tokyo hongkong or
london. Even in singapore and sydney. Same thing: slicing the order to
minimize the impact

~~~
scottlocklin
I'm amazed people are downvoting this. It's 100% correct. Algotraders work the
book to get a good price; they're modern block shoppers. Quantitative traders
look for profit. There are entire books on algorithmic trading as "work the
book."

Max was a fresh out of college graduate at the time, so his usage was a bit
off. Knowing this distinction is a sign of actual experience on the street.

~~~
FabHK
I think both of you are correct. That’s also what Wikipedia says.

However, loads of people (and, according to Max’ document, Chicago) refer to
automated trading as “algo trading”. Cue the descriptivist vs prescriptivist
linguist debate.

~~~
HenryBemis
Please allow me for this topic to quote Investopedia [1], and not Wikipedia
(which I respect and appreciate).

"..is a type of trading done with the use of mathematical formulas run by
powerful computers.."

"..makes use of much more complex formulas, combined with mathematical models
and human oversight, to make decisions to buy or sell financial securities on
an exchange.."

[1]:
[https://www.investopedia.com/terms/a/algorithmictrading.asp](https://www.investopedia.com/terms/a/algorithmictrading.asp)

Ps: In simple english, I go to work, that thing makes profit (if done right).
I go to sleep, that thing makes profit. Someone important twits, that thing
either makes profit, OR goes bust.

~~~
webmascon
Investopedia makes the same mistake. When you sleep and computer makes you
some money, it's automated trading, systematic trading, mechanical trading.
This kind of trading was in the industry for centuries: you incent some rules
and follow them. Algoritmic trading is about executing large orders by slicing
them and filling them thru a trading day. These are the terms seriours
industry professionals are using. If you apply for algo trading job in a bank
you will not be writing a cash making bot.

~~~
FabHK
Yeah, my impression is that professionals distinguish algorithmic trading
(minimising price impact of a given order) and automated trading (rule based
trading to generate profit), while amateurs (including Investopedia) tend to
use the term "algorithmic trading" either for both, or for the latter while
being completely unfamiliar with the former.

~~~
webmascon
Correct. And it is not just Investopedia who is misleading. Type Algorithmic
trading in amazon search and you will see tens of books on this topic that
have nothing to do with algorithmic trading. No wonder some people are
claiming they are doing algorithmictrading and even high frequency trading
using their laptops from their bedrooms.

------
polishrobotsguy
When did we move away from smart people sending humanity into space to smart
people making billions on market inefficiencies (making everyone else
collectively poorer)?

~~~
mruts
Making markets efficient helps everybody, and historically, has probably been
the biggest driver in making earth a better place.

Let’s take HFT for example. HFT firms are the reason that Mom and Pop
investors can get efficient execution on the public markets. Compared to the
inefficient markets that existed before, they have generated many many
billions of dollars to pension funds, mutual funds, and individual investors.

I don’t think it’s possible to overestimate the positive effect efficient
markets have had on everyone’s lives, from the richest to the poorest.

The world isn’t a zero-sum game, and markets aren’t either, everyone profits
from a good free market.

~~~
jmrobertson
Ehhhhh. yes. But that glosses over dark pools, some of the shadier exchange
infrastructure arbs that go on, etc.

To be candid, I'm fully on the same page as you: what's going on here is no
different than what a really good pit trader did from 1800's -> 2000\. It's
just the digital, semi-invisible version, and because volumes are so large,
the money is exponentially larger as well.

But, that's not all the HFTs get up to; a lot of it doesn't hit public
markets, or isn't particularly helping w/ efficiency.

~~~
atemerev
A "dark pool", despite an ominous-sounding name, is just a type of exchange
which does not display order prices and sizes. It is something in between of a
regular exchange and an auction, better suited for posting and executing large
orders. There is nothing shady in it; it helps larger players (like retirement
/ index funds) to obtain better prices for their large orders.

~~~
jmrobertson
Yes, not shady, but it totally unravels the argument concerning HFTs helping
price discovery. I know the "hey retirement and index funds are doing it,
think of the pensioners!" argument is common defense of it, but that's not at
all the target audience of DPs. It's liquidity going off exchange, which hurts
price discovery, simple as that. Considering how competent trade
execution/slippage capabilities are for the types of broker-dealers that would
be handling index/ret/pension volumes, that argument is nonsense

~~~
atemerev
Well, dark pools are a counter-HFT measure, of course they have limited price
discovery. If you want price discovery, you can go to lit markets. If you want
matching without showing your orders, you can put orders to the lit book
algorithmically, or go to a dark pool. It's not that there is no choice. And
people traded off-exchange like forever, OTC market still exists and predates
exchanges.

