
Ask HN: Trading Algorithms - jchonphoenix
So I'm a university student that has been working with recommendation systems recently (netflix challenge etc) and wanted to try my hand at implementing some trading algorithms for the stock market to see if I could at least break even. I'm fully expecting to lose everything I invest, so that's not a problem, but I figure it'd be a good learning experience. Does anyone have some links to good papers about machine learning applied to stock trading (or any trading algorithms in general)? How about API's that could allow my programs to perform trades?<p>Thanks for all your help HN!
======
ggruschow
Don't try to teach the machine things you don't know yourself. Study the rules
and actors in the systems you're studying, and come up with theories to test
against the data. You'll need to do this work anyway, so it's at least a good
way to start before you complicate things further with AI/machine-learning.

Understand this first though: The vast majority of money made in trading is
made without anything more than a trivial prediction of the future. Investing
is different, but very very few have enough money to make active investing
worthwhile.

Start out with "Algorithmic Trading & DMA" by Johnson:
<http://www.amazon.com/dp/0956399207> If you choose not to, you're better off
not reading anything about the actual trading than trying to pick another..
The vast majority of published material is so bad that it'll set you back.

------
Kjeldahl
For API connectivity and instruments, few beat Interactive Brokers, at least
for the price offered. As for strategies, avoid walking into the complex
indicator traps (combining indicators x,y,z to make a new super indicator).
Stick with the simple stuff for trends etc (moving averages), and try to
combine with other stuff, possibly other data (from other markets, or
completely different sources). And forget about having a model that runs
unattended on its own while you're at the beach. Markets change and you should
stay on top of it, adapting the models for whatever the markets demand
(watching out for sell-offs the last couple of trading hours is one recent
theme). And don't forget about money management. Good luck!

~~~
greyman
Question: Why combining indicators doesn't work?

~~~
wisty
The curse of dimensionality, I'd guess.

------
billpg
"I'm fully expecting to lose everything I invest"

You'll never get the taxpayers to cover your losses with an attitude like
that.

~~~
retube
depending on your leverage you could lose a lot more than you invest....

------
lrm242
This is a fun game and is very challenging. If you apply yourself you can be
successful.

I would highly suggest picking up the book, Trading and Exchanges: Market
Microstructure for Practitioners (<http://www.amazon.com/dp/0195144708>) by
Larry Harris. To trade effectively you need to understand how the markets work
and this book provides and outstanding tour through the markets, who
participates in them, and why they do or don't make money.

There are innumerable ways to make money in the markets. Long term, short
term, technical or fundamental, with retail platforms like Ninja or going very
sophisticated and connecting directly to an exchange like NASDAQ using native
protocols like ITCH and OUCH. Don't let naysayers distract you from your goal
--for every naysayer there is always a counter point.

If you want some motivation, read through this IamA at reddit:
[http://www.reddit.com/r/IAmA/comments/9s9d7/iama_100_automat...](http://www.reddit.com/r/IAmA/comments/9s9d7/iama_100_automated_independent_retail_trader_i/)

------
arthurdent
Interactive Brokers has paper trading and good docs
[http://www.interactivebrokers.com/en/p.php?f=tws&p=p](http://www.interactivebrokers.com/en/p.php?f=tws&p=p)
[http://www.interactivebrokers.com/en/p.php?f=programInterfac...](http://www.interactivebrokers.com/en/p.php?f=programInterface)

<http://www.maxdama.com/> talks about the IB api a bit.

I skimmed your comments and kept seeing the word python. There's a 3rd party
python-IB package here (can't vouch for it): <http://code.google.com/p/ibpy/>

<http://epchan.blogspot.com/> talks a little bit about basic pairs trading
strategies. My experience has been that adding complexity off the bat is
usually not the way to go. I don't personally know anyone who is using
AI/machine learning as their primary strategy generation mechanism, but maybe
I just run with the wrong circles.

I'm interested in collaborating with smart people re trading ideas, so if
you're interested as well, fire me an email.

~~~
jchonphoenix
I've heard of IB before so its a name I'll probably look into.

I'm mostly interested in machine learning because I feel like a lot of the
algorithms I'm applying to recommendation systems could be applied to stocks.
If you think of the stock market as a giant system where someone wants to
recommend stocks that will go up, it appears these ML techniques may be
applicable. At this point however I don't have a way to model the stock market
as an acceptable input to the most promising algo I have, which is one that I
developed myself and has and extremely low RMSE for certain types of data.

------
gtani
<http://quanttrader.info/public/testForCoint.html>

[http://quantivity.wordpress.com/2010/01/12/how-to-learn-
algo...](http://quantivity.wordpress.com/2010/01/12/how-to-learn-algorithmic-
trading-part-2/)

[http://quantivity.wordpress.com/2010/01/12/how-to-learn-
algo...](http://quantivity.wordpress.com/2010/01/12/how-to-learn-algorithmic-
trading-part-3/)

<http://www.markjoshi.com/downloads/advice.pdf>

<http://www.markjoshi.com/RecommendedBooks.html>

[http://www.reddit.com/r/MachineLearning/comments/by97n/finan...](http://www.reddit.com/r/MachineLearning/comments/by97n/finance_and_machine_learning_masters_thesis/)

<http://www.stanford.edu/~cover/portfolio-theory.html>

------
andr
One of the key lessons in finance is that you can't make money doing what
everyone else is doing. Every known technique is being applied a lot and can
rarely bring you abnormal returns.

I suggest reading through the links other commenters posted and trying to
think of ideas that you never saw mentioned. Brainstorm wild ideas - for
example, can you adapt a lossy compression algorithm to predict the stock
market?

~~~
ggruschow
_One of the key lessons in finance is that you can't make money doing what
everyone else is doing._

This is only true in the long run, and not true for individuals. As a simple
example, consider the practice of selling far OTM options.

------
RockyMcNuts
I would probably start with the Turtle Rules - an ultra simple system that was
used successfully in the olden days, as a good overview of the elements of a
successful system.

<http://bigpicture.typepad.com/comments/files/turtlerules.pdf>

Possibly of interest - [http://www.amazon.com/Trading-Systems-That-Work-
Evaluating/d...](http://www.amazon.com/Trading-Systems-That-Work-
Evaluating/dp/007135980X)

MetaStock - system backtesting software - <http://www.equis.com/>

~~~
regularfry
Seconding Turtle Rules - I've just read it, and it's a good, quick,
informative read. It's more a meta-strategy book in that it tells you how to
approach building and running a system, rather than focusing on the system
itself.

------
owyn
I stumbled across this paper a while ago on HN and found it interesting:

[http://www.cs.bham.ac.uk/~wbl/biblio/gp-
html/Dempster_2000_Q...](http://www.cs.bham.ac.uk/~wbl/biblio/gp-
html/Dempster_2000_QF.html)

~~~
twp
Dempster was my PhD supervisor and I built on his and Chris Jones's work.

The idea was to use genetic algorithms to find combinations of technical
indicators that would turn a profit in high frequency foreign exchange
trading. Another PhD student (Yazann Romahi) tried a reinforcement learning
approach.

The GA found combinations that worked OK on the training set as long as
transaction costs were very low. Outside the training set they sometimes did
OK, but, this being genetic algorithms, you could never be sure that it was
just luck and over-fitting to the training set. As soon as transaction costs
approached realistic levels everything lost money. Another student based his
PhD thesis on identifying the 50 successful runs out of 1000 which were
significant at the 5% level.

I wouldn't recommend this as a useful approach - the best way to make money in
FX is to be a market maker or broker, i.e. make money on the bid/ask spread or
to get commission on every trade.

Ultimately I found the mix of finance and artificial intelligence to be a case
of the blind leading the blind. The financial people had lots of money and
deeply wanted to believe that there were magic patterns in the market that
computers could discover. On the other side, the AI people wanted research
funding and deeply believed that their magic AI black boxes (neural nets,
genetic algorithms, support vector machines, etc.) could discover these
patterns. The naivety of and misplaced belief of both sides was quite
depressing to observe, although many did (and do) exploit it for their own
personal profit - primarily through exorbitant consulting fees.

------
sharpn
You can start here:

<http://www.wilmott.com/categories.cfm?catid=38>

------
hogu
don't loose all your money, interactive brokers has paper accounts that let
you fake-trade without consequence.

------
waveman
<http://seykota.com> \- not what you think you are after, but very useful
regardless.

Also <http://www.trendfollowing.com/>

Trading is very hard. Markets are efficient at a first approximation and where
they are not efficient, simple arbitrages do not exist. You are up against
smart, well capitalized people. Once you find an edge your irrational brain
cuts in.

Books: "Investing by the numbers" by Jarrod Wilcox. "Trend Following" by
Covel. "The only three questions that count" by Fisher. And of course the
"[Stock]Market Wizards" books. Typically the top traders seem to have read a
couple of hundred books on the topic and have done thousands of hours of
research.

Your assumption that you will lose at first it probably right. But that's
better than winning at first, which is likely to falsely convince you that
you're a genius.

Beware the following which can look like skill for a while. 1\. Randomness
(temporary luck) 2\. Excessive Leverage (-> gambler's ruin) 3\. Selling
insurance without realizing it eg selling out of the money options.

------
Mamady
Have a look at <http://www.collective2.com>

They have many trading systems, and have automated trading with various
brokers. By looking at them, you can probably figure out which brokers have
APIs for stocks, forex, options etc.

You may even be able to reverse engineer some of their trading algorithms ;)

------
bismuth
The quantitative finance section on arxiv: <http://arxiv.org/archive/q-fin>
though the papers are quite hard to get into terminology wise.

------
ritonlajoie
You may like Ninja Trader, it's free for personal use. This is a great free
automatic trading application, you code in C# using their API.

------
regularfry
Nobody has suggested it yet, so for context I'd suggest reading The
(Mis)Behaviour of Markets, by Mandelbrot. It's not about trading algorithms
per se, but it does put the maths behind a lot of them into perspective. It'll
also tell you why certain strategies (like selling cheap options) are much,
much more dangerous than others.

------
SkyMarshal
Not quite what you asked for, but check out <http://www.collective2.com> for
this.

Also check out Janestreet, algorithmic trading is their business and they open
source some stuff. Maybe something useful there:

<http://ocaml.janestreet.com/>

------
henrikschroder
I actually thought googling "bankruptcy engine" would yield some good results,
but apparently not. Oh well.

~~~
SkyMarshal
Rofl, <3 that. Too true.

------
arthurdent
Seems there have been a lot of posts on HN in the last week or so about
algorithmic trading.

There doesn't seem to be a shortage of interest in this field either, but the
threads quickly get lost in the HN jumble. Does anyone want to volunteer to
take this conversation somewhere else?

------
iworkforthem
I am into proprietary trading too. For retail investors like myself, seems
only NinjaTrader & MetaTrader allow algorithmic trading capabilities via API.
But both uses its own language.

------
CoachRufus87
i'm curious as to how you plan on implementing this system...algorithmic
trading has always been an interest of mine, but I never quite understood how
it really worked

