
Ask HN: How to launch an algo trading side project? - jason_slack
I&#x27;ve been writing my own implementations and using 20 year historical stock data. Have been test trading to see how my logic is working.<p>However, I have an issue. When I think I am ready to use real money how do I do this? Do I need to get ETrade or Robinhood to actually start automated trading?
======
boulos
Interactive Brokers (IB) is your best bet. However, I’d like to reiterate as
others have stated that you need to be prepared to lose it all (and moreover
that you likely need much more capital than you realize to stay solvent on
down swings).

Don’t take this the wrong way, but it seems like you haven’t done a lot of
research into this: Interactive Brokers is one of the first things that comes
up when you search on Google (etrade, Robin Hood or any other retail trading
just isn’t credible).

Finally, is your 20 years of historical data for just open/close? Is that your
execution plan? (Otherwise you can get really tripped up as you move to
intraday trading)

~~~
jason_slack
I know about IB but initially I thought that I couldn't use their API unless I
also ran on their platform. Meaning I can't use my own VPS, etc.

I have been using 1 minute bar data.

~~~
gilj
When using IB, you can trade on your own machine/vps/whatever. I suggest you
do this, and set it up such that you trade on an IB paper account, make sure
everything is running correctly and in line with your backtests, and then
start trading with some real money. Good luck!

~~~
jason_slack
Great advice. Thank you.

------
rco8786
To other folks in this thread - your warnings are justified but please stop
conflating “algo” and “HFT”.

------
1024core
1\. Only use money that you can 100% afford to lose.

2\. Once (if..) you've made some profit, take out your initial investment and
now you're gambling with free money.

~~~
dkyc
_" Once (if..) you've made some profit, take out your initial investment and
now you're gambling with free money."_

I never understood this sentiment, and I'm surprised to see it so often on
Hacker News. It's irrelevant whether the money came from profits or initial
investment - money is money? There is no need to distinguish the both, it's
purely psychological.

~~~
lev99
It's a classic example of loss aversion[0]. In my mind the real loss risk of
this venture is the opportunity cost spent developing the algorithm. Many
people that fail to build trading algorithms would have been very successful
if they invested their time/expertise doing something else. The person stands
to lose not only his cash investment, but all of the cash he could have
received if they focused their efforts on a less risky venture[1].

[0]
[https://en.wikipedia.org/wiki/Loss_aversion](https://en.wikipedia.org/wiki/Loss_aversion)

[1] I'm all for risk, but Vegas has better odds.

------
wpasc
1\. Read Flashboys

2\. a) Realize how CRAZILY far many HFT firms have gone to inch out literally
every single microsecond and achieve economies of scale with billions in
assets

2\. b) individual user-scale tools like ETrade won't get your trades in fast
enough/any cost per transaction fees will erase any gains

3\. Invest said capital in something like Betterment

If your model has been truly beating the market for a sustainable amount of
time, I apologize for my comment and go all out :)

~~~
topbanana
HFT is only one type of algo trading.

You can make money with simple trend following algos that execute perhaps only
a few times a week.

To the OP, if you aren't sure about market execution, it makes me think you
haven't modeled for slippage. Make sure you do, as that nudges many profitable
algos into the red.

~~~
cjalmeida
And maybe trading costs.

------
lordvon
I have only tried one algo trading platform, Interactive Brokers. API
documentation: [http://interactivebrokers.github.io/tws-
api/index.html#gsc.t...](http://interactivebrokers.github.io/tws-
api/index.html#gsc.tab=0)

Their live market data is gated at 250 ms, so definitely not for HFT. Market
data subscriptions are only $5 / month (US Equities, and others I don't
remember), and commissions are low.

You can program in C#, Java, VB, C++, or Python. I use their C++ platform. It
is not particularly pretty, but it seems well-tested and works. Perhaps most
importantly, they have an awesome support phone number. They are responsive
and very helpful.

------
linsomniac
The easiest way to make a small fortune in trading is to start with a large
one. :-)

HFT is like picking up pennies in front of a steamroller.

~~~
jason_slack
> HFT is like picking up pennies in front of a steamroller.

here's to hoping I can stay one step ahead, lol. It's risky.

------
cdnsteve
Check out [https://www.quantopian.com/](https://www.quantopian.com/)

~~~
LiamPa
+1 Read the zipline source, I learnt a lot in how to structure trading code /
solving common problems.

------
kolbe
How often do you trade and how important is fast executions? I've noticed many
programmers try to shoehorn in execution algorithms into a strategy that
doesn't need it. If you can run an analytics program that just sends you a
trade alert from which you manually execute, that will save you a lot in
systems risk.

Also, the fact that you're asking this question tells me this is very very
unlikely to be a good idea. Be careful.

~~~
jason_slack
My thought is to trade as infrequently as possible, sort of batching trades to
do all at one. Nothing is automated for me yet. I am just manually reviewing
my output.

------
patatino
Check out this podcast. Dis guy had a algo trading as a sideproject (now
fulltime):
[https://www.youtube.com/watch?v=Co_GYku903k](https://www.youtube.com/watch?v=Co_GYku903k)

~~~
SirLJ
Looks like this guy is looking to create a Quantopian clone, so in the end it
feels like the whole interview was a bit of advert for the signal service to
come...

In fact all podcasts of this guy are like than, a lot of charlatans and
scammers are appearing on the podcast all the time, probably 99% of the
guests...

~~~
patatino
I listened to ~10 podcasts and most people are very humble and talking about a
lot of mistakes they made and I learned a lot. I cannot share that 99% of the
guets should be charlatans or scammers. Did you listened to most of the
episodes?

~~~
SirLJ
Yep at least the first 50, maybe more, beside the occasional guest, every one
is selling a chat room, indicator or similar scam, just listen to this
particular podcast where the host comments on the expected 60% yearly return
and that it should be easy to do...

~~~
patatino
It's crazy in this business how everyone tries to sell something but I think
there is still value there to extract, don't you think? It's a honest question
because I'm new in this space and I always enjoy learning from others,
especially from their mistakes so I can try to avoid them.

I would never copy anything they say or buy anything they offer.

~~~
SirLJ
The problem is that they will “teach” you the wrong things, because they do
not have trading experience... all is smoke and mirrors with made up stories
and nothing to learn... I was listening in the beginning as entertainment and
because there were one or two real guests, but in the life is too short to
waste your time on those scammers...

------
a_c
Mind sharing where you get your market data? Are you using a paper account
that you can simulate the trades? what markets are you participated in?

Apologies for the burst of questions. Am interested in testing out some ideas
too

~~~
jason_slack
Historically I have scraped data, but now:
[http://eoddata.com/default.aspx](http://eoddata.com/default.aspx)

~~~
turc1656
Be careful. EODdata has a lot of data issues. Seriously...a lot. I would
recommend checking out Alpha Vantage. It's free and I didn't see any of the
issues with their data that I saw with EODdata.

~~~
jason_slack
WOW! OMG. Thanks for mentioning your experience. Can you share more about
this? My e-mail is in my profile if you want to do it privately.

Thanks for mentioning Alpha Vantage. I will check them out.

~~~
turc1656
No need for private communications. Probably better to have here anyway for
anyone else who is curious.

I was thinking of using their data to help test and run a system I am working
on. I was excited to see that their stuff is free for what I need, as long as
I wait until after a specified time to grab it (9 PM EST). I started looking
at the data and noticed that it doesn't track corporate actions well at all.
This becomes very problematic when you need to track your day over day returns
for a security. There were also unexplained gaps and spikes in historical
data. For index data, the lists seemed inconsistent in that things would
seemingly drop out and then back in from day to the next leaving you with
having to manually acquire the value of certain indices manually or having to
process a historical file for that index to fill the gap. Historical files are
not adjusted properly for actions like splits and ticker changes. Basically it
boils down to there being a very, very limited use case for a provider like
EODdata. Also, it should be noted that while I was never a paying subscriber,
their subscribers have said there is virtually no customer service. I thought
that might be the case looking at the website. It seems like EODdata was
basically put on autopilot a long time ago and just left to run with no one at
the helm. From a security standpoint, I recall them not using HTTPS (that
should give you an idea of how long they haven't updated their site or
security) and I clearly remember them storing passwords as plain text because
I did a "forgot password" and it emailed it to me. In fact, I just did it
again now to make sure I wasn't crazy, and sure enough, they haven't fixed
anything. Still no HTTPS (even when manually typing in the HTTPS:// prefix)
and it just emailed me my password (which was set to a bullshit password that
I would never remember after discovering this gaping, unforgivable security
travesty). I also remember having issues with some of the downloads not
working correctly.

All in all, I'm damn near certain someone has this site on full autopilot and
never fixes, updates, or even maintains anything. The plain text password
nonsense should be enough right there to disqualify its usage, even if you are
only using the free service(s) and never provide your payment details, and
even if you use a unique password. If you were actually thinking of providing
payment details, it's a clear and total no-go.

There are additional reviews from other people at these links:
[https://quant.stackexchange.com/questions/3284/is-
eoddata-a-...](https://quant.stackexchange.com/questions/3284/is-eoddata-a-
good-data-source) [http://www.trade2win.com/reviews/data_feeds/82-eod-
data#revi...](http://www.trade2win.com/reviews/data_feeds/82-eod-data#reviews-
wrapper)

They mention a lot of what I have talked about and more.

~~~
jason_slack
Thank you. I appreciate the word of caution. I'll look at the place you
mention too.

------
kerberos84
If you are not planning to do HFT then let your system create signals and send
them to your e-mail. Then buy/sell manually.

------
iothetiger
I went with Interactive Brokers and their python API.

------
someguy_lvl9001
I am working on the same thing. I'm still in the model building phase and I
still need to complete my backtesting framework. You seem to be ahead but I've
already looked into the trading API process.

Go with whatever broker offers a trading API obviously, and has a fee
structure that matches your trading strategy. If you can afford to lose a bit
on the trade through commissions and your strategy is swing trading then you
can pick pretty much any broker. If you need specific fill times and
commission percentages before your strategy pays off then you are stuck with
Interactive Brokers ($10k minimum[1], low comms, fast exec) or Robinhood (low
minimum, free comms, slow exec). And technically Robinhood doesn't have a
public API[2] so you would have to do a little reversing of their app's
messaging. And then there's terms of service... yeah. Probably not a big deal
for a small fry like yourself.

Next would be testing. You've got isolated tests with historic data but try
get a broker that has a paper money version. TDAmeritrade's paper & api is
good (but they are expensive). To be clear you would be using your algo to
trade in a paper money account. That lets you get more realistic trade tests
and if you blow something up you are still ok!

Keep in mind Robinhood is free because they are paid to route your orders to
hedge funds and they are re-ordered in the book. You will get ok price
execution through them but never the best. One thing you should include in
your backtests/simulations is order execution delay and fill price variance.
If you place market orders they won't always fill at the quote price. If you
place limit orders they may take a while to fill (or not at all) unless the
price is within the other side's order prices.

Last is the pattern day trading rule. Make sure you have a strategy that won't
make more than 3 day trades in a 5 day window, otherwise your account might
get suspended for a few months. It's a pile of horses but you can get around
that by simply having more money in your account. > $25k will buy you
unlimited day trading privileges.

Just curious what your strategy is based on if you don't mind giving a few
details.

Good luck!

[1][https://www.interactivebrokers.com/en/index.php?f=4969](https://www.interactivebrokers.com/en/index.php?f=4969)
[2][https://support.robinhood.com/hc/en-
us/articles/210216823-Ro...](https://support.robinhood.com/hc/en-
us/articles/210216823-Robinhood-API-Integrations)

~~~
testeroniblla
One suggestion to avoid the PDT rule (and in general be more efficient with
your capital) is to look at futures. For instance rather than trading SPY
trade ES. ~8k per contract minimum to control ~140k (50*Index Points) of the
SP500 and no PDT.

~~~
someguy_lvl9001
You say more efficient, I say leveraged. Potato Potato. :) Yea futures are
much more interesting in my opinion and no PDT is way better!

------
crimsonalucard
What if I want to do fundamental analysis? Anyone know of any cheap or free
data sources that have parseable financial statements?

------
thisisit
If I may, what is the timeframe for your algo trades?

If it's short one like say in minutes, then you might want to test it with
small amount of real money before going all in. The reason is orders fills are
imperfect. So, cost price you had during backtests vs realtime might differ a
lot.

~~~
jason_slack
This is the hard part. How much money am I willing to part with :-) Some of
these places I think need $25k in an account.

~~~
boulos
Actually, that threshold is an SEC one for being a “frequent trader”. So
anyone who doesn’t is just not telling you that you’ll need at least 25k (and
IIRC, _at all times_ ) to make the volume of trades you want.

------
paulgb
A friend of mine did this and had the algorithm output the trades and then he
made them manually. If it trades too quickly for that to work, trading costs
will likely be an issue for you anyway.

Remember hindsight bias, and as others have said, don't invest more than you
can lose.

~~~
mring33621
This is how I did it. My problem was that I listened to my algos for entry
points but let my emotions take over for exits, so I often left money on the
table.

~~~
jason_slack
What you did is a natural instinct/emotion! I can see this happening to me
also.

------
usgroup
Depends...

What instruments are you trading?

What frequency?

What size?

Why does it need to be automated?

------
csomar
You said you have been test trading. What does that mean? Where are you
testing? Which platform are you using?

------
godelmachine
Since this topic is trending, I would like to grab this opportunity to ask if
anyone's knows any material on how FPGAs are used in HFT/ Quantitative
trading? Any books/ web sites / blogs?

Any pointers would be highly appreciated?

------
julox
for development in Python you can use
[https://www.backtrader.com](https://www.backtrader.com) with simulations and
live trading

------
1001101
TradeStation. I wouldn't bank on any HFT, unless you're doing FPGAs, and
you've got the 'good' port on the switch.

~~~
jason_slack
I have always wondered if FPGA's are really needed. Can you share more info
about this?

~~~
jetti
Are you doing HFT trading? If so, then the FPGA could help. Are you just doing
plain ol' algo trading then you wouldn't really need it.

~~~
jason_slack
plain ol'. Currently I dont even have this running on a VPS yet. I am just
using a Mac Pro in my house, lol. But I am not "live" yet.

------
tty7
Robinhood have an unplublished api. There are hard limits on the number of
trades you can do. But i would use that

~~~
jason_slack
Can you share more about this? IIRC web isn't ready yet so mobile only.

~~~
tty7
[https://github.com/Jamonek/Robinhood](https://github.com/Jamonek/Robinhood) I
used that a few months ago, you might need to sniff the traffic if there has
been changes. it wasnt too difficult

------
SirLJ
I am glad you ran with my idea and found an edge pretty quick!

What is your average yearly return on the 20 years back testing?

~~~
jason_slack
Your advice was absolutely amazing and cemented my desire to continue! Now I
need to figure out how to actually execute. If you have any advice! I keep
staring at a list of possible transactions and manually checking how my trade
might have turned out and keeping a gain/loss figure.

I was hoping you would chime in on this thread. I was not so comfortable with
the work I was doing and felt like my direction was wrong and I was
questioning why I was even going to try this. Your advice helped me decide to
continue.

~~~
SirLJ
Hey Jason, I am glad I could help!

Before you start with the execution, make sure you did use the 20 years of
data, go and back test all the years, they encompass few market cycles, so you
will know how your strategy will perform in bull and bear markets.

For the execution part, when I started, I had an ssh session open to my server
with the output of my bot running on the screen and had it configured to chime
with something like” EXECUTE buy 2000 HYZ at $65 stop 56. I did it manually
for a few weeks until I made sure everything worked properly and then did the
order execution with the API. Interactive Brothers are a good choice with a
good API.

I would stay away from Quantopian and other platforms that are holding your
code, because this is the prefect target for a hacker and the perfect crime –
nothing to steal or sell, they will just copy your winning strategy…

------
1e-9
The Interactive Brokers (IB) API would probably be your best option for
trading on the side.

Here are a few words of advice regarding IB:

\- Thoroughly read and understand the IB documentation. \- Be sure that you
never get close to any of IB’s risk limits that can result in the automatic
liquidation of some, or all, of your positions. \- If you don’t want to put up
enough cash in your IB account for day trading, be sure you don’t exceed the
trading limits that will get you classified as a day trader and thus prevent
you from further trading until you add more cash. \- Be aware that the IB
market data can have significant inaccuracies and/or delays.

Here are a few words of advice on automated trading:

\- Thoroughly understand the exchanges that handle the instruments you want to
trade. Each exchange has online documentation explaining details of traded
instruments, matching algorithms, pricing, and market data. Even though you
won’t be accessing the exchanges directly, these details have an effect. \-
Thoroughly understand IB’s order routing and how they construct their market
data (if you use it). \- Just because your historical data says an instrument
closed at a particular price, that does not mean you could have actually
bought or sold at that price with IB or even with direct market access. The
same is true of tick data constructed at 1 minute, 1 second, or even
millisecond resolution. Trading simulations can be terribly inaccurate if you
don’t have a reasonable approximation of the exchange order books, the
exchange latencies, IB’s latencies, and your own trading system latency. To
counter this, you need to be extremely conservative in your assumptions about
getting an order filled at a particular price. \- Even with accurate market
data and exact latency measurements, predicting future profits by backtesting
over historical data is perilous. I advise having a strong background in the
statistical/machine learning methods for constructing and analyzing
appropriate training, testing, and validation sets. \- Don’t make a trade just
because some analysis suggests the market will go in that direction. Doing
this will probably get you hurt and it harms markets by pushing them to
overshoot or undershoot the correct prices. The trend is your friend until he
turns around and stabs you in the back. Make a trade because that’s where the
market _should_ go and probably _will_ go eventually.

My best advice for trading profitably is to do something to improve a market.
Focus on areas that can benefit from your strongest personal advantages
(mathematical, technical, domain knowledge, etc.). Gain an understanding of
the exchanges and instruments that allows you to recognize ways to make prices
more accurate. This improvement could be in an absolute sense for individual
prices or it could be in a relative sense in terms of how instruments should
be priced with respect to one another. It could also be in terms of adding
liquidity (offering to buy or sell) where needed and justified. Improving
markets improves our economy by helping it to better prioritize resources and
limit risks. That is a valuable service and the market will pay you for it.

------
adjkant
Consider cryptocurrency over stocks, as it comes with exchange API's and
low/no fees.

~~~
toomanybeersies
Crypto doesn't have 20 years of historical price data though.

~~~
wruza
Isn’t history-based trading a myth? I mean, technical analysis of markets in
general.

~~~
gilj
It's not about using the 20 years of data to fit your model, it's about using
the 20 years of data to test your strategy on (a process called backtesting).
Once you know how it performs historically, you can develop an intuition for
where it breaks down, and try to alleviate the problem (keeping in mind
overfitting and all those types of issues).

------
arthurcolle
have you looked into quantopian?

~~~
jason_slack
Thanks. I just looked at them. I like their idea of letting you test, but all
of my code is in c++. I have 6 apps that need to continually run to fetch new
data, import it, create models, make predictions and execute them. I'm not
ready to share code, but I could always share how my architecture is works.

~~~
rb808
[http://quantconnect.com](http://quantconnect.com) is a better alternative for
you, take a look.

~~~
jason_slack
Thanks. Why do you feel they are a better alternative?

~~~
rb808
Considering you can't use Quantopian, it can't be much worse.

TBH I've never used them, I just remember looking at a platform that was much
more tech agnostic, I'm not sure if this was what I looked at, but looked
potentially useful.

------
lee101
As others have said checkout cryptocurrency, im founder of
[https://bitbank.nz](https://bitbank.nz) providing ai forecasts, charting and
bulk data for backtesting bots. Checkout our blog at
[http://blog.bitbank.nz](http://blog.bitbank.nz) for info on how to easily
develop profitable trading algorithms from some of our open cryptocurrency
data. Also checkout our GitHub

