
Python For Finance: Algorithmic Trading - pastefka
https://medium.com/@kacawi/python-for-finance-algorithmic-trading-60fdfb9bb20d
======
lowpro
The main issue I found in algo and financial aspects of programming is that
the market is a zero sum game, and my intro knowledge of finance and
algorithms, even when I know python, are no match for MIT PHD Quants who does
it full time. There's no real way to compete with that, and therefore I would
lose money, even if the data showed it might be successful in the future,
firms and full time workers on algo trading would simply be faster, more
focused, have more funding, and be able to quickly and constantly adapt at the
scale an individual could not.

So despite the fact that the subject is interesting, I'd consider it a waste
of time to try and gain anything but a basic understanding of the industry and
how algo trading works.

~~~
milcron
An economist and a normal person are walking down the street together. The
normal person says “Hey, look, there’s a $20 bill on the sidewalk!” The
economist replies by saying “That’s impossible- if it were really a $20 bill,
it would have been picked up by now.”

~~~
Berobero
This quiq seems to support the original posters thesis, though, no? No one's
going to make a living wandering the streets in search of $20 bills.

~~~
rxhernandez
I think you might be extending this model beyond its boundaries of usefulness.

------
pmart123
Personally, I struggle to see the competitive advantage Quantopian brings.
They use retail brokerage platforms to facilitate trading, which rules out
anything close to HFT. Then, they are tied to any financial data vendor
(Morningstar in this case) to not offer too much visibility on the underlying
data. As others have mentioned, this makes it tough to validate aspects like
adjusted vs. as reported earnings, how delistings are handled, etc. From my
experience, getting/making sure the data is accurate is a ton of work, even if
it is from good sources and you can see all the actual data. The moment an
investor/trader on the platform gets traction is the moment they want
Compustat data, exchange data, Bloomberg for fixed income, and will trade
through Instinet/Flextrade, etc. The moment the platform is successful is the
moment Morningstar could pull the rug out from them. If someone has better
understanding or knowledge on Quantopian in particular, I'd be interested to
hear why.

~~~
TACIXAT
My understanding is that Quantopian has no interest in being a platform for
HFT. It's there to democratize trading strategies and test them in a sound
way. They do a lot to have a good backtesting platform and clean the data that
is available in the platform. If I wanted to automate a strategy, I'd have to
figure out the Robinhood API and basically recreate what they have. Instead, I
can use their platform, their backtests, their free data, and build strategies
that efficiently lose money.

I think the other goal being that if you create a profitable strategy, you can
enter it into competitions, trade with other people's money, and make a
profit. I think the platform is a value add, I have an account and use the
research notebooks. Some day when I more spare time I'll make some trading
algorithms.

~~~
SirLJ
The main problem with Quantopian is that the data missing especially for
delisted stocks, here are some examples you can test yourself: AA, CWH, NIHD,
PANL, HTZ, DSL - I have a whole list I found very fast, just imagine what
other mistakes could be there... And the other problem is that they are my
competitor, so I would never give them my trading strategy or ideas I am
testing at the moment...

~~~
dsacco
This is false. Quantopian sources their historical and real time data from
Nanex, and the data is explicitly free of survivorship bias (i.e. they
maintain data for delisted equities).

I don't know what you verified, but it directly contradicts my own experience
and the FAQ: [https://www.quantopian.com/faq#data-
sources](https://www.quantopian.com/faq#data-sources)

To your second point, as another commenter said Quantopian doesn't see your
algorithm unless you give them permission, as outlined in the terms of service
(which are very readable, by the way).

 _NB:_ I am not unaffiliated with Quantopian and don't use the platform for my
own trading, but I have tried it out before.

~~~
SirLJ
Please go back and check and you'll se the data is missing for the symbols
stated above, and I am sur for many more based on my limited test... everyone
can see I am right by trying to request the data, nothing false about that...

~~~
dsacco
_> everyone can see I am right by trying to request the data, nothing false
about that..._

Oh, they can? Well that's funny, because I just went onto Quantopian and
checked for myself.

[http://imgur.com/a/VwMUJ](http://imgur.com/a/VwMUJ)

[http://imgur.com/a/VV68C](http://imgur.com/a/VV68C)

Gee, would you look at that...they _are_ there.

This took me all of five minutes. Do you have anything else I can easily
disprove while I'm at it?

~~~
SirLJ
The problem is the data is missing for specific dates... I just can imagine
what else is missing or it is wrong...

~~~
dsacco
This is really trying my patience. You made a clearly incorrect claim without
mentioning specific dates and challenged me to disprove it. When I did
disprove the claim, you shifted the goalposts with this backpedaling remark
about how only specific dates are missing. You continue to make these claims
and put the burden of proof on everyone else because you can't be bothered to
defend them when you're challenged.

I'm not searching for a needle in a haystack for you. Make a falsifiable
claim, and make the _entire_ claim, with the specific dates you're talking
about, or stop this astroturfing against Quantopian and QuantConnect that
you're doing.

~~~
SirLJ
My goal is to show people how start to think for themselves and hopefully
create lifestyle business around trading or invest for themselves instead of
using "buy and hope strategy" and how to do it without losing a single penny
while learning and to show them the barrier for entry is pretty low... But you
are right, why bother when every time I gave an idea, the day traders with the
thousands of dollars data feeds are all over my posts... no good deed goes
unpunished here at HN I guess :-(

~~~
gnaritas
Shitting on other people products with vague and inaccurate criticisms isn't a
good deed, it's just bad form. People don't need you to "show them how to
think for themselves." The other guy is right, either make a sound and
specific criticism or don't say anything. You did exactly what he said, you
made a bad argument which he refuted, then you moved the goal post so as not
to appear wrong, while still leaving out the necessary detail to disprove your
new vague claim, rather than admitting your mistake and making a better
argument that included specifics about what is missing.

------
hendzen
If you want to build a real algorithmic trading strategy.. please pick up a
real textbook such as Qian's Quantitative Equity Portfolio Management.

You should understand the following concepts at a minimum:

\- Markowitz portfolio optimization (mean-variance analysis)

\- Beta-neutral portfolios (i.e. using MSCI BARRA, sector ETFs or PCA factors,
etc)

\- Alpha decay

\- Time series analysis (autocorrelation, GARCH, ARMA processes)

\- Basic price-based signals (momentum, volatility, value, etc) \- etc, etc.

~~~
ndesaulniers
Any others you (or anyone else) recommend?

~~~
remyp
Evidence-based Technical Analysis by David Aronson is a great intro.

------
JabavuAdams
So, I got semi-seriously interested in this around the end of last year.

My takeaway is: it's not about implementing a couple of trading strategies.
It's about implementing a pipeline that rapidly allows you to test what-if
scenarios.

I might have like 10 ideas a day for strategies. How many of those can I
rigorously validate per week? What about variations? I.e. tuning various
hyper-parameters? Combinations?

How quickly can I recombine data from various sources into exactly the layout
I need to train and test this or that strat?

Me:

1) Build some simple forecasting NNs.

2) Realize I need to be able to generate and test ideas waaay faster. Start
working on infrastructure.

3) Get impatient -- get some "bright" ideas, do some manual trading.

4) Make some money on the first day, lose double that on the next two days.

5) Decide to ban myself from manual trading as obviously I'm an addict.
Resolve to only do algorithmic trading.

6) Back to coding infrastructure. Get bored ... do some manual trades.

7) After losing around $20k, stop the trading madness, and go back to buying
and holding great tech companies.

8) Enjoy my 25% returns.

~~~
ndesaulniers
> 5) Decide to ban myself from manual trading as obviously I'm an addict.

> 7) After losing around $20k

------
jaredbroad
QuantConnect recently announced full python library support; and we have
launched
[https://www.quantconnect.com/tutorials](https://www.quantconnect.com/tutorials)
to help people write quantiative strategies in Python.

QuantConnect & LEAN gives you ability to do tick->daily resolutions; for
equity, morning-star, future, option, forex and cfd trading - all with a fully
open source project which includes samples of data to get you started.

The grunt work is still done in C# so its faster than other full python based
backtesting engines. Edit: I'm the Founder @ QC.

~~~
SirLJ
When promoting your company, please say so, also on QuantConnect one cannot
actually see the data, so there is no way to verify how good the data is...

~~~
jaredbroad
Sorry SirL! I edited it within 10 sec to be explicit but you must've refreshed
before I'd updated it =)

We provide FX/CFD data for free download; the other data is restricted by the
exchanges sadly so we can't make it available. Instead we put the tools we
used to make it into LEAN format into LEAN (/Toolbox) so you can purchase it
and convert it yourself.

~~~
SirLJ
Thanks jaredbroad, just out of curiosity, what are the average yearly returns
on your top 3 users, how much are you investing with them and how do you split
the profits?

------
tejaswiy
On a side note, the market continues to do well and I've been noticing this
trend of active-trading, real-estate investing gurus crawl out of the wood
work selling services.

Please do not try to trade actively unless that's your full-time job. Passive
investing using index funds is definitely not sexy, but it gets the job done.

~~~
gnaritas
> Passive investing using index funds is definitely not sexy, but it gets the
> job done.

Only if you have a significant amount of capital to play with. People looking
into active trading are doing it because they don't have the necessary capital
to make passive investing meaningful; they're looking for much much larger
returns that you can get from an index fund, which necessarily comes with more
risk. Traders are gamblers.

~~~
slyfocks
This is extremely reckless investment advice.

Active trading, by definition, incurs higher transaction costs than passive
investing. Since transaction costs are generally priced per trade, active
trading is more costly on a percentage basis for individuals with less
capital.

~~~
gnaritas
> This is extremely reckless investment advice.

It wasn't investment advice, nor did I say it was a good idea; I'm merely
telling you why people choose active investing: they believe they can do
better and there's enough stories out there of the "lucky" person who did
better to keep that hope alive. In all likelihood they're going to lose their
money, but you'll never convince people trying to win a lottery that they're
better off accepting they can't. I literally said traders are gamblers,
obviously that means I think active trading is gambling, that's not investment
advice.

------
TACIXAT
I went to the Quantopian conference for their basic training on algorithmic
trading. This blog post was pretty much what they covered, intro to pandas and
a simple strategy. There is a lot of educational material on their site too
(which is what you ended up getting in the paid training).

My biggest thing with the Python for Finance books - I know Python, I want to
learn finance. All these books are the inverse of that, for people who know
finance and want to learn Python. There is a good site for quantitative
economics [1] that has tutorials in Python and Julia. I would love a
mathematics of finance book that had the examples in Python.

1\. [https://lectures.quantecon.org/py/](https://lectures.quantecon.org/py/)

~~~
wtvanhest
The problem with learning finance that it takes a lot og effort. A good place
to start would be taking the 3 CFA exams. After about 1000 hours of study, you
will have a basic grasp of finance.

~~~
dsacco
This is utterly the wrong way to learn finance for quantitative trading. That
would be an unproductive use of time.

Read _Options, Futures and Other Derivatives_ and _Algorithmic Trading and
DMA_ and you basically know everything you'll get from public sources that
could be meaningful for trading.

If he knows how to code and is looking for "mathematics of finance" he should
start with those, not the CFA.

~~~
wtvanhest
That is your opinion, but if you look at the resumes of scientific active
equity fund managers and quants, they all have it. Just understanding the
quantitative side leads to vast underperformance over the course of market
cycles...

~~~
dsacco
What resumes? Can you show me these vast numbers of resumes with CFAs? Are you
talking about fund _managers_ or quants? If your goal is develop quantitative
trading strategies, you shouldn't be getting a CFA because it's superfluous.

[http://news.efinancialcareers.com/us-en/150266/why-the-
cfa-w...](http://news.efinancialcareers.com/us-en/150266/why-the-cfa-wont-
help-you-get-a-job-in-a-hedge-fund/)

On another note, even if what you were saying were true (and it isn't), it
wouldn't demonstrate that it's useful to have the CFA. Certifications are
usually helpful for HR or for regulation, not for actual knowledge. And
everything you need to know about finance can be learned in _far_ less than
1000 hours for trading if you're in a research or development role.

~~~
wtvanhest
Linkedin. Go there, search for scientific active equity (that is what a
professional quant manager is called).

Sort by who manages real money and who doesnt. Anyone managing $1bn or more
all has it. That is not very much money to manage.

~~~
dsacco
I feel like we're talking past each other. I'm talking about a role where you
develop strategies as either a researcher or a trader. Higher up the chain,
sure, I can admit there's a utility to the CFA.

~~~
wtvanhest
I think the main issue is that you need a solid grasp of the qualitative side
of finance to build working quant models. Pms are responsible for quant model
construction and they all get cfas because that is base level finance
understanding.

------
ncyclopediae
Here's a dataset on Kaggle that can be used in this process
[https://www.kaggle.com/biomimic/periodic-table-of-
elements-m...](https://www.kaggle.com/biomimic/periodic-table-of-elements-
mapped-to-stocks?03)

------
kyleblarson
This is nothing more than gambling. Let's say you have 20k to play with. You
would be far better off in the long run to put 15k in Wealthfront and use the
other 5k as a bankroll of 25 buyins for 1/2 no limit holdem to learn the game.

------
brobinson
The article mentions a few of the pitfalls of backtesting, but it does not
mention one of the best tools at your disposal in backtesting: Walk Forward
Optimization/Analysis

[https://en.wikipedia.org/wiki/Walk_forward_optimization](https://en.wikipedia.org/wiki/Walk_forward_optimization)

See also: "The Evaluation and Optimization of Trading Strategies" by Robert
Pardo

------
moonforeshot
I'm wondering... why not scala? scala gives me type-checking tools (eg
dependent types) so I don't shoot myself on foot

~~~
dsacco
Scala is absolutely used in finance, especially for interoperability with Java
in Java-heavy firms. But it's data analysis libraries aren't as mature as the
pandas/scipy stack. You'd use them for different parts of a trading pipeline.

------
hagakure0c
I use R for almost all quantitative tasks but not as a trading platform. It's
great for research and testing though.

------
NicoJuicy
I have a simple manual algorithm that took me from 650€ to 1450€ now (as test)
in 2 months. Does someone here have experience with algo trading in c#?

~~~
Veratyr
I won't say I have "experience" but quantconnect.com is as good a platform as
I've found for C#. Their engine is open source so you don't have to rely on
them either, though you'll need to provide your own data if you run it
yourself.

~~~
rl3
QuantConnect also offers tick-level data, though not full resolution if memory
serves. Still way ahead of Quantopian in any case.

~~~
jaredbroad
Futures have millisecond timestamps (trades/quotes), equity trade ticks are
rounded to the nearest second, cfd/forex are millisecond quote bars. For
options we have minute resolution data =)

~~~
Veratyr
Worth noting though that Options data is super hard to get running on
QuantConnect due to the memory limits and lack of index options.

~~~
jaredbroad
Agree, we're working on making it easier now. We patched a bug today which
should make it more efficient.

~~~
dsacco
Do you have a public email for getting in touch with you? I'm interested in
chatting.

------
nickthemagicman
How is algorithmic trading not equivalent to astrology?

Nothing can be predicted because there's way too many confounding factors.

~~~
PaulRobinson
Algo trading typically utilises technical analysis which is basically patterns
proven to repeat in markets for a variety of fundamental reasons, or
fundamental analysis (e.g. algorithmically valuing and pricing options based
on underlying fundamental data, sentiment analysis, etc.), and buying/shorting
as appropriate. It is based on scientific methods: empirical evidence being
used to validate hypothesis that produce results that can be shared and
repeated by others.

Astrology is story-telling with no scientific grounding.

~~~
gnaritas
> technical analysis which is basically patterns proven to repeat in markets
> for a variety of fundamental reasons

Technical analysis is merely another name for hindsight bias. Those patterns
only look like they repeat in hindsight because you're ignoring all the failed
patterns that setup right but failed to play out and thus don't look like the
pattern in hindsight.

Technical analysis is exactly like astrology and is in no way grounded in
science. Quantitative analysis is based in the scientific process, technical
analysis is superstition created by visual traders who think they see patterns
but fail to understand biases. You are confusing the two, technical analysis
is a pattern library of nonsense like "head and shoulders" or "dead cat
bounce" based purely on visual patterns in price data that only "appear" to
repeat due to hindsight bias. It is in no way remotely scientific.

I've spend thousands of hours testing trading patterns in Forex, they're all
nonsense that lose as much as they win over any decent sample size which
equates to random.

Edit: to the guy below, You should ask for sources that back up the claims
that it does work. You won't find any, it's trader superstition, not science.
You don't have to prove things don't work; you have to prove they do. At least
if you want to call it based in science. However, the field of quantitative
analysis itself is proof technical analysis doesn't work; it's what happened
with people who actually knew math got into trading and discovered the
technical analysis superstition that visual traders who lack math skills made
up.

~~~
lz400
Can you provide sources that back up your claim with studies of inefficiency
of technical analysis?

I'm not busting your balls, I actually agree with you but I haven't seen it
really proven.

~~~
sumedh
> Can you provide sources that back up your claim with studies of inefficiency
> of technical analysis?

Not the original guy but I just want to point out that Warren Buffett, Peter
Lynch do not believe in technical analysis.

~~~
lz400
Well they are obviously fundamental type of investors, and I have no doubts
this type of investment is superior to pattern matching and chart gazing. But
I see all those guys doing their pretty technical analysis charts in stocks
and bitcoins and even though I think it's all BS, I wonder, is it? I mean, can
you get a small edge? why would they be at it all day if it wasn't making them
money?

~~~
gnaritas
> I mean, can you get a small edge? why would they be at it all day if it
> wasn't making them money?

It's gambling, they win randomly and falsely attribute the wins to tech
analysis and dismiss all the losses as them "reading" it wrong. The random
wins attribute to an addiction and superstition but they'd win just as much
flipping a coin to decide whether to buy or sell. These people are very bad at
math, if they were any good they'd be doing quantitative analysis and looking
for actual patterns in the data that stand up to real scientific inquiry.
Technical analysis is to quantitative analysis what alternative medicine is to
actual medicine.

------
SirLJ
It is a great article, but why on earth someone will use a service like
Quantopian or similar service?

They are your competitor and who will prevent a disgruntle employee or a
hacker to steel your successful trading strategy?

Just buy some data from eBay, you can get 20 years of historical stock market
data for less than $100 and you can test any trading strategy or idea
imaginable, including trend following, buy and hold ETFs, etc...

The barrier of entry is pretty low and you can develop a great lifestyle
business with no customers, employees and investors around that...

~~~
Blackstone4
Have you done this? If so it would be great to get your perspective

~~~
SirLJ
Yes, I am doing it as a side project and I am looking to retire in a few years
and live off that...

~~~
gricardo99
I'll third this comment. Some more details on what you trade, where you trade,
how long you've been doing it, performance metrics, etc... would be
fascinating.

~~~
SirLJ
As you can see I am paranoid, because the more people trading the same
strategy, the less effective it becomes, not to mention, if someone knows your
trading strategy it can trade against you... it’s a zero sum game after all...

All that said, here are the 2 most important advises I wish someone told me
when I started years ago:

Have an iron clad risk management strategy for your portfolio, e.g. if you are
willing to risk 1% of your capital on any given day and tomorrow happens to be
the Black Monday and all stock go down 50% you still have to be sure you’ll
loose only 1% if you have to sell everything... This is where the back data
comes into play – the goal is to back test not how much money you are going to
make, but how much money you are going to loose in the worst case scenario if
you have to liquidate everything and move to all cash. If you have this
working 100%, you just need a trading strategy with a minimal edge and you’ll
make money in the long run in the market and the compounding effect will
really make you a rich person (rich person to me is someone who makes more
money then he spends, WITHOUT working more than 1 hour a day)

And the second advice is never ever trust and buy a trading strategy from
someone, the fact he is selling or "teaching" a trading strategy means it is
not working and he is trying to make some money by scamming people and selling
the dream, all those trading educators cannot make money trading and that’s
why they are coming with up with those fake trading courses... just check this
website to see the magnitude of this scam and how people are loosing their
life savings... I just don’t know why FBI, SEC and the government is not doing
something to stop those guys...

[https://www.tradingschools.org/](https://www.tradingschools.org/)

Good luck and as I said before, just treat is as a hobby at first and spend
the time testing and you’ll find something that is working for you...

EDIT: some spelling errors fixed.

~~~
princeb
> more people trading the same strategy, the less effective it becomes

you should pick a more scalable strategy, the kind of strategy that becomes
more profitable the more you talk your own book.

