

Show HN: Optimized trading algorithms using IPython parallel and ec2 - twiecki
http://twiecki.github.com/zipline_in_the_cloud_talk/

======
smnl
Cool technical presentation, but trying to optimize moving average trading
strategies is a fool's game - you might find a set of parameters that work in
all out-sample/cross-validation tests, but there's still a good chance it'll
lose money in actual trading going forward - market paradigms can shift,
unforeseen world events, t-cost/slippage higher than what the model accounted
for (especially with more frequent rebalancing), etc.

If you actually want to trade moving average strategies, your best best is
just to diversify and run several different strategies across a variety of
parameter sets and across various sectors/asset classes, without trying to
overly optimize a single strategy

~~~
twiecki
Certainly that's always a risk. Zipline does however support simulating
transaction-costs and slippage so those will be accounted for (as best as
possible).

As for market-paradigm shifts, I totally agree. One way to deal with that is
to constantly re-optimize the parameters based on recent data. This is also
known as walk-forward optimization on which there is a slide at the very end
(see <http://blog.quantopian.com/parameter-optimization/> for a more
information).

Finally, as to running multiple strategies with different parameter settings.
The OLMAR paper that describes the algorithm
(<http://arxiv.org/abs/1206.4626>) has a variation of this where they use a
range of different-length moving averages, rather than just one.

------
aleyan
I noticed you got close prices from Yahoo that you are using for your
backtest. If you get the close prices in real life and optimize, when are you
going to trade?

You have to deal with intraday prices now. Got those? Well they are not split
or dividend adjusted. Good luck.

~~~
fawce
Don't forget mergers. I spoke about this problem recently at Matt Turck's Big
Data Meetup (<http://vimeo.com/60598560>). Quantopian provides zipline powered
backtesting over fully adjusted intra-day data (minute bars) for free.

~~~
aleyan
Unfortunately I wasn't able to attend February's Big Data Meetup, so I missed
your talk. EDIT: (I did just view it online, my comment takes what you said
into account.)

My point about intraday data with corporate actions wasn't really about the
difficulty with corporate actions, but rather the leaky abstraction of trading
present in the slides. To give you credit, I don't know what issues Quantopian
addresses. On the other hand, has any one tried running Zipline-backtested
strategies in real life? Does any one know what issues aren't addressed by
Quantopian? Corp actions was one part, T-cost model was hinted at in the
slides, but was any though given to borrow costs and availability? There leaks
everywhere. It is not that you don't seem like smart guys who made this cool
thing freely generously available to everyone, but that you seem like you
spent too little time downtown NYC.

PS. Fully adjusted bars are nice, but they have an epoch to be adjusted to.
Unfortunately having an epoch that is not today() means you can't add today's
data to it. If you can't add today's data to it, you can use this system to
generate real trades to trade. Now you need two sets of data and two sets of
code to work with. Good luck.

~~~
fawce
Thanks for taking a look, and for the feedback. Zipline does model transaction
costs, both commissions and price impact of your own trading (slippage). The
commissions and slippage models are pluggable, so you can use what is there or
roll your own.

Quantopian does not have data for stock borrowing costs or availability, and
Zipline's slippage/cost model does not account for them either. We'll find a
way to get that data and plug the hole. The challenge has been finding a clean
way to get it from the brokers, or finding an aggregator with a reasonable
price (any advice?). In the meantime, we've been open about this limitation,
and the zipline code is opensource, so I think/hope anyone who cares to know
does probably know.

Quantopian is building our live trading environment now, so we don't yet have
comparisons between the backtest results and real trading.

Regarding your point about the epoch, I'm not sure I entirely follow you. Part
of the point of zipline's design is to allow easy swapping of datasources,
mainly to allow the transition from backtesting to paper trading and then to
real trading to be seamless. One algo code can run either historically or
live. Adjustments from splits and mergers are back-projected, so that current
day prices need no adjustment. Dividends are dealt with as announce, ex, and
pay events, meaning we do not smooth out the over-night drops, instead we
increment/decrement cash.

I'm in NYC regularly to host the NYC Algorithmic Trading meetup - it would be
awesome to talk to you about these issues in person, please consider coming:
<http://www.meetup.com/NYC-Algorithmic-Trading/>

------
jschulenklopper
An interesting course on using software algorithms for stock analysis and
trading can be found at Coursera:
<https://www.coursera.org/course/compinvesting1>

That course also uses Python as programming language in the examples. Short
description from that URL: "Find out how modern electronic markets work, why
stock prices change in the ways they do, and how computation can help our
understanding of them. Build algorithms and visualizations to inform investing
practice."

------
md224
Fascinating from a technical perspective, but how does it add value to the
world beyond the search for increasingly complex ways to make money?

~~~
fawce
Investing is one way people plan for the future. Helping people plan for the
future is a good thing. Algorithmic trading has mostly focused on (ultra)
short timescales, without much regard for future planning. I think the world
would benefit tremendously if more of that effort went toward investing on
longer timescales. Investing is still almost fully manual today, and packed
with inefficiencies and costs that could be automated away.

------
ngoel36
Very cool! My question is, how do you use something like Zipline to actually
_execute_ trades.

~~~
vshastry
For those with an account, E*TRADE offers a REST based API @
<https://developer.etrade.com> . Onboarding is manual but once you get your
key you can execute trades for your account via the API. Equities / ETFs only
right now, they were working on support for other asset types but not sure
when they are going to launch it.

Full disclosure - I used to run their developer platform.

~~~
fawce
How do commissions work? per trade? Does the api provide any data, or just
execution?

~~~
vshastry
Commissions are the same as your E*TRADE account; list price is $9.99 for
equities although if you are doing higher volumes you can negotiate discounts
($7.99 if you do 150 trades / quarter, and call them up if you are doing a lot
of trades for lower pricing).

Data: [https://developer.etrade.com/ctnt/dev-
portal/getDetail?conte...](https://developer.etrade.com/ctnt/dev-
portal/getDetail?contentUri=V0_Documentation-MarketAPI-GetQuotes)

There's also a call for data on option chains.

------
arxanas
I can't scroll down the code on slide 8.

~~~
twiecki
Yeah, the code is cut-off unfortunately. It's the new IPython nbconverter that
still has some kinks. The idea of the algo is expressed on the next slide.

You can, however, view the full IPython NB here:
[http://nbviewer.ipython.org/urls/raw.github.com/twiecki/zipl...](http://nbviewer.ipython.org/urls/raw.github.com/twiecki/zipline_in_the_cloud_talk/gh-
pages/Zipline%2520in%2520the%2520Cloud%2520--%2520PyData%252013.ipynb)

------
radikalus
Beautiful presentation; really loving all the stuff built on top nbviewer.

As an aside, I think an enterprise-quality zipline solution could be a worth a
pretty fat chunk of $$ especially considering that the budget for KX and its
support is generally > 1M per year per firm. (Yes, there's complex logistics
and legal issues)

I believe seeing on github some requests for h5 support -- this + support
could potentially be a big inroad...

------
pyre
Finally got it to load. Don't know what the problem was. I kept getting a
blank page (wouldn't even let me right-click). Cracked open the dev tools, and
the DOM was there, and all of the resources were loading. No clue what the
problem was.

[ Running: Chromium Version 26.0.1386.0 Ubuntu 12.10 (177362) ]

~~~
pathdependent
The third or forth slide causes my IPad's instance of Safari to crash. (IOS
6.1.3)

------
gourneau
This was a great talk, find it on pyvideo.org in a few days

