
How I made $500k with machine learning and high frequency trading - jspaulding
http://jspauld.com/post/35126549635/how-i-made-500k-with-machine-learning-and-hft
======
tarr11
This is just glorified gambling. I am not sure what special insight or
advantage he had, other than his own model. Every trader has a model.

It could have easily been called "how i lost 500k with machine learning". Like
gambling, it's easy to manipulate statistics to show that you did well in some
period of time.

I worked for a large investment bank about 10 years ago, writing trading
programs for quant traders who were market makers. The quants called guys like
him "retail" investors and they gleefully picked off all those trades. It's
how they made all their money.

So, everyone else, beware of making this a case study in how to make lots of
money really fast. You are more likely to lose money.

~~~
beagle3
It is only gambling in the sense that any business is gambling: Your customers
might stop coming tomorrow because the fad wore off, or a competitor provides
a better/cheaper/hipper alternative.

(And indeed, living is gambling. It's all just a matter of the risk/reward
portfolie).

But jspauld has apparently made $2/trade after fees on 250,000 trading, with a
very small standard deviation (I would guess less than $2/trade) - which makes
it one of the best businesses one could ever have.

You can't live without gambling - by e.g. going to be a salaried employee for
Yahoo rather than Google or that weird newfangled "TheFacebook" thingy back in
2004, was a gamble.

jspauld, statistically speaking, has made less of a gamble there than almost
anyone else posting on HN.

> So, everyone else, beware of making this a case study in how to make lots of
> money really fast. You are more likely to lose money.

True. But that's true for every single success story posted on HN, reddit, or
USAToday.

~~~
gweinberg
No. Every business has a risk element, but what makes this gambling is that
there is no good or service being produced. It's a game of trying to outguess
the other players, with one trader's gain being another trader's loss
(relative to market returns).

Because there's a commission on trades, and because you pay taxes on net gains
but your minimum tax is zero, high frequency trading by its very nature must a
loss for most players.

~~~
heyitsnick
As a professional poker player, analyst and journalist, and being fairly well-
read on classifications of gambling vs skill game in different jurisdictions,
I have not before come across a definition of gambling that was rooted in the
idea that "no good or service is being produced."

~~~
001sky
Any zero sum or negative expected return conctract would meet this definition
(quantitatively). if it floats your boat (makes you hallucinate etc), of
course that might be considered a good or service. so you're right, if I was a
lawyer those words would be loose language.

------
danso
I don't have much experience with finance or working experience with machine
learning, but I've always wondered how much room there was for a clever
amateur to profit in this space, even as it's crowded with much more
sophisticated professionals with much more sophisticated algorithms and
machines.

I'm thinking back to Garry Kasparov's piece in the NY Book Review a couple
years back: [http://www.nybooks.com/articles/archives/2010/feb/11/the-
che...](http://www.nybooks.com/articles/archives/2010/feb/11/the-chess-master-
and-the-computer/?pagination=false)

He talks about a chess tournament in which it was "anything
goes"...competitors could be human, computers, or humans with computers. The
expected outcome was that a grandmaster using a Deep Blue-like computer would
win, but the winners ended up being a couple of amateurs with three computers:

> _The surprise came at the conclusion of the event. The winner was revealed
> to be not a grandmaster with a state-of-the-art PC but a pair of amateur
> American chess players using three computers at the same time. Their skill
> at manipulating and “coaching” their computers to look very deeply into
> positions effectively counteracted the superior chess understanding of their
> grandmaster opponents and the greater computational power of other
> participants. Weak human + machine + better process was superior to a strong
> computer alone and, more remarkably, superior to a strong human + machine +
> inferior process._

So in HFT, how much room is there for an amateur to profit over professionals
by having a sophisticated process?

~~~
bloaf
Good point, I also wonder about the potential to exploit the algorithms used
by the "professionals." In other words, if you can come up with a reasonable
approximation of what the pros will do, can you use that information to beat
them?

~~~
aggronn
'Theoretically', no. Its hard to be optimistic about these two ideas because
while the chess example is a good story, its not analogous for many reasons,
ranging from disparity in available information to players to a difference of
several magnitudes in saturation. Not to mention HFT just isn't chess.

HF traders are just as much hackers as anyone on HN (and there are plenty of
HF traders on HN). So 'theoretically', they've already done what is being
suggested here. If someone comes along and develops a winning strategy, it
really shouldn't be considered as having anything to do with 'professional
strategy vs novice strategies'. It would just be about one person either
getting really lucky or coming up with something that is genius in its own
right.

\--

If there are 'professionals' and then other 'professionals' whose strategy
depends on information about how other 'professionals' trade (and there is),
you end up with strategies at all valid points in the sample space of possible
strategies and counter strategies. Theoretically, there should be no other
possible strategies. Inevitably someone will come up with one though, and the
'sample space' will grow. But its extremely unlikely that additional unique
strategies are successful just because they 'counter' the strategies in the
sample space. But then again, this is real life and these things aren't
impossible.

------
quant123
There is an air of either incredibility or sheer jealousy in these comments.
Nevertheless, I just wanted to tell the OP that he did a great job. Thanks for
sharing. I work in the finance industry as a quantitative software developer,
and it certainly is not an easy job for one person to do. In fact, I tried
(independent of my professional work) doing this myself, and I ended up losing
a lot of money. If people are trying to do this, please please be careful. Big
companies, like ones I have worked at, have technical and human resources that
are vastly more powerful.

~~~
jspaulding
thanks quant123 ;)

------
OldSchool
Great work, very interesting to me. Counter to what we're constantly told
through the media this stuff can be done. Doing it year after year seems to be
the elusive part. Intuitively, once you've proven your technique on 1000+
trades it's not luck.

I developed a fully-automated low-frequency stat arb system that I ran in 2007
based on a perhaps even simpler algorithm. It traded various equities equally
to the long and short side regardless of market conditions so widespread rally
or collapse was irrelevant. I logged about 20-30 trades/day - much slower.

The results, using no leverage, were +90% in a year with a worst drawdown of
2% and a Sharpe ratio of 2. Total trades were 5000+. Month-to-month the
results were very consistent until the uptick rule was nixed in July 2007.
August 2007 was a record winner for me, but Sept-Dec 2007 fell flat, not
losing, but with greatly diminished profits and the same variation and more
frequently getting slammed all-long or all-short instead of a mix that was
often near-neutral. Also getting fills better than my orders then completely
disappeared, as this was the beginning of the HFT middlemen - including your
own brokerage. I shut it down at the start of 2008, keeping the profits intact
and moving on to other priorities.

I continued to monitor the theoretical results for a couple of years but the
conditions didn't return so I eventually cancelled my data feed.

------
dmmalam
Being pedantic, 4000 trades a day isn't HFT. This is stil algo trading, of
which HFT is a subset.

I consider HFT to be any strategy where speed itself is the what gives the
edge. Colocation is usually a prerequisite, though not sufficient. It's a
shame HFT gets all the attention, when it's really a tiny portion of trading
activity. Algo-trading in general is 70%+ of market activity in the US.

Also limiting trades isn't really adequate risk management. The tech exists to
very accurately model your exposures. This is something I see underdeveloped a
lot, and what separates the top trading firms from the rest.

Still I commend you creating a model, working out how to test and execute it
automatically and actually trading your own money.

I really think more hackers should be actively managing their money, (in
general, not like in the article). We have these amazing liquid markets, all
time low spreads/commissions, products like ETFs/derivatives to accurately and
cheaply execute a given strategy, and a huge increase in tech to model risk,
but personal personal investing is the same as the 60s.

~~~
beagle3
In the US, HFT is mostly synonymous with "all out tech war, flooding the order
queue so your less-equipped peers get lags". (Nanex publishes analysis on
these events, which are not occuring several times a month and keep
accelerating).

In Europe, HFT is mostly what OP describes, because they have reasonable
control (e.g., you have to have one execution per 10 orders or pay a fine; in
US exchanges, you can sometime finds 10,000 orders submitted in 3 seconds,
hundreds of thousands per hour, with 10-20 executions).

> Also limiting trades isn't really adequate risk management. The tech exists
> to very accurately model your exposures.

That's basically what AIG did with copulas. Unfortunately, the assumptions in
these models tend to break during crisis, when correlations go to one. And AIG
went bankrupt.

Limiting trades, done correctly (mathematically AND legally) is the ONLY way
to do risk management properly. With more assumptions, you can have "more
efficient" risk management in terms of leverage (e.g., you can net S&P and
RUSSELL exposure by assuming their correlation structure) - but as AIG has
shown, that does not mean you are doing a better job of managing your risk.

~~~
dmmalam
When I say limiting trades, I mean naively saying 'I will have at most x
positions outstanding'. Each trade has an associated risk (variance), that
interacts in complicated ways in a portfolio, which I'm sure you know.

silly example, 100 small positions could be less risky than 1 large position
or, 1 long, and 1 short trade will cancel each other out and create a riskless
portfolio (with 0 return).

You need to have a risk budget, account for each trade, and work out the risk
for the composite portfolio. Obviously this is not fool proof, but it's a way
better approximation of the real world.

~~~
beagle3
> interacts in complicated ways in a portfolio, which I'm sure you know.

Yes. And assumptions about this are bound to break at the most inopportune
moment, see e.g. AIG, which I already referred to. Read about the "copula
model" disaster, as your statement indicates you are unaware of its details.
[https://en.wikipedia.org/wiki/David_X._Li#CDOs_and_Gaussian_...](https://en.wikipedia.org/wiki/David_X._Li#CDOs_and_Gaussian_copula)

> 1 long, and 1 short trade will cancel each other out and create a riskless
> portfolio (with 0 return)

This is true if and only if the long and short are in the same exchange, AND
exchange rules allow netting long vs. short deterministically. Otherwise, you
have counterparty risk. E.g., you can be long SPY and short SP contract (in
equal underlying), which would theoretically mean your only exposure is
interest rate changes (and sometimes not even that!)

However, since this is in different exchanges, it might happen that during a
flash crash, your SPY position will be liquidated for insufficient margin at a
low price, but then the price bounces back, and you've lost money on a
perfectly hedged position.

OP's model (limiting exposure and assuming the worst, if I understand
correctly) is not statistically efficient use of margin, but it's way better
at actually managing risk than any statistical model.

------
lrm242
Take a look at the VIX from 2009 until today and you'll understand why you
stopped making money.

~~~
S_A_P
[http://finance.yahoo.com/echarts?s=%5EVIX+Interactive#symbol...](http://finance.yahoo.com/echarts?s=%5EVIX+Interactive#symbol=%5Evix;range=5y;compare=;indicator=volume;charttype=area;crosshair=on;ohlcvalues=0;logscale=off;source=undefined);

very interesting, a spike in sept-dec 2009 that roughly follows his PNL...

~~~
jcampbell1
Without a ton of volatility, any homebrew HFT is going to lose to commissions
and spread. Basically, he was trading in one of the few periods where is was
possible to make money.

------
hafabnew
I've been considering trying HFT myself for a while. I'm competent with
Machine Learning and am a Software Developer by day, so I can program and can
sysadmin well enough to get something up and running without any trouble at
all.

But, every time I've tried to actually get started, I've always found the
amount of research required before being able to begin is just staggering.

It seems like the logical course of single-programmer HFT trading being:

\- Find sample data

\- Build your trading program using sample data

\- When you're happy: connect to live API and set your trading program loose

\- Iterate.

However, the first step and the third step seem like the ones which require
the most research.

Is there somewhere which has a straightforward dump of timestamped market data
available to download (free or not), in order to actually develop a working
program?

Likewise, figuring out what to actually trade with, and which service to use
is also pretty taxing.

~~~
sseveran
Nothing in HFT is free. People doing this for a living use precision time
protocol in a colocated data center to build their own timestamps. However you
can get started by buying ITCH, OpenBook, etc... data which has the full
market depth feeds for the various exchanges. There are a couple of brokers
out there specializing in the space.

If the easy part was building a working model either you got incredibly lucky
or the model is wrong.

------
HockeyPlayer
I run a 12 person HFT group in Denver. This is an excellent description of how
an individual can bootstrap themselves into success. Great story, and nicely
presented.

The one interesting point that he glossed over is what his indicators were. He
wrote, "The indicators that were most useful were all relatively simple and
were based on recent events in the market I was trading as well as the markets
of correlated securities."

Which doesn't really tell you much.

~~~
jspaulding
Thank you. I'm looking back through my code and there are really a lot of
indicators. It would get pretty technical to explain them. They are all
explainable it's just that each one corresponds to slightly different market
conditions and I just didn't want to get into it.

~~~
HockeyPlayer
Fair enough. Most people considering trying this probably have a few ideas for
indicators.

But for anyone coming to HFT from a coding background instead of a trading
background, an explanation of one of your indicators would have been
fascinating.

~~~
jhales
Yes, I would find this very interesting. Doesn't matter if the indicator is
now defunct.

------
unreal37
There is a coursera course called "Computational Investing, Part I" that I am
taking that aims to build a market trading simulator to test a trading model.
It just started so it's not too late to join.

<https://www.coursera.org/course/compinvesting1>

------
confluence
A lot of people are stating that this is like gambling - it is - but not in
the sense that you think. Firstly he doesn't use his entire bankroll on each
trade, secondly he goes long-short consistently over very short periods of
time, thirdly he's too tiny to actually move markets, and fourthly he is in
and out within a day - where his max var. is 100% on thousands of tiny trades.
Think of it like this - he has $100 - he bets $10 of that each day. He can
blow that $10 - no problems.

Worst case he runs out of capital over a period of weeks.

He can't blow up in the way that you think - but he can have large drawdowns
over a period of weeks.

Markets are eventually consistent scalable systems - and that is why we prefer
them over central planning. In the medium term they prices things correctly,
cheaply and efficiently (decade+).

In the short term however (sub-decade) - they can't price jack.

Markets are inefficient period - if they weren't, well then P=NP and you could
just put your NP-hard problems into a market and get back cheap, quick,
accurate results. Oh - wait - protein folding is actually harder than that.

There are 2 major ways to make money in the markets. Value-Growth and
statistical arb (often high frequency). The former (Buffett) is highly
concentrated bets on the future of business (I'm value - long
TSLA/GOOG/Samsung). The latter is looking for thousands of small diversified
statistically significant correlations above 50% (random guesses) and trading
costs between securities/price movements over short time intervals (aka
statistical ghosts in the data - RenTech/Shaw).

Both work. Both work well. And will continue to do so as long as markets
exist.

------
Judson
This reminds me of an AMA from a few years ago. Really interesting if you are
into stuff like this:

[http://www.reddit.com/r/IAmA/comments/9s9d7/iama_100_automat...](http://www.reddit.com/r/IAmA/comments/9s9d7/iama_100_automated_independent_retail_trader_i/)

~~~
jspaulding
thanks for posting hadn't seen that

------
runjake
Relax with the vitriol. The guy is sharing an interesting personal story, not
providing a step-by-step HOWTO or recommending people follow his suit. In
fact, the article is really an ad for his startup Courseware.

~~~
jspaulding
CourseTalk. CourseTalk! <http://coursetalk.org> .. You are right kind of :)
But I've made a decision to start reaching out generally so I can attract cool
people to work with on whatever projects I may be interested in in the future.

~~~
runjake
By the way, no offense meant by the advertising thing. I enjoyed the article.
Although, as a technical person, would've enjoyed more details on the code and
algorithms.

------
tomp
This article is missing a crucial piece of data: what was the initial
investment.. earning 500k with 10k initial investment is genious, with 10M
initial investment it's just another year on the stock market.

~~~
beagle3
This is a different style of trading than what investors do. He said that he
was never more than a few contracts in. A single contract needs $10K day
trading margin usually (depends on time frame and specific contract, but it's
a reasonable estimate). Therefore, if he was never more than 10 contracts long
or short at the same time, the risk was of $100K.

Making $500K on $100K over 6 months is good business.

~~~
jspaulding
You are correct. I think I had to have $10-30k in my account. I was extremely
low risk so they weren't concerned.

------
rhplus
The charts show he was trading between Jun 2009 and Oct 2010. How much of his
gains could be attributed to the market recovery in general? The Dow went from
about 7000 to 11000, the Russell from about 600 to 800.

~~~
washedup
Well, in the article he said tat he did not care about direction, he would
simple buy when his expected price was up, and sell when down. However, there
could have easily been a bias in his model that "preferred" and performed
better during upward movements. If so, he got lucky.

~~~
jspaulding
Longs and shorts were 50/50 and my program showed no preference for up or down
days. High volatility and high volume was what it liked.

~~~
debacle
I guess the real question is: what was your alpha in that timeframe?

~~~
jspaulding
I'm probably showing my ignorance here but what do you mean by alpha? And how
is it quantified?

~~~
solutionyogi
Alpha is how much excess return you had over the market (or risk free return(

E.g. if you made 10% when overall market was up 15% for the year, you have
negative alpha. [As someone could have bought index and held it through year
to generate better return]

If you made 20% when market was up 10%, you have positive alpha.

That is why everyone in the investment community is 'seeking alpha'.

------
nashequilibrium
Pls do not follow the advice of the OP. I started a hedgefund in 2004 doing HF
platform arbitrage and ran it for 5yrs and i can honestly tell you that this
is just survivorship bias. This is a very complex field and being off
slightly, having a slight bias, a fraction of a point off your execution
pricing and a slightly flawed money management system is recipe for disaster.

The biggest issue is the confusion that you can apply machine learning to HF
trading. HF trading sub 15min mark is more about playing the deal flow, and
only the institutions have an edge on this. This is why goldman had to
separate the buy and sell sides in the early 2000's. Above 15mins you are able
to find an edge using time series analyses since the market is scaling
invariant according to Benoit Mandelbrot and this does not apply to dealflow.
Also having access to dealflow allows you to predict volatilty seconds ahead
which allows you decrease your risk and increase you reward as well as handle
your costs since the volatility will impact your transaction costs even if
transaction costs themselves stay the same. There is just so much stuff to
cover that a comment will not do justice in explaining what is wrong with this
guys logic.

~~~
jspaulding
There are plenty of shops making tons of money with HFT who do not have deal
flow at all - it's got nothing to do with luck.

Survivorship bias would mean I simply got lucky. If you're going to say that
you're at least going to need to look at my P&L charts and say how I could
possibly achieve that much success with luck alone.

Finally, machine learning has everything to do with my success. There were
hundreds of variables in my algorithm that were ALL optimized using ML. If you
read the article you would know that I built an accurate model for backtesting
that I used to optimize variables as well as confirm that I was going to make
money before I even started live trading.

I'm pretty confident that whatever you were doing in 2004 has nothing to do
with what I was doing.

~~~
quant123
agreed. jspaulding got it right although I can see nashequilibrium's
skepticism as the US futures market is incredibly crowded (read: competitive,
no free cookies).

------
jspaulding
If anyone has questions for me happy to answer as best I can.

~~~
jcromartie
I have one question:

Why doesn't every hacker do this to make extra money? Is it within the grasp
of anybody who can program to automate trading like this?

EDIT: Sounds like it's not really for everybody. You have to own or rent a
server with access to direct lines to the exchanges, or else your lag will be
such that profiting from HFT is impossible. How much do these cost?

~~~
beagle3
> You have to own or rent a server with access to direct lines to the
> exchanges, or else your lag will be such that profiting from HFT is
> impossible.

A profitable predictor is a much, much harder problem.

At a place like Goldman Sachs, a quant with a working predictor gets paid 5
times as much as the IT guy who makes that predictor talk to the market
quickly enough.

Because, as your question implies, it is (relatively) easy to do the IT work
or hire someone to do it. Not so for the predictors.

~~~
otakucode
At a place like Goldman Sachs, you don't need quants or predictors. You call
up the CEO of a company you want to post record profits, and you tell them if
they don't do absolutely desperate, self-destructive things (screwing
employees and customers for immediate gains), you will crash their stock and
destroy their entire company. That CEO then has a choice. Be the one who
killed the company, or be the one who kept it running for a few more weeks and
delivered a record quarter that made Goldman Sachs happy.

Once you have the assets and capacity to actively manipulate the price of any
stock at will, the market is a VERY different animal and no longer need to be
understood at all. You simply force its hand.

------
padobson
Don't do this with your own money. Found a startup building HFT tools, and
then raise money for it, and use other people's money to test your tools.

If the tools work, sell them. If they don't, tweak them, try it again, and
sell them until they do.

This is risk management.

------
rbc
I started the infrastructure for this kind of thing a while ago. It's BSD
licensed.

It is a software implementation of the Viable System Model (VSM), a model for
autonomous systems developed by Stafford Beer. It provides structure,
communications, auditing and alerting for autonomous systems.

Part of it is base code for dealing with stocks and options, treating
securities positions as autonomous systems that have the scaffolding for
running simulations on themselves. It's in Smalltalk and runs under Squeak and
Pharo. It can be found at:

<http://home.rbcarleton.com/rbc/software/smalltalk/VSA/>

I won't try to advertise it as a complete solution, but it might be the start
of one.

------
nesu
Wait. This is not HFT. There's a huge difference between automated and high
frequency trading. What he does is only automated scalping at best (or at the
fastest).

Automated trading is more on strategy, while HFT has more to do with volume
and speed. With automated trading, you predict price movements. HFT involves
being a liquidity provider. You don't use market technical indicators in HFT,
you wait for some really huge orders.

HFT firms won't bother him. Those are dealing with an entirely different set
of algorithms. He should have contacted brokers instead.

~~~
crucialfelix
quite true. but HFT has become a buzzword like "the cloud" and even many
financial industry specialists claim to do HFT but in fact they are just
market making. auto-scalping or even hedge funds that day trade with robots
are not HFT.

this:
[http://books.google.de/books?id=dobO95EBcqsC&redir_esc=y](http://books.google.de/books?id=dobO95EBcqsC&redir_esc=y)

even though its over 10 years old, is real HFT. moving correlations, windowed
FFT (of bid and transaction events), microscopic operators, negative first-
order autocorrelation of returns.

it is NOT about supplying liquidity or being a market maker. that's just
market makers trying to say they are in on the latest trend.

------
stcredzero
Is there a way to do this with Python or Ruby? I could just as well program
this in C#, but I have a friend who can code a little, but doesn't really need
everything in C# to do what he wants. The value add of offering the simulator,
including the taking into account the bid/ask prices and a stochastic model
for latency. Combine this with a web based code editor and easy hosting, and I
think this would be a viable product.

~~~
owyn
Nope, there is absolutely no way to do this with python or ruby. :)

Seriously though, there are some existing frameworks and products that you
could check out. I haven't used this myself, it's just in my bookmarks:

<http://www.rapidquant.com/features>

A C++ library with python bindings: <http://quantlib.org/index.shtml>

And a low level library for data analysis: <http://pandas.pydata.org>

~~~
grumps
If you use their library/SDK/Framework.... how do you actually trade, do you
still need a brokerage? Sorry a bit new to this field.

~~~
owyn
Yep, almost any of them have an API these days. Your software would make the
list of trades which is uploaded and executed...

Oh, I forgot about another backtesting framework for python. It has a built in
IDE as well.

<http://www.quantopian.com>

~~~
fawce
I work at quantopian. We are working on broker integration, so you will be
able to trade algos you develop live.

Our backtesting engine, Zipline, is opensource -
<https://github.com/quantopian/zipline>

Zipline was unveiled at PyData NYC, and the presentation materials are here:
<https://app.quantopian.com/posts/hello-from-pydata>

------
stickydink
Looking at your first chart there, is there a reason (other than market
conditions) you were making significantly more at the end of '09 than mid '10?

~~~
jspaulding
To be honest I don't know exactly what happened. My theory is that over time
more and more market participants started integrating the types of analysis I
was doing which rendered my program ineffectual. It's a pretty normal pattern
that there is some inefficiency in the market and over time it disappears.

~~~
stickydink
Being a machine learning program, how much of it did you tell it to forget?

Were you compounding the data always, or telling it to forget what was going
on several months ago? Or somewhere in between?

(I'm pretty unfamiliar with machine learning, apologies if this is obvious or
something)

~~~
achy
This. If the 2009 model worked well, did you try letting your algorithm to
'forget' the 2010 data and see if the model worked better?

------
nrmn
Could you comment on how your "curve fitting" algorithm worked? Did you end up
with an equation for each curve? Im working on something that requires curve
fitting and any kind of tip would be helpful.

~~~
jspaulding
I basically just brute forced it. I came up with a cost function which would
measure the difference between a possible curve and each data point. I think
you're supposed to do the squared difference but I can't remember if I did
that. With a cost function in place it's just a matter of zooming in on
variables that minimize the cost function.

------
arbuge
I am curious as to exactly why the profitability decreased steadily and rather
rapidly all the way to ~zero. The article doesn't seem to expound on that
unless I missed something.

Is this a result of bots on the other side adapting in some way to what you
were doing? I would have thought you would be too small a player for them to
notice.

~~~
jspaulding
I don't think anyone was adapting to what I was doing in particular but rather
simply adapting to the opportunities in the market.

~~~
arbuge
Thanks. Makes sense... I guess those opportunities tend to get ironed out
rather fast then.

------
mcarvin
Very important to understand that making $500k speculatively is not evidence
of an 'edge', nor is trading frequency evidence of the absence of luck. From
March 2009 through much of 2010, the market was strongly bullish - if his
algorithm showed a positive market bias then his returns would primarily be a
function of timing (read luck: and there are a million variants on the nature
of the bias that could be unwittingly responsible for his returns, despite the
frequency of trades).

We cannot even tell if $500k is a good risk adjusted return - we have no
information on volatility, nature of the exposure or most importantly how much
money he started with?

Not exactly shocked Jim Simons didn't return his email. But completely
shocking that he walked away from a successful automated trading strategy...
the only thing rarer than a free lunch is a man willing to walk away from one.
suspect.

~~~
fr0sty
1\. Author graphed his daily returns which should give you a handle on his
volatility. He also stated his max drawdown was $2k.

2\. Author has stated elsewhere than he began with $10k in seed capital.

3\. The "bullish" market you cite increased only about %70 in that timeframe
while the author's returns were multiples of that number.

4\. Author walked away from a _previously successful_ strategy that no longer
produced profits. His montlhy returns went to nearly zero so he stopped.

Edited to fix numbers.

~~~
mcarvin
1\. I cannot get even a remote sense for the nature of his risk exposure from
looking at his daily returns. 2\. ok. 3\. The point here is that a systematic
bias in his algorithm will expose his trading strategy to the good graces of
market fortune (luck) regardless of whether he trades a million, billion or
once a day. The source of the bias is irrelevant. 4\. did not see where he
said that but that very much confirms 'timing' / which in this case I
interpret as luck as being at least a contributing factor.

------
ChuckMcM
_The high point of my trading was October 2009 when I made almost 100k. After
this I continued to spend the next four months trying to improve my program
despite decreased profit each month. Unfortunately by this point I guess I’d
implemented all my best ideas because nothing I tried seemed to help much._

It is of course possible that once you made "real" money with your algorithm
it was spotted by the other algorithms which then started working against it.
(Aka exploiting it) Having talked with people in that space (hft) I was left
with the impression that an insane amount of analysis was done on those
trades.

------
pdog
Would you be able to open source any of the code behind your trading system?
Maybe not the "secret sauce", but it would be interesting to see how you
processed the data feeds, modeled the data, entered orders, etc.

~~~
jspaulding
I certainly could open source it. I may just wait a bit on the off chance that
somebody wants to purchase it.

~~~
Juuumanji
that would require some proof that it works today.

~~~
beagle3
Not really.

A lot of people in the business would pay e.g. $5,000 for exclusive rights for
something that worked this well in 2009 (with proof that it worked in 2009,
e.g. verifiable broker statements), and a smaller amount (say, $5,00) for non
exclusive rights.

If he claimed it still works but he wants to sell it, it is a completely
different game -- because when these things work, they are cash cows.

~~~
jspaulding
Yes i'm pretty sure it wouldn't work today. By buying the code I realistically
mean hiring me to work for them based on what I achieved.

~~~
beagle3
The guys you want to work for (2sigma, RenTec, Jane Street, Susquehanna, ...)
are unlikely to call you up as a result of this blog post / hacker news
exposure/discussion.

If you want to go back to trading, you'll probably have to actively try to get
a job -- at the very least, let someone who's still in the business know that
you are looking. In my experience in this field, word of mouth and friends-of-
friends are infinitely more successful hiring strategies, for both sides.

(Re:releasing the source - I would like to have a look at the strategy, but I
would recommend against releasing anything that is even close to being useful,
unless you want to spend the next year screening "where can I get a good XTAPI
broker" and "I've got XTrader_PRO set up, but I'm getting error 10013, what
gives?" emails).

the Nuclear Phynance message board is probably a better place to look for
business offers.

------
Sharma
Trust me, you earned that much because of your luck. Otherwise Andrew Ng would
have partnered with another finance professor and they would have been the
richest people on earth!! Imagine trading with their expert systems on global
markets.

I traded stocks and Forex for years and my experience says, it is not for
everyone. What ever indicators,discipline or model you follow it is going to
work only if you have the right intuition or luck!

~~~
zdwalter
I like to trade Forex using mql4, any suggestion? Thanks.

------
wtvanhest
You said "Growth stopped" but your P&L shows negative growth approaching zero.
Would it be more fair to say that your profitability turned to zero?

~~~
jspaulding
To be honest I'd given up and moved on to other stuff well before I actually
shut down the program.

------
vincegata
Thanks for the post, it's very inspirational.

Could you explain this part, specifically what do you mean by "bucket"?

"To accomplish this I tracked predicted price moves in 50 buckets that
depended on the range that the indicator value fell in. This produced unique
predictions for each bucket that I was then able to graph in Excel. As you can
see the expected price change increases as the indicator value increases."

------
vincegata
Thanks for the post, it's very inspirational.

Could you explain this part, specifically what do you mean by "bucket".

"To accomplish this I tracked predicted price moves in 50 buckets that
depended on the range that the indicator value fell in. This produced unique
predictions for each bucket that I was then able to graph in Excel. As you can
see the expected price change increases as the indicator value increases."

------
dataisfun
This is an activity that adds no value to our world.

------
jcfrei
It is pretty clear from his own graph that this stopped working in october
'10. which was an eternity ago in terms of financial markets. algorithmic
trading has increased manifold since then, so finding another arbitrage
opportunity like he did is only going to be more difficult.

------
mrchess
Cool article but I hope people don't start trying to follow this path. Ask
yourself -- why did he stop?

~~~
jspaulding
yes. and yes.

------
sejje
I don't understand trading enough to even understand many of the terms in the
article, however I'm curious to one thing: is it possible that a program could
be written specifically to exploit yours? And/or is that a potential reason it
became unprofitable?

------
vinayan3
Have you ever thought of making a trading system that would buy tons of stock
when a flash crash happens? It is going to happen again. If your system is
ready and you buy before they shut the market down or roll back orders you
could make a hefty profit.

~~~
crntaylor
The difficulty is in identifying what is a 'flash crash' (i.e. a temporary
downward blip in prices caused by computer or human error) and what is a
genuine downward price movement.

If the market dives and you quickly get into a big long position, and then it
dives some more - what do you do? You can either close out your losing trade
and take the loss, or hope that the market comes back up, all the while
holding on to the risk of further losses.

Also, there's no guarantee that trades in the middle of a flash crash will
remain valid after the crash. The exchange could nullify all trades in a
certain period of time, which would completely wipe out your upside potential.

~~~
beagle3
> The exchange could nullify all trades in a certain period of time, which
> would completely wipe out your upside potential.

This is the most important thing: In every single "flash crash", the exchanges
have retroactively canceled trades, in a rather arbitrary manner (e.g., "every
trade between 16:30 and 16:38 is null and void"). There is some underlying
justification, but it is also arbitrary (e.g., "anything below 3% of the price
when the flash crash started", with no specific justification for the 3%
number, or a well defined methodology for the time of the crash).

That could easily turn a +$100K profit into a -$500K profit, depending on
circumstance.

~~~
fr0sty
Nitpick: When exchanges have busted trades there is a "at or below $XX.XX"
condition as well as the "between XX:XX and YY:YY" condition.

In the Flash Crash as well as the Knight Capital incident "up/down 30% from
the Previous Close" was the price collar (anything outside that was busted and
anything inside stood).

Of course there is no guarantee that the same criteria will be used the next
time around so caveat emptor.

------
MonteChristo
The prototypical example of why a tax on financial transactions is urgently
needed.

------
datashaman
Excellent article on this posted here: [http://www.lrb.co.uk/v33/n10/donald-
mackenzie/how-to-make-mo...](http://www.lrb.co.uk/v33/n10/donald-
mackenzie/how-to-make-money-in-microseconds)

------
doki_pen
Isn't profit meaningless without knowing initial investment? I read the first
few paragraphs and got bored. Why not say upfront what the bankroll was to
start?

edit: I found what I was looking for in the comments.

~~~
JoblessWonder
Just to save someone else from having to look for it like we both did, his
initial investment was

    
    
      $10k loan from dad. Built it up to 30k trading manually before my automated program went live.

------
suyash
@Author ? Can you create an online course and teach us all? Thanks

------
FireBeyond
Hopefully not buried too deep, but any books recommended for getting into day-
trading, either manual, or algorithmic? Kindle preferred, but definitely not
the deciding factor.

------
namank
Might've been what it was a couple of years ago but this post, dated today, is
the perfect advertisement for the author's current business.

Well done!

PS: no sarcasm intended, it truly is an excellent advert.

------
photorized
I am skeptical for two reasons:

1\. when you have a good system (even one you cannot "improve" further), you
don't talk about it. 2\. You don't just stop using it.

There should be more to this story.

~~~
frankster
Can I recommend that you read the article and you will find therein the
answers you seek!

------
junto
Out of interest, how much capital did you start off with?

~~~
jspaulding
$10k loan from dad. Built it up to 30k trading manually before my automated
program went live.

~~~
intel4004
What were your average transaction costs per trade? 1000 to 4000 trades per
day, lets say 2500 on average, translates to about 625K transactions per year.
I assume you did not have to pay something like ETrades 3$ commission per
future contract, which would result in almost 2M of fees per year.

------
dwk9080
If you're interested in working on this kind of stuff in the San Francisco
area, send a resume to Headlands Technologies. careers@headlandstech.com

------
amalag
What contracts did you trade? Looks like you did futures contracts? Did you
look into forex at all, or was this strictly equites.

~~~
jspaulding
I was trading stock market index futures: Russell 2000, DAX, and a bit of
Nasdaq. I experimented with other stuff as well.

------
derryl
This might be a dumb question but... what system does your algorithm interface
with?

Last time I checked, NASDAQ and NYSE don't exactly publish API's

------
davidw
Looks like it's written in C# - is that correct?

~~~
jetti
It stated in the article it was C#.

"In 2008 I was “manually” day trading futures using software called T4. I’d
been wanting some customized order entry hotkeys, so after discovering T4 had
an API, I took on the challenge of learning C# (the programming language
required to use the API) and went ahead and built myself some hotkeys."

~~~
davidw
Yes, that's what I was going off of. I was curious about that aspect of it, if
it had stayed C# or what.

~~~
jspaulding
Actually I believe when I switched from T4 to TT it was C++ I was using.

------
hnruss
Just because you CAN do it doesn't mean you SHOULD. Even if you don't think of
it as "gambling", you're still taking in tons of money without providing any
tangible benefit to society.

If you want to make money from investing, why not do so in a socially
responsible way? Invest in companies that are changing the world for the
better. You might not bring home as much money, but at least you'll be able to
sleep well at night.

~~~
wyan
You mean, something like Instagram?

~~~
sjm
Not really defending the parent, but believe it or not Instagram provides
something that makes a lot of people happy. Winning a bunch of money on the
stock market does nothing for anyone but the winner. Not really a fair
comparison.

------
JuDue
"Since I'm no longer running my [half million dollar] program I’m happy to
tell all"

This does not ring true.

------
ianstallings
I could do the same with poker. Or I could lose it all. That variance thing is
a bitch.

------
ww520
Very cool article. Gave a detail explanation of the ENTIRE process. Thanks!

------
iandanforth
I'm glad to see a healthy respect for investment among the hacker community.
It's traders like this who commit to nearly a full 10 seconds of ownership
that are the backbone of economic growth for this country.

~~~
unreal37
I think there is a healthy respect for computer code that can let loose upon
the world and make money.

~~~
mattyppants
Your missing the point that owning part of a company for 10 seconds brings
zero social value to the economy. Which I also tend to agree with. Slightly
off-topic from the point of the article, which was that he achieved it without
backing of a fairly large institution.

~~~
unreal37
I didn't miss that point - I just chose not to address it. Lots of things add
zero (or negative) social value to the economy including Farmville or the
gazillionth Instagram clone. That doesn't mean they're not worth doing.

And HFT or day-trading does bring some value to the world. It enables
companies to go to the public markets and raise capital. And investors to sell
their shares without having to wait too long for a buyer. But those positives
come along with negatives as well.

------
curationary
how is it "machine learning"? A few if-else doesn't make it "machine
learning", or does it? - @curationary

~~~
jspaulding
It takes data (last 4 weeks of market update data) and learns how to predict
price movements and how to make profitable trades.

------
curiousdannii
I'm sceptical that HFT is good for the public. What did you do to ensure your
system wouldn't make a flash crash worse?

~~~
beagle3
Why is it his business to defend other market participants?

~~~
curiousdannii
I didn't say it was. But everyone should be in the business of not being evil
and not exasperating the problems of others. In trading that means not
contributing to flash crashes.

I am not saying that this guy's trading did contribute to flash crashes! He
may have successfully implemented systems to prevent that. I hope he did, and
if so I'm interested to hear how.

------
thisismyname
How much did you start with?

------
leminhhai
Amazing, love your project

------
hellsten
A rising tide lifts all boats. "Price move predictions" sounds as effective as
tea leaf reading.

------
shizzy0
And nothing of value was created.

Just kidding. It was a fascinating article. Thanks for sharing.

------
gubatron
of course no firm would respond to his noob low yield model.

The level of the coders doing HFT is beyond the comprehension of this guy,
added to the team of Mathematicians, Physicists and computer scientists at
your average HFT firm, they probably laugh when they read this.

Good try though, it was awesome that eventually he tuned it to profitability,
but there's no way in hell they'd buy that amateur software/algorithm.

Kudos though for taking on the task of learning how to code and making money
with ML.

~~~
jspaulding
This is such an odd comment. What motivates you to write such a comment? And
what makes my system amateur and noob in your eyes?

------
dschiptsov
Basically this is a story about a guy who was smart enough to script up his
trading tool (he discovered that there is an API and wrote some code to use
it).

He trade other people's money, using other people's (probably employer's)
account and resources, I suppose.

His employer have paid all the fees, and, took all the risks - if there is
profit - it is mine, if there is a lose - it is theirs.)

The essence of trading is about having a special (insider) position of even
being a market maker, who just collecting fees from every trade other people
do.)

But this is just my guess.

~~~
mattyppants
This is not even close to an accurate summary. He never stated that he had any
employer backing, and he wasn't collecting market making fees. In fact he was
paying brokerage fees which is the exact opposite.

~~~
dschiptsov
He used his own money to test his algos? Come on.

Taking a Machine Learning and Statistics course does not make you a trader.
You need something else - access to the system.

This is why my guess is that he was an employee.

~~~
oijaf888
An employee of what? He states above that he was paying roughly $200/month for
a server and $1800/month for the software/data connections to his broker.

~~~
dschiptsov
_Prior to setting up my automated trading program I’d had 2 years experience
as a “manual” day trader._ \- mom's and pap's 401k?

------
loup-vaillant
While this was quite fascinating, I couldn't see this form of trading as
anything but a zero-sum game. Some players win, the other lose, like in any
other game.

Except finance is supposed to be "serious". In most serious, legitimate
activities, extracting money means you provided value somehow. So, what value
high frequency trading could possibly provide?

~~~
gd1
A strange position to take, I'm guessing you've absorbed it from the media
somehow. His machine was in the market enough to trade 4000 times a day (I
would suggest passively, or he would have been eaten by the cost of crossing
the spread), so he was basically continuously offering a service to the market
- an offer to sell and a bid to buy at the price he thought fair. Do you
demand to know what value your local 7-11 provides by selling you milk? The
value is that you can buy milk 24 hours a day, so you don't begrudge them the
15c a carton they are making. And they probably charge more than your local
supermarket for the convenience too. The dirty thieves.

~~~
loup-vaillant
I refine my position here: <http://news.ycombinator.com/item?id=4752742>

I do get the value of trading, investing, and some form of arbitrage. Heck, I
don't have to search for the farm to get my milk. That's a service, and it
does deserve a reward. What the author did is a bit different:

> _Most of the market volume was other bots that would only execute a trade
> with me if they thought they had some statistical edge._

I understood it meant "they would trade with me only if they think _I_ was the
sucker". And of course, he would trade with them only if he thinks _they_ were
the suckers. It's not providing a service. It's fighting in a zero-sum
economy.

Now he did say " _most_ of the market volume". So there's a fraction that's
not bots, and probably also a smaller fraction that does not even play the
zero sum game, but instead does some positive-sum trading with the mostly-
zero-sum players. But this interface boundary seems incredibly _thin_ ,
compared to the internal zero-sum behemoth. That looks like a highly
inefficient use of time, energy, and brains.

And even then, I'm not sure the zero-sum game provides any service to the rest
of the world: zero-sum players base their models on the behaviour of each
other, not on the actual performance of companies. That would add no new
information to the system. At best, that only amplifies the effect of the few
that actually predict company performance. And I doubt it does it well.

~~~
gd1
It doesn't really matter _why_ anyone is in the market. The fact is that they
are posting prices to the limit order book, and in doing so providing
liquidity and accepting risk and providing a service. The only way the OP
could get executed passively is by offering a price equal to or better than
the market price, yes? So someone got a better price (or more liquidity at the
prevailing price) by him being there.

There are really only two actions in a market, providing liquidity (i.e.
market-making) or taking liquidity (including arbitrageurs). Stretching the
shopkeeper analogy, any time you put a price tag on an item you are taking a
risk and exposing yourself. You slap a price on some bananas and then there is
a tropical cyclone (it happened in Australia) and the price of bananas
doubles. Someone smart swoops in (call them an arbitrageur), grabs your
bananas and goes to the till and you legally have to sell them to him even
though they're worth double now. You also have inventory risk, if you're
holding a lot of bananas out back then there is the chance that they go
rotten.

Every limit order (an offer to sell, or bid to buy) has an inherent risk that
you should expect to be compensated for (or you wouldn't do it), and enhances
the market. The more competition amongst people posting prices the better, it
just means more liquidity (you can buy or sell as many bananas as you want)
and tighter prices.

The liquidity taker on the other hand is all about exploiting a misprice. Say,
Intel release earnings and are down 10%, but the price of the highly
correlated ARM hasn't moved yet, if you are the fastest you can take advantage
of some poor person who still wants to buy ARM at the same price as before
this information was known. Keep in mind here, a price can move without any
trades happening (or goods changing hands). The shopkeepers can whip around
and change the price tag on their bananas if they are fast enough before
anyone buys them.

It is harder to justify the liquidity taker - what marginal advantage is there
to having the price of ARM react to that news in 2 milliseconds rather than
20? They force the transmission of information yes, but why is it better to
have it happen a bit marginally quicker? On the other hand, how can you have a
realtime market without them? It keeps the providers on their toes, and there
will always be someone fastest to react.

Incidentally, providing liquidity must have been the chief function of the
OP's algo - 4000 trades a day doesn't work for a liquidity taker that has to
cross the spread. He isn't getting 2000 mis-price signals a day to swing at,
good enough to justify crossing the spread. DAX traded 120k contracts
yesterday, he would have been averaging 2-3% of daily flow.

~~~
loup-vaillant
Okay, thanks. A few minor points.

I agree that "why" doesn't matter, if you know "what". When you don't however,
"why" is a useful predictor.

I understand the value of arbitrage, to some extent. In its current form, it's
quite heavy, but I can imagine we're better off allowing it.

We should compensate for value, not for risk. The two are not always
correlated. I know trading is risky, but that's not the point. If it doesn't
create value, it shouldn't be compensated. And I suspect some forms of trading
create little to no value at all.

By itself, closing a deal doesn't mean you provided value. It means the other
party _thinks_ you provided value _to them_. When both parties think that,
either it's a win-win situation, or someone got tricked. (Delayed bids and
offerings complicate this, but it's the same principle.)

If I got it right, the OP did what we could call "short term speculation". If
you predict something will rise, buy from a sucker who didn't. If you predict
something will fall, sell it to a sucker who didn't. And of course, pray _you_
are not the sucker. Locally, it's totally zero-sum.

Now maybe the whole system facilitates real transactions (with non-
speculators) at the boundaries? But even then, for a given volume your reward
doesn't seem to be proportional to your facilitation power (which I have no
idea how to compute), but to how well you manage to trick your fellow traders.
I very much doubt that such twisted incentives can foster a useful, let alone
efficient, system.

~~~
gd1
You're still missing it. Every single additional limit order in the market
creates _real_ value. If I want to buy 500 shares of company A, and one market
participant is offering 250 at $10. Another is offering 250 at $11. It is
going to cost me 250 x 10 + 250 x 11 = $5250. Now, the "short term speculator"
you are complaining about shows up and puts his tiny little order in, he is
offering 1 share at $10. Now the equation if you want to execute is 251 x 10 +
249 x 11 = $5249. He/she just saved you $1. One _real_ dollar. As someone who
wants to get a transaction done on a market, more people posting bids and
offers cannot _possibly_ hurt you. Only help you. You have the complete
freedom to decide to meet their offered price, to cross the spread, to pick up
that carton of milk at the 7-11 and walk to the till and make the transaction.

Now maybe that little speculator who sold you the share at $10, he goes and
works the bid, he advertises that he wants to buy a share at $9. Someone fills
him. It take 2 minutes of trading to work to the front of the queue (easily
possible with an equity). During that time, he is exposed to the risk that the
stock might spike up in price, a risk you are no longer exposed to since you
got your desired trade done $1 cheaper and 2 minutes ago. Then if he succeeds
he makes his $1 profit. It doesn't always work - maybe 60% of the time the
price ticks up and he just breaks even. He pays exchange fees and clearing on
both the in and out trades too.

In theory, you could get involved in all of this with your 500 share purchase.
Try to work the bid, get a better price, etc... or if you cross you are
effectively paying for a service. Every order posted in the market is a
service.

How wide are bid/offer spreads these days? How can that possibly be a bad
thing?

~~~
loup-vaillant
_(Edit: "limit order" looks like technical jargon. If so, I don't know what it
means. Maybe you didn't say what I thought you said)_

In your example, it looked like you wanted your 500 share for something else
than selling them. So you're talking about "real transactions (with non-
speculators) at the boundaries". So yes, those transactions are a useful
service provided to you by the speculators. I'm not denying that.

On the other hand, your wording seems to imply that most transactions are of
this type (non-speculator with speculator). As far as I understand the system,
they are not. If the OP is to be trusted, the vast majority of transactions
are between 2 speculators trying to outsmart each other. Do we at least agree
on that narrow point?

My second point is that a such a transaction (between 2 speculators) is zero
sum. Locally, because whatever the first speculators won, the other lost. And
globally, because wherever the share is, it could still be sold to a non-
speculator. Making the transaction between speculators doesn't make it any
easier. I'd say it could make it _harder_ , for the non-speculators now have
to buy the share faster than the speculators (or pay extra to have traders do
it for them). Seriously, where is the value in that?

Now maybe a good speculator tends to provide a good service to non-
speculators. However, they are not selected by the quality of the service they
provide. They are selected by their ability to rip each other off. How quality
of service arises from such a cut-throat competition is mysterious to me.
_(Usually, we compete for quality of service directly, so the connection is
obvious. Speculating is not the same thing.)_

~~~
gd1
I think we are getting closer to understanding here. You're kind of quibbling
over the definition of value in your other post - someone can provide a
"valuable" service even if it isn't currently being used (by your 'real
transactors'). I might sleep better at night knowing that if I want to liquefy
my investment in Apple tomorrow, I can do so without having to worry that
nobody will be there to take the other side of the trade, that I'll have to
pay an enormous spread or be left holding a position I want to be out of. An
international corporation can be more confident of planning their cashflows
knowing that the FX market will always be liquid, and will be quoting at most
2 ticks wide in the majors 24 hours a day every day. This is the oil of an
economy, the freedom to allocate capital when you want, where you want and pay
the smallest possible friction costs to do so.

Many things in our economy can be described as a zero-sum game. Society only
needs so much of any given good, and at the retailer level you aren't involved
in increasing demand. Two corner stores might get into a price/advertising
war, but let's argue that there is still only a fixed amount of product they
can sell to a small community. So advertiser's get fat on fees (in our case,
exchanges do). So what is the point you ask? Evolution, competition, it can
only be good for the consumer. You still haven't disputed that by the way, the
consumer wins don't they? How could they possibly have lost? It takes a tin
foil hat to think that it isn't better and cheaper to invest now.

" they are not selected by the quality of the service, just ability to rip
each other off" - people have a bit of a naive view that there are some sort
of magical harlem globetrotter moves you can pull on an order book. The
mechanics are dead simple I'm afraid. It isn't chess, not even checkers. You
can't even see the other participants, everything is anonymous on the
exchanges I've dealt with. Spoofing is illegal, and that is about as clever as
it gets. Very little you can do to make your algo better will put it at odds
with providing a better quality service.

To prove that, remember the two types of algo. If I write a liquidity provider
that _doesn't_ offer a tighter spread than the competition, I will miss out on
trade flow and make little or nothing. If I write a liquidity taker that has
an opinion that is _wrong_ about the correct value of a security, someone else
who is _right_ will smash me. So effectively we are selecting for more
liquidity, tighter spreads, and more accurate prices reflecting available
information.

~~~
loup-vaillant
My definition of value is simple: something is valuable when it manages to
raise people's utility functions (right now, that's about as rigorous as I can
be). A trade that happens to be closed by a plain old investor is valuable. If
it's another speculator, however, the value is zero. Only consequences
ultimately matter.

But, if I got you right, the anonymity of it all mean we cannot separate the
two… hmm… Then we've got to multiply the potential utility of the trade by the
(quite low) probability of it being closed by a non-speculator. Still
valuable, but much less. And I'm back wondering to what extent this is worth
the (collective) effort. You did change my mind a little, though. I'll need to
learn more.

> _You still haven't disputed that by the way, the consumer wins don't they?
> How could they possibly have lost?_

The consumer winning does count as creating value. However there are 2 parts
in the retail store competition example: the part where they compete on
quality, price, diversity… and the part where they pay fat fees to the
advertiser. The first part benefits the consumer, the second just add
inefficiency in the loop: if both stores could only agree to not use ads,
everybody would win.

When Pareto Optimum and Nash Equilibrium are at odds, life sucks.

------
mempko
Wow, fuck that guy. no really. Should be title "How I stole 500k with machine
learning and high frequency trading".

~~~
oijaf888
Who did he steal it from?

