
MIT computer scientists can predict the price of Bitcoin - dpmehta02
http://newsoffice.mit.edu/2014/mit-computer-scientists-can-predict-price-bitcoin#.VEadQI474jk
======
minimax
The problem with the paper is not overfit. They claim to have run their
simulation with out of band ("live") data. The _actual_ problem with the paper
is that we have no idea if their simulator is any good, which means that their
result (89% return in 50 days) could be totally bogus. In other words, we
don't know if the actual bitcoin exchange would fill their orders at the same
prices (if at all) as their simulator does. A decent simulator for a high-
frequecy strategy like this is not trivial because you have to incorporate all
the exchange behavior (documented or not) into your simulator, and then you
have to validate your results by comparing your simulated results to the
results of some actual trading. The fact that they spent none of the paper on
the details of the simulator makes me extremely skeptical.

~~~
lrm242
People so often overlook the role that execution plays in trading. As time
scales shrink the impact of execution increases, and indeed many HFT
strategies are not profitable without top tier execution (both in terms of fee
schedules and technology). This is also one of the places that most academics
fail when analysing a trading strategy. They make typical "assume a
frictionless surface" types of assumptions that break down in a real market.

To your point, high-frequency simulation is damn hard and it is highly likely
that they failed here and have tainted their results with bad simulations.
Most of these papers would be well served to avoid dollars and cents and
simply analyse the statistical qualities of their signal to their target over
time. That is the first step in this business, anyway. Simulations only happen
once the signal has been through the wringer.

~~~
hft_throwaway
I see no mention of the spread, costs or even ability to put on a short
position, exchange lag variability (huge issue when simulating even on modern
exchanges, let alone fly-by-night bitcoin markets). Additionally, it is very
easy to find trend-following signals that "work" but break down when
conditions change very quickly. That seems to make up most of the prediction,
along with an order book imbalance signal that might be more stable.

I think a better approach would be looking at order book features and lags vs.
other markets. The costs are high on BTC markets so it would be pretty tough
to overcome those though. Anyone with experience to do this is probably doing
it somewhere more lucrative. I think BTC markets only trade a few million USD
a day.

~~~
reduce
About 35 million USD in trades on the public bitcoin-paired markets today.
About 40 million USD today in total volume for all pairs on these exchanges.

[http://www.cryptocoincharts.info/coins/info](http://www.cryptocoincharts.info/coins/info)

~~~
hft_throwaway
More than when I last looked, but to put that in perspective, you could trade
$40mm in a few price ticks in major currency pairs and CME avg daily turnover
alone is > $100bn: [http://cmegroup.mediaroom.com/2013-06-07-CME-Group-Sees-
Reco...](http://cmegroup.mediaroom.com/2013-06-07-CME-Group-Sees-Record-
Trading-in-its-Global-FX-Complex-Transacted-239-billion-in-Notional-Volume-
Across-FX-Futures-and-
Options?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+mediaroom%2FCMsF+%28CME+Group+Media+Room%29)

Public spot FX markets and OTC trading do huge volumes too.

Some BTC exchanges charge like 60 bps per trade. Finding a signal to overcome
that cost and do enough trading to make it worthwhile would be quite
difficult.

------
Animats
This is short-term trading. "Every two seconds they predicted the average
price movement (on OKcoin) over the following 10 seconds. If the price
movement was higher than a certain threshold, they bought a Bitcoin; if it was
lower than the opposite threshold, they sold one; and if it was in-between,
they did nothing." I don't see them allowing for commissions and fees.

OKcoin, at peak, had a trading volume so high that it's generally considered
to be fake - the exchange operators manipulating the price. What this group at
MIT may have done is reverse-engineered the fake trade generation algorithm.

~~~
chollida1
> What this group at MIT may have done is reverse-engineered the fake trade
> generation algorithm.

Just to be clear, there is nothing wrong with this. Infact, sitting around and
reverse engineering what other traders are doing is what many funds do. I'm in
this group so I"m happy to answer questions if anyone has any.

> Every two seconds they predicted the average price movement (on OKcoin) over
> the following 10 seconds. If the price movement was higher than a certain
> threshold, they bought a Bitcoin; if it was lower than the opposite
> threshold, they sold one; and if it was in-between, they did nothing.

To be clear, this is the core of what most HFT systems do these days. Consume
many different factors, give each factor a decay factor to tell the system
when the signal goes stale and distill them all into a value that says, but or
sell or stay.

It's worked well for Renaissance Technologies:)

[http://en.wikipedia.org/wiki/Renaissance_Technologies](http://en.wikipedia.org/wiki/Renaissance_Technologies)

~~~
gaze
Where can I read about the actual strategies used? Perhaps some that aren't
used any more. I've read an absolute ton about how to set up a trading system
with hadoop and cassandra and blah blah but that's all pretty trivial. I'd be
more interested in the strategies... this decay factor is interesting. Where
did you find out about this?

~~~
hft_throwaway
A simple signal would be more shares bid than offered at the inside market.
Imagine a market with 1000000 shares bid at $4 and 100 shares offered at
$4.01, it's more likely to tick up than down in the very near-term. This isn't
tradable though since the predictions aren't large enough to overcome the
spread and you can't expect to join the 1000000 share bid and trade when it's
good (the 1000000 shares that made you think it was going to move up all have
to trade or cancel for you to trade, not looking so great anymore).

Real HFTs use other features of the order book, movements in related products
(maybe a move in oil futures in the last minute impacts the price of an
airline stock in the next 10 seconds, etc.), and so on. The signals are not
really that complicated, but competitors eventually converge on knowing the
same signals and compete them out of existence/profitability. Also, how you
execute around your prediction matters just as much if not more.

[http://queue.acm.org/detail.cfm?id=2534976](http://queue.acm.org/detail.cfm?id=2534976)
and
[http://queue.acm.org/detail.cfm?id=2536492](http://queue.acm.org/detail.cfm?id=2536492)
have some more examples.
[http://www.decal.org/file/2945](http://www.decal.org/file/2945) also has some
ideas for old signals that probably won't work anymore.

------
barisser
This seems like massive historical overfit, which can lead to arbitrarily
precise fit, but no predictive capability.

Any model, if given enough parameters, can be made to match historical data to
an arbitrary degree.

I also run several Bitcoin bots. I can tell you that slippage is not
insignificant. If you make transactions every ~10 seconds and incur 0.1% fees
each time, this is an extremely significant effect in aggregate. Also bid-ask
spreads, while usually small, often aren't in periods of high volume.

~~~
leeber
Agreed. According to the paper: They trained the data, once, with 3-4 months
of data from Feb to May. Then they tested the data, once, with ~6 weeks of
data from May 6 - June 24.

There was no cross validation involved in assessing the performance of this
model. Using one subset for training, and one subset for testing and calling
that conclusive is naive.

No interesting conclusions can be taken away from this paper due to flawed
methodology and failure to take into account real world variables like bid-ask
spread, commissions, how quickly trades can be executed, whether orders will
even be filled, etc.

~~~
beejiu
Cross validation too uses training sets and test sets. This sort of time-
ordered data will not be independent, so the prequential approach seems to be
a more suitable approach to measuring forecast accuracy. (I haven't read the
paper.)

~~~
leeber
Cross validation for time series, goes something like this:

[http://robjhyndman.com/hyndsight/crossvalidation/](http://robjhyndman.com/hyndsight/crossvalidation/)

(near bottom of the article)

Anyways, as somebody who has spent GIANT amounts of time experimenting with
machine learning using market data (mainly stocks), I can give you TONS
algorithms I've created that would show similar, even much higher, gains when
you only test on 6 weeks of data. Been there done that. For example 6 weeks
you make a 100% return, then in the next 3 weeks you suffer a 50% loss.
Reality sets in...

------
Houshalter
Sometime in 2013, before the bitcoin prices exploded, I downloaded some
bitcoin historical price data and ran symbolic regression on it with Eureqa.
It came up with a formula that fit the observed data fairly well, and wasn't
very complicated.

But when I extrapolated it forward a few months, it predicted the price would
explode to unreasonable levels. I was disappointed and threw it away, assuming
that it must be wrong.

~~~
gchokov
Thank you for sharing the Eureqa software. Wasn't aware of it. +1

------
Macuyiko
I'm late to comment, but something which I'd like to point out is that this is
done by the same team behind the Twitter trending topic prediction technique
from a few years back, as mentioned also in the article [1].

When their Twitter technique was released, I spend a few weeks reading through
Nikolov's PhD thesis (the advisor gets most of the the fame in the press
articles but Nikolov's thesis has all the details) and trying to implement it
in R. My observations at the time: extremely simple algorithm which would be
shot down by most peer reviewers for being not very novel (the affiliation
helps a lot here). That said, I believe greatly in pragmatism, and the
approach was actually working well. What I did find out however is that their
was a great deal of data selection and pre-processing involved making the
approach hard to implement in a real-life, real-time setup. I get similar
feelings from this work.

[1]: [http://newsoffice.mit.edu/2012/predicting-twitter-
trending-t...](http://newsoffice.mit.edu/2012/predicting-twitter-trending-
topics-1101)

~~~
atomroflbomber
Could you specify what kind of data selection and pre-processing was required
and why exactly it was hard to implement it in a real-life setup?

------
jacobwcarlson
The paper states that the strategy was simulated with live data and makes no
mention of slippage. I've never traded bitcoin so I'm not sure how difficult
it is to get fills, but that along with spreads are non-trivial components of
real trading.

~~~
dnautics
it would have been better if they set up an wallet and put bitcoin in it and
demonstrated the trades that were automatically executed by this wallet. Since
bitcoin is a ledger history, we could be 100% certain that SOME algorithm 'did
the right thing', although we couldn't be certain that they spun up more than
one algorithm and then just showed us the best one.

~~~
ne0n
...That's not how Bitcoin trading works. In order to trade, your bitcoins need
to be deposited into an exchange. Trades are not recorded on bitcoin's public
ledger because there isn't a transaction for every trade. Bitcoin cannot
currently handle that many transactions.

------
HockeyPlayer
They didn't include any discussion of:

1) execution (are they expecting to buy on the bid and sell the offer?).

2) commissions. They only made 3,362 yuan on 2,872 trades. A yuan is about 12
cents, so they are making 15 cents USD per trade.

A .1% commission would cost them roughly 5 yuan per trade, but they are only
making 1.17 yuan/trade.

~~~
princeb
OKcoin (the exchange they relied on in the paper for data) claims 0%
commissions and a bid-ask spread of just 0.04 RMB. how real is that, I have no
idea. but if you relied on this information you could believe that you can
still take a few additional spreads worth of slippage (in addition to
crossing) to compensate for execution latency and still come out very far
ahead

------
runeks
Success in predicting markets is measured in profit.

This research team should start a company that offers a service that allows
users to deposit bitcoins, which the company then invests according to their
alleged predictions, and then pay interest on deposits, and keep a part of the
profit for themselves.

Doubling the initial investment one time is one thing, but this hypothetical
company being able to double its investment every 50 days for years is
something else. I doubt they can. A doubling every 50 days is x160 every year.

I think claims of being able to predict market prices should be met with great
skepticism. Especially prices of easily traded commodities, including
bitcoins.

The only proper measure of an ability to predict market prices is profit,
because profit also measures the extent of the predictions: how much can you
move the market (by trading according to your predictions) until you can no
longer predict what will happen? Obviously, there's a limit. No one can
extract unlimited profit from any market. So there definitely _is_ a limit to
how much you can earn from your algorithm. If you can earn 10% p.a. on an
investment of maximum $5000, your algorithm isn't really worth much. If you
can earn 1000% p.a. on an investment of up to $100M, your algorithm is
_great_. But without knowing these figures we really only have a claim,
seemingly a claim of them being able to make _a lot_ of money, but choosing
not to do so.

------
sillysaurus3
I fixed the article's headline image:
[http://i.imgur.com/QVgcgNI.png](http://i.imgur.com/QVgcgNI.png)

If everyone began using the paper's strategy, would the strategy still work?

Also, the strategy seems less effective than portrayed in the news article. If
you look at the "results" section, it seems like the profit flatlined shortly
after starting, then had success due to some major trading event, then
eventually flatlined again:
[http://i.imgur.com/CBjEjgo.png](http://i.imgur.com/CBjEjgo.png)

Wouldn't it be more accurate to say "this strategy is effective under some
very specific circumstances"?

Also, does anyone know how equation 4 was derived?
[http://i.imgur.com/vkx8ZEC.png](http://i.imgur.com/vkx8ZEC.png)

It seems like the key insight of the paper, but there's no mention of where it
originated from. Is it a common equation in statistical modeling? I'd like to
learn more about it. Does anyone have any suggested reading or coursework I
should study?

~~~
dash2
I think equation 4 is just derived from equation 3. The top bit counts every
time before that y_i has taken a particular value y, and multiplies it by the
distance of the x value then, x_i, from the value of x now. (Squared and
exponentiated because this is the pdf of the normal distribution.)

Here's an interpretation: if there were no noise, you might just count the
number of times x took on a particular value and y took on a particular value,
and divide that by the total number of times x took on that value. This would
give you an empirical estimate of the prob of y given x.

Because there's noise, they weight the counts by the pdf of the normal
distribution of x - x_i. So, whenever x_i was close to current x, and y_i was
a given y, that increases the probability of y occuring now.

------
andrea_s
How many of the people who are bashing the paper in this discussion are
machine learning experts? Especially the overfitting crowd - looks like
emotional attachment to one's favorite topics is not really impacted by said
person's overall education and expertise.

~~~
Houshalter
Here is the discussion on /r/MachineLearning:
[https://www.reddit.com/r/MachineLearning/comments/2jwpv1/mit...](https://www.reddit.com/r/MachineLearning/comments/2jwpv1/mit_computer_scientists_can_predict_the_price_of/)

------
api
... for the next 15 minutes.

When you predict the future of a market, you change the future of that market.
People start investing on the basis of your predictions and whatever
opportunity for profit you found is closed. This is why HFT people iterate
constantly and also why they put their servers as physically close to the
market as possible.

~~~
mattfrommars
> This is why HFT people iterate constantly and also why they put their
> servers as physically close to the market as possible.

Do you happen to have read this :-

[http://www.nytimes.com/2014/04/06/magazine/flash-boys-
michae...](http://www.nytimes.com/2014/04/06/magazine/flash-boys-michael-
lewis.html?_r=4)

------
joshdance
No they can't. If they could they wouldn't tell anyone, and they would make
millions (billions?) of dollars.

------
rdmcfee
This kind of innovation is cool, but it's a zero sum game. The bitcoin markets
are already driven by competing bots. Their profits will be reduced as other
bots iterate on their algorithms.

~~~
joosters
Pleas don't spout out 'zero sum' like you think it means something insightful
here. Any marketplace is 'zero sum' if you think about it. There's a buyer and
seller. So what?

~~~
panarky
Most marketplaces are not zero-sum.

For example, futures and options are zero-sum because every dollar of profit
that one trader makes is offset by a dollar of loss from another trader.

But stock markets are not zero-sum. Prices can be bid up without a single
share changing hands, creating new wealth out of thin air, and everyone wins.
Conversely, prices can go to zero, destroying wealth and making everyone a
loser.

Same thing for most other markets you can think of, from real estate to used
cars to your local flea market. As with bitcoin exchanges, none of these are
zero-sum since there doesn't have to be a loser for every winner.

~~~
lrm242
You're mixing the concepts of realized and un-realized profit. A market price
change marks your open position and gives you an unrealized (loss)gain. But to
realize that you have to trade, plain and simple. And someone must take the
other side of that.

A more compelling argument against zero-sum is that traders and investors have
different time frames and objectives. And, indeed, the classic argument in the
futures market is that a farmer who buys a hedge from a speculator represents
a potential positive outcome for both.

~~~
panarky
I start a company to make cheap autonomous flying cars. I invest $1,000 of my
own money and build a working prototype. It looks promising, so you buy 50% of
my shares for $10,000.

I just made a 20x return on my investment, and you own 50% of a great
opportunity. Who's the loser here that makes this a zero-sum market?

Our autonomous flying cars go into production, and now your 50% is worth $100
billion, and you sell your shares to Elon Musk. Where's the zero sum?

~~~
lrm242
First, I never claimed the markets were or were not zero-sum, only that your
attempt to explain was wrong as it mixed realized and unrealized profit.

Second, zero sum requires a definition of utility for each participant. If
those definitions vary (and they almost assuredly do) then a reasonable
argument can be made that to define whether the activity is a zero-sum game or
not requires a commonality amongst those utilities (dollars, for example).
This is often not the case in trading (for example, the farmer hedging his
crop with a speculator who seems to profit short term).

------
Symmetry
They should have made more money rather than publishing more quickly. It used
to be possible to do these sorts of things to the stock market but when these
sorts of regularities are discovered the process of exploiting them also
eliminates them once enough money is being made. Heck, a major trading firm
got started by noticing that stocks went down on the weekend (and of course
they don't any more).

------
Belmont1
89% over 2 months. Not bad. My algorithm did 8,000% over 6 months using real
money on real exchanges and I have the trade history to back it up.

~~~
damian2000
What exchanges do you recommend for bot trading? ... I've dabbled a bit
trading manually on bitfinex but that's all.

------
ISL
Can a HFT-knowledgeable commenter chime in on the viability of the Sharpe
ratio here?

From a physics perspective, it appears that the Sharpe ratio of 4.1 is roughly
equivalent to a 4.1-sigma claim that their algorithm is better than random
trading. I can't check easily, but I'd guess that the movement of Bitcoin
prices isn't normally-distributed (looking at the paper's time series suggests
that there's more low-frequency power there). If so, I'd guess that a more
robust measure of the claim's significance would show it to be less
significant.

Put differently, I'd guess more than 1 in 15,000 random sets of 2872 trades
(their number of trades) would yield comparable profit. Furthermore, a simple
buy-and-hold would've yielded a 20+% return over the same period.

~~~
lrm242
A sharpe of 4 is ok. Most "HFT" type strategies have sharpes so high they
don't talk about sharpe anymore as it is meaningless. It is simply a different
type of trading. See Virtu's pnl distribution in their S1. As a point of
reference, Blair Hull is on record saying they are interested in nothing below
a sharpe of 10 (r-finance talk, should be googleable).

~~~
petercoolz
Thanks for sharing.

Link:
[http://www.nasdaq.com/markets/ipos/filing.ashx?filingid=9443...](http://www.nasdaq.com/markets/ipos/filing.ashx?filingid=9443224)

------
liaboc
The fact is anyone can *Predict the price of bitcoin, but will you actually
put your own money to trade?

Like what we did here,
[http://financeai.com/forex/btc](http://financeai.com/forex/btc) it does show
some degree of correlation between sentiment and the price.

------
stokk
Ever thought how our big data overlords (Google, Facebook, MS, Twitter, etc)
just need to check for correlations between their users' data input and stock
exchange movement?

They own the stock "matrix".

~~~
liaboc
You mean something like this?

[http://financeai.com/stock/nyse/twtr](http://financeai.com/stock/nyse/twtr)

------
xs
Those interested in automatic trading of bitcoin using algorithms should check
out [https://cryptotrader.org](https://cryptotrader.org)

------
jaekwon
By publishing the paper you essentially invalidate it, as people take
advantage of it. Happens time and time again in any market.

------
crimsonalucard
doesn't predicting the price change the price? Just like how knowing the
future changes the future.

------
zoba
He says "Give me your money and I’d be happy to invest it for you." Alright,
I'll do it... just tell me how.

------
xxcode
SLIPPAGE SLIPPAGE SLIPPAGE!!!

------
jff
Can they predict how to get real money back in exchange for their Bitcoins?

