
Analyzing Cryptocurrency Markets Using Python - quotable_cow
https://blog.patricktriest.com/analyzing-cryptocurrencies-python/?utm_source=hackernews
======
Galanwe
My 2 cents:

\- it is not really pertinent to compute the correlation between prices. This
takes the currencies trend into account since prices, contrary to e.g.
"returns", are non stationary. This will lead to a biased higher correlation.
Just do a ".pct_change()" before the correlation.

\- also averaging the price between exchanges is a bit naive. It hides
arbitrage opportunities and does not reflect the underlying traded volume. A
VWAP would at least provide a better approximation.

~~~
quotable_cow
Thanks for the tips! I'll update the tutorial code with that info in the next
day or so. Until then, I've just added a note warning about the bias, and
recommending the use of the ".pct_change()" method.

~~~
hodder
(As a quant) when calculating correlations of returns, we use log changes.

~~~
Kranar
As a fellow quant I wanted to clarify that we use changes of log prices. Your
sentence suggests calculating the change in price and then taking the log
rather than taking the log of the prices and then calculating the change.

~~~
hodder
Yes. Thanks. My comment was poorly worded.

------
plaidfuji
This would be more appropriately titled "downloading, cleaning, and plotting
cryptocurrency price data using pandas and plotly in a Jupyter notebook."

TL DR the analysis is just a couple of time series correlation coefficient
heatmaps.

That being said this is a great tutorial for people just starting out with
data handling and analysis in pandas.

------
ghgr
It's great how readily available financial data is with cryptocurrencies! As a
complement to this post, I've been recently working with Jupyter notebooks to
analyze high-frequency trading activity in Bitcoin markets [1]. I've been
listening to the GDAX socket since end July, so I have almost a month's worth
of tick data (~ 7GB+ gzip compressed JSON data, it surely explodes to ~100 GB
after extracting). If somebody is interested to carry out further analysis I
can give you a link to download it.

[1]
[https://nbviewer.jupyter.org/github/ghgr/HFT_Bitcoin/blob/ma...](https://nbviewer.jupyter.org/github/ghgr/HFT_Bitcoin/blob/master/hftbitcoin.ipynb)

~~~
jetti
What are you using to listen to GDAX sockets? I've been able to successfully
connect using their node library but when trying to connect directly without
using their library I'm unable to connect.

~~~
ghgr
I use the Python library WebSocketClient. Here is the code:
[https://github.com/ghgr/HFT_Bitcoin/blob/master/download_boo...](https://github.com/ghgr/HFT_Bitcoin/blob/master/download_book.py)

~~~
jetti
Thanks for the code. I've been trying with the Elixir library websockex [1]
but it has been giving me nothing but troubles. I just gave up and poll every
X minutes now.

[1] [https://github.com/Azolo/websockex](https://github.com/Azolo/websockex)

------
clarkmoody
(Shameless self-promotion)

You can watch a couple dozen the Bitcoin markets trade in real-time all on one
chart with my site:
[https://bitcoin.clarkmoody.com/tickers/](https://bitcoin.clarkmoody.com/tickers/)

It gets very interesting when the price really starts to move, since all the
markets tend to move in lockstep. The response time reveals how active the
trading bots are, making sure to reduce arbitrage opporunities.

------
bdfhjk
I'm surprised they didn't mentioned Machine Learning to further analyze the
cryptocurrencies. Especially, that a few years ago an experiment shown, that
if we would trade algorithmically on S&P500 via Machine Learning, traders
could earn 8.5% returns comparing to the 5.6% by a random tactic. Here is a
nice explanation: [https://sigmoidal.io/machine-learning-for-
trading/](https://sigmoidal.io/machine-learning-for-trading/)

~~~
jetti
I've started dabbling with R (started with Python and just got fed up with it
and switched to R) and ML to analyze bitcoin from GDAX over the past few days.
Coming from a non-stats background (I did time-series analysis in undergrad
ala Econometrics but that was over a decade ago) I've had issues trying to
create meaningful models. With packages like Caret[1] it is easy to slap code
together and throw factors at the algorithms to see what sticks. The problem
is that the models that are generated typically suck (I think the best RMSE I
was able to get was around 12 and that was with linear regression model)

ML can definitely help trading by using things like Ensemble Learning[2] but I
would argue that for the novice it isn't going to add much to a trading
strategy than doing other analysis on their own would do.

[1]
[https://cran.r-project.org/web/packages/caret/index.html](https://cran.r-project.org/web/packages/caret/index.html)

[2]
[https://en.wikipedia.org/wiki/Ensemble_learning](https://en.wikipedia.org/wiki/Ensemble_learning)

------
grenkatost
I wanted to do something like that, eventually I did it on the elasticsearch.
Then I wanted to share with the society and added grafana.

The final result is real-time analytics of trading on main exchanges and for
major pairs: [https://cointradeanalysis.com](https://cointradeanalysis.com)

~~~
mathgenius
Wow, that is very cool.

Some feedback/ideas:

1) things like xrp_usd need more decimal places for price (Max price 0.16, Min
price 0.15, you really need at least 4 decimal places.)

2) What you call "Amount" should really be called "Volume".. "Count orders"
should be "Count Trades" or perhaps just "Trades".

3) The buy/sell graph: i would place the buy/sell bars beside each other
(instead of having sell underneath pointing down). This would make it much
easier to see which is higher, buy or sell.

~~~
grenkatost
1) I'll try to fix it.

2) Corrected, thank you very much.

3) The first option was like you describe, but something I did not like about
it - I'll think about it :)

------
indescions_2017
Don't think I've ever really seen the full $BTCUSD chart since inception like
that. More than slight resemblance to Nasdaq-100 chart circa late 1999-early
2000. Just saying, exercise caution out there ;)

Next step: prediction! As an active research subject, crypto-currencies may be
the ideal candidate for using deep learning to forecast non-stationary time
series data.

Theory and Algorithms for Forecasting Non-Stationary Time Series

[http://www.cs.nyu.edu/~mohri/talks/NIPSTutorial2016.pdf](http://www.cs.nyu.edu/~mohri/talks/NIPSTutorial2016.pdf)

------
bkolobara
I'm a big fan of Stellar. It looks like the only cryptocurrency with an actual
use case that works today and minimises costs for companies[1][2] building on
top of it. Everything else looks like mostly hype driven. I would say that
this is one of the reasons it trades so different from everything else, except
Ripple.

As many developers hang out here I want to point out that the Stellar
Foundation has a rolling competition for developers:
[https://www.stellar.org/lumens/build/](https://www.stellar.org/lumens/build/).
You can see here all the projects submitted to the last round:
[https://galactictalk.org/t/sbc2017april](https://galactictalk.org/t/sbc2017april)

[1]: [https://tempo.eu.com/en](https://tempo.eu.com/en) [2]:
[http://chippercash.com/](http://chippercash.com/)

~~~
StavrosK
Is that the thing Stripe (or someone) launched years ago? The one where they
were giving 50 coins free per subscriber?

~~~
andreasklinger
Semi-related question:

I got some of those ICO coins back then - i registered with username/password
back then

Still have username/password but i am unsure where to actually login? Anyone
familiar w/ Stellar knows how to get to the account?

~~~
sensecall
I just went through the same process – had some coins from the ICO a couple of
years ago.

If you head to
[https://launch.stellar.org/#/login](https://launch.stellar.org/#/login) you
can log in. I got a message prompting me to contact Support, which I've done
and now awaiting a reply.

~~~
StavrosK
Same here, it asked me for a confirmation code over email, but I received
nothing, so I'm waiting for Support.

------
Finnucane
Is it possible to get data on economic activity in cryptocoins? Which is to
say, not just the coin trading, but goods and services being paid for with
coin tokens? Every now and then you see a post to the effect of 'so-and-so is
taking payment in Bitcoin.' Does anyone actually do that? Or are the coins
just hoarded for their investment value?

~~~
sbenitoj
At the moment, there aren't a ton of real-world use cases aside, but the
future will be very different!

Check out Open Bazaar for an entire market place that's facilitated via
cryptocurrencies.

------
ativzzz
What's a good resource to learn the technical statistical background for an
analysis like this? It's beyond the basics of calculating P values but it also
builds on the basics by analyzing a large space of correlation coefficients.

Would this be considered data science? It doesn't dive into machine learning,
or anything advanced computer science wise.

------
mrchicity
Layering, Spoofing and Momentum Ignition are not "HFT Strategies." They're
illegal market manipulation techniques, most often used by manual traders. You
don't need to be ultra fast to bully prices around or enter large non bonafide
orders. I've never heard of a legitimate proprietary trading firm (i.e. one
that pays a salary and hires highly qualified people, not a boiler room
operation) intentionally using these techniques. Most are in the arbitrage,
market making or statarb business.

It would be much more interesting to see the author dive into which events
lead to fast price changes. How quickly are price shocks on other markets are
reflected on GDAX? How are changes in outright contracts like BTC/USD and
ETH/USD reflected in ETH/BTC? Do USD denominated pairs drive price discovery
more or less than EUR denominated ones, and does this change when US based
traders are asleep? Lots of stuff to look at here.

~~~
ghgr
Directly from Wikipedia [1]:

> In July 2013, it was reported that Panther Energy Trading LLC was ordered to
> pay $4.5 million to U.S. and U.K. regulators on charges that the firm's
> high-frequency trading activities manipulated commodity markets. Panther's
> computer algorithms placed and quickly canceled bids and offers in futures
> contracts including oil, metals, interest rates and foreign currencies, the
> U.S. Commodity Futures Trading Commission said.[109] In October 2014,
> Panther's sole owner Michael Coscia was charged with six counts of
> commodities fraud and six counts of "spoofing". The indictment stated that
> Coscia devised a high-frequency trading strategy to create a false
> impression of the available liquidity in the market, "and to fraudulently
> induce other market participants to react to the deceptive market
> information he created".[110]

I guess it's HFT if it's executed fast enough.

[1] [https://en.wikipedia.org/wiki/High-
frequency_trading#Strateg...](https://en.wikipedia.org/wiki/High-
frequency_trading#Strategies)

~~~
mrchicity
Panther was a one-man shop that did nothing but market manipulation. Yes, they
used automation but for every Panther, there are 1000 manual traders being
fined by CME for spoofing every year. Automation doesn't imply state of the
art HFT. The Flash Crash spoofer was using a Trading Technologies system that
any small trader can rent with a few scripts loaded in.

Look at real players with hundreds of employees. None of them make their money
from market manipulation. It's all from low-latency arbitrage. If you're fast,
you can make way, way more money doing legitimate trading than market
manipulation.

------
fiatjaf
Nice and everything, but you can't make money with this, or can you?

~~~
jetti
From the anlysis that was written up you would most likely not make consistent
money on. That said, it could be used as a basis to start exploring more to
try and find other factors that could influence price and try to trade on
that.

------
johnt113
What's the point of correlation ? If I put 2 buys either I will have double
profit or double loss. If I put 1 buy 1 sell I get zero.

~~~
ucha
If two assets are correlated and one of them moves up but not the other, you
expect the return of both over a longer time period to converge, you can make
a statistical arbitrage by selling the one that moved up and buying the other
c.f.
[https://en.wikipedia.org/wiki/Pairs_trade](https://en.wikipedia.org/wiki/Pairs_trade)

------
krath94
Anyone having trouble with the link?

