
Using a Keras Long Short-Term Memory Model to Predict Stock Prices - austin_kodra
https://heartbeat.fritz.ai/using-a-keras-long-shortterm-memory-lstm-model-to-predict-stock-prices-a08c9f69aa74
======
goodside
This is pure nonsense. This isn’t even the right way to begin thinking about
this as a forecasting task — the target series should be log-normal returns,
not raw asset price. The performance of this model is laughably bad, which is
probably why he spends zero time evaluating its effectiveness. You could
trivially get better forecasts than this by _naively repeating the last-
observed price_.

This isn’t ML. It’s cargo-cult performance of words and ideas that ML people
use.

~~~
jwiley
Can you clarify why "the target series should be log-normal returns" is
important or provide a pointer for more information?

~~~
jpeterson
Investment markets operate on relative gain, not absolute gain. E.g.: if you
invest in a stock and it gains $5, this would be a great return for a $1 stock
but a poor one for a $1000 stock, so the absolute gain doesn't mean anything
on its own. A 5% return always means that you've gained 5% on your investment.

~~~
ak39
Would it be the same (valid) with percentage returns?

~~~
goodside
Less so. Log-normal returns are better because they have the property that a
summation of log-normal returns over contiguous intervals is equal to the log-
normal returns of the combined interval. In other words: Losing 5% and then
gaining 5% doesn’t put you back at exactly 100%, and log-normal fixes that.

~~~
theothermkn
In the extreme, two successive trades, where the first gains 110% and the
second loses 100%, “average” out to a 5% return. However, you don’t want to
make that pair of trades.

------
DevX101
I'm not a purist believer in the efficient market hypothesis. BUT, I doubt
there's much alpha to be gained simply from looking at price data which is
widely and publicly available. Also keep in mind that markets are dynamic
feedback loops so even if this model had an edge, the act of publishing this
article would work against you to neutralize that edge in the future.

There's a good reason the most successful 'quantitative' trading fund,
Rennaisance Technologies is so secretive and subjects its employees to a
lifetime NDA/non-compete.

If there is edge in the market (and I think there are anomalies that can be
traded on), I would assume it to come from correlating proprietary datasets
with price. Or datasets that have a high barrier to access/analyze. One
example that I'd bet still works in some industries would be counting trucks
from a supplier to estimate product demand before the companies announces
earnings ;)

~~~
everdev
These predictions can also be self-reenforcing. If enough money believes the
model is accurate, it will create it's own market conditions. In a distopian
AI stock prediction world the best model will be the one with the widest
publicity and adoption, not the best data points.

~~~
aggronn
The problem I see with that hypothesis is that the asset pricing must
eventually be tied to actual performance--while it may be self-reinforcing to
some extend, if it _is_ fundamentally wrong, there will be a reckoning and an
adjustment after a high value stock goes bankrupt, for example.

~~~
derefr
But the stock market isn’t just a bunch of people placing speculative bets
about company performance; it’s a bunch of people placing speculative bets
about company performance by _buying shares of the companies_. If the market
is irrational, it can actually _prop up_ companies that would otherwise go
bankrupt.

~~~
aggronn
I suppose they company could continuously issue stock in this case.

But in this world, it actually makes sense for those companies to stop doing
their normal business and just go into the business of selling their shares.

This reality sounds absurd, but you could argue that BTC market is there.
Enough of the market thinks that "always buy" is a good investment, regardless
of the real-life value of the asset. Or maybe, less controversially, gold is
that market. Any asset that is always increasing in value and not related to
the real value of that asset is just a store of value.

~~~
candiodari
Both of those things have valid services that I pay for.

BTC provides a number of services. From money changing and international money
transfers to actual investment brokerage. Granted, the number of securities
available in BTC is less than spectacular, but it's not zero. I pay for both
those services. Now you may argue that those are unregulated services and
therefore have trust issues, but one might argue that all markets have trust
issues, and the only difference is the level. BTC, so far, seems to be more
trustworthy than, for instance, the ECB (e.g. the Greek payment limits and the
Cypriot bail in, one of which affected me, and both of them used MY money to
achieve political aims, without my approval).

Gold provides a store of value, with a good story behind it. I pay my bank, I
believe, around $40 per year for that same service. With frankly, not as good
a story behind it (as I trust my bank less than I'd trust a bar of gold under
my pillow when it comes to still having value tomorrow. Not that I have the
kind of spare change to make that a pressing issue, but ...)

So given that both BTC and Gold provide services that clearly people are
willing to pay for, who's to say they shouldn't have a valuation based on that
income like every other financial service provider in the world ?

------
samfisher83
All these LSTM Stock Market tutorials seem to be a variation of this guys
tutorial which he did a while ago:

[https://github.com/jaungiers/LSTM-Neural-Network-for-Time-
Se...](https://github.com/jaungiers/LSTM-Neural-Network-for-Time-Series-
Prediction)

Except Jakob explains the folly of the method.

Whereas the author of this piece says the following:

From the plot we can see that the real stock price went up while our model
also predicted that the price of the stock will go up. This clearly shows how
powerful LSTMs are for analyzing time series and sequential data.

~~~
mmq
I made this code [https://github.com/mouradmourafiq/tensorflow-lstm-
regression](https://github.com/mouradmourafiq/tensorflow-lstm-regression) at
least 6 months earlier than the first push in the github repo you mentioned,
and I am sure many people did some version before that. A lot of people
sometimes just need a reason to play with some technology. I assume that OP
used stock market to learn about LSTMs.

~~~
samfisher83
The guy I posted uses Keras which is what this guys code looks like.

I am sure plenty of people have done lstm on the stock market but the form of
the code looked similar between the two.

------
citilife
The problem with these types of systems are it's difficult to do backtesting
on a single continuous data stream... I write about that here:

[https://blog.projectpiglet.com/2018/01/perils-of-
backtesting...](https://blog.projectpiglet.com/2018/01/perils-of-backtesting/)

I recommend doing something similar to the original post. Neural networks tend
to produce valid looking output for a stock price easily, a random walk does
too. You have to find correlations, causations, and then review the results
carefully.

For reference, I wrote my own financial advisor (not directly utilizing deep
learning, that functions relatively well):

[http://projectpiglet.com/](http://projectpiglet.com/)

It works relatively well when checking the "causality" which also has it's
limits: [https://blog.projectpiglet.com/2018/01/causality-in-
cryptoma...](https://blog.projectpiglet.com/2018/01/causality-in-
cryptomarkets/)

EDIT: Added some evidence:

* 2016 stocks: [https://imgur.com/a/j8YWR](https://imgur.com/a/j8YWR)

* Early 2018 crypto: [https://blog.projectpiglet.com/2018/01/30-weekly-returns-usi...](https://blog.projectpiglet.com/2018/01/30-weekly-returns-using-piglet/)

* Early 2018 stocks: [https://twitter.com/AustinGWalters/status/976347632439209985](https://twitter.com/AustinGWalters/status/976347632439209985)

* All 2018 on Robinhood only: [https://imgur.com/a/2CxEFqI](https://imgur.com/a/2CxEFqI)

* Also (I am lettergram on HN), created: [https://hnprofile.com](https://hnprofile.com) and [https://redditprofile.com](https://redditprofile.com)

~~~
kdma
Interesting but i am a non believer can you provide figures that show gains
through years?

~~~
citilife
Added in an edit update - I have more, but just tossed some together for you
from 2016 - 2018.

------
sergioisidoro
I preliminarily did the same thing as in this article a while back (with
crypto data, since it's easily accessible), just as a way to learn keras.

Somehow my network always learned to output a delay of the input no matter how
hard I tried to shape it. I've searched through literature briefly and some
examples. Some blog posts even claimed they had good time series prediction
when I clearly saw they were having the same problem.

What I mean to say is, be critical of what you find online, and be critical of
your results. This is definitely not an easy problem, and some argue that it's
unsolvable all together :)

~~~
jorgemf
You are not alone. With time series that is the easiest solution for any
model. That is why you need a lot of work creating your datasets and the
experiment setup with time series. To avoid the easy solution and obtain
something useful.

------
zugweg
Do not minmax scale the data. New data outside ranges seen before will be
chopped off. Do make relative (high/low), so patterns you find generalize to
other domains. Don't fit complex LSTM models on tiny datasets. Somehow this is
the shiny thing all starters jump on, while the most simple models are the
most proven and robust. Don't just share such a model without giving a
disclaimer ("I never tried this out on real data, and would not invest my own
money in this solution"). This is real money, real economies, and you could
horribly crash these. That is also why you don't write with authority about
how to build a bridge (while you never crossed a bridge you've build before).
The only reason for the "clearly showing how powerful LSTM models are" is it
responding to test set samples it got wrong, and correcting with a lag. How
about some evaluation, preferably on another stock than trained on?

~~~
rl3
> _This is real money, real economies, and you could horribly crash these.
> That is also why you don 't write with authority about how to build a bridge
> ..._

Setting aside that first bit of hyperbole, I don't see the danger in the
bridge example. I mean, if engineers are building bridges based on information
they've gleaned from internet articles, there are larger issues at play.

As far as markets, retail traders basically exist and have existed within a
massive bubble of misinformation since forever. There's a reason order flow is
so valuable.

~~~
zugweg
ML is dangerously overhyped (that is an issue at play). This article uses min
max scaling, so it will treat a 30$ stock as if it were 20$. You want someone
like that managing your pension fund?

I refer to engineering code, because it is clear to me we don't want our
bridges build by those not skilled enough to make safe bridges. Yet, it is not
so clear with ML/AI, while the potential for damage may be even greater.

[https://en.m.wikipedia.org/wiki/Financial_Modelers%27_Manife...](https://en.m.wikipedia.org/wiki/Financial_Modelers%27_Manifesto)

------
rademacher
These types of experiments pop up from time to time. You have to compare the
performance relative to the single lag error. So the baseline is to use the
previous time step to predict the current time step. Ultimately, it's not
worthwhile to attempt to predict stock timeseries the SNR is far too low. It's
more valuable to attempt to predict a trend, e.g., +1 is there will be upward
movement in time window x, -1 if downward, 0 is within some epsilon of the
current price.

------
dointheatl
11:15, restate my assumptions: 1. Mathematics is the language of nature. 2.
Everything around us can be represented and understood through numbers. 3. If
you graph these numbers, patterns emerge. Therefore: There are patterns
everywhere in nature.

~~~
all2
> Everything around us can be represented and understood through numbers.

One of the biggest problems facing someone trying to assemble a numerical
model of some thing or process is whether or not their model is _adequate_ to
describe the phenomena they are observing.

This is why psychology remains a statistically modeled science, largely
relegated to anecdotal research.

(Note that my intent is not to speak ill of psychology and its cohort, only
illustrate that just because you can slap numbers on a thing doesn't mean
you've described it well.

Look at the transition of understanding from Newtonian mechanics to relativity
and quantum mechanics. Newtonian mechanics was enough until someone saw
Mercury was doing weird things it shouldn't.

And before Newtonian mechanics we had a broken description of the solar system
with more exceptions than rules. We were throwing numbers at a system and
failing, in some part, to describe it with consistency.)

~~~
throwawaymath
Well said. Having to add increasingly many edge cases to your model is a sign
your model is insufficient.

------
inputcoffee
This doesn't take away from the exercise, but I notice the model seems to
"predict" the movement just after it occurs.

It just lags the current price. If there is no alpha in the price signal, then
the rational thing to do is to just stay at the current price, which is what
the algo seems to have learned to do.

------
levesque
Pretty underwhelming. From the very limited experiment demonstrated, the LSTM
seems to be playing catch up rather than actually predicting trends.

~~~
minimaxir
I noticed this as well.

Many data science thought pieces don't quantify the results, and it's
suspicious that they're not included in detail (typically, the R^2 of a simple
stock market prediction model is super low, making it impractical putting
actual money on the line as it's barely _better than guessing randomly_ :
[https://twitter.com/minimaxir/status/1021885939361042432](https://twitter.com/minimaxir/status/1021885939361042432)
)

~~~
jefftk
_> barely better than guessing randomly_

In the stock market something that is consistently better than guessing
randomly is very valuable! You don't need much of an edge to make a lot of
money.

(I'm very skeptical about approaches like the OP though)

------
halayli
You can use whatever prediction technique you like, but if your model is
wrong, then so will the prediction. Predicting stock prices requires
considering as many factors as you can gather that goes into setting the stock
price, and how the factors correlate with each other.

> previous price of a stock is crucial in predicting its future price

This is a poor and incorrect model. The stock price has plenty other variables
and many of which are unknown. What's used here is just a single variable.

------
zz34
This is complete rubbish. Zero evaluation compared to any sane baseline, zero
thinking about representation (predicting raw price?!). I'll just say it, ML
was frankly better when fewer people knew it and there was less money in
claiming to be an expert.

------
usgroup
Signal/noise on the market is extremely low, and determining whether you’re
predicting something or just data snooping your way to a function that looks
like it predicts something is a serious competitive advantage in itself (...
because all you need then is novel functions to try).

Working out a function that passes a split test is inevitable and easy tbh.
That function then making money is highly unlikely. This is especially true if
your data sources are public.

------
truculent
My favourite part of this is that Francois Chollet (the author of Keras)
specifically warns against this exact use of LSTMs in his book (Deep Learning
with Python)

------
nabla9
The author uses stock market as a toy example. He is not actually attempting
to do serious forecasting.

Only comparing the prediction accuracy against simpler models like ARIMA or
VARMAX (as the author suggest) can tell you if the model has any use.

~~~
currymj
It seems like if you do anything with financial data like this, people often
aren’t capable of accepting that it’s just a toy example — they judge it as if
it must be intended to really make money. I don’t know why.

------
btbuildem
Again, as always, and every time -- if a stock market price-predicting scheme
worked, it would not be published on the internet for free. It would not be
for sale. It would be in constant use to make money for the authors.

------
AznHisoka
I'm skeptical of any system that proves it can predict stock prices vaguely.

I would be less skeptical if it was more humble, and specific in its
constraints. For example, could it predict the % probability a stock would up
X% if it had N consecutive negative days? Or, how many minutes/hours after
earnings are announced is the after-hours price usually indicative of what the
price will be the next day?

You can't predict everything, but you might be able to have a probabilistic
prediction of _specific_ scenarios. though even then I'll still be skeptical..
daily stock movements are totally random.

------
narrator
I saw some guys present who did this with currency trading. They had to
retrain the model every night and it didn't trade for the first four hours
after the market opened.

------
fredliu
Total noob question in this space (algotrading with DL algorithms).

What DL is good at, is to automate tasks that are easy for humans (e.g.
telling cat from dog, understand a sentence from sound wave, translation, etc)
but hard for machines without DL.

Now, if a task is even hard/borderline impossible for humans to achieve (e.g.
predicting stock market with higher accuracy and consistency), why would we
believe DL could do a better job than humans?

~~~
fossuser
DL isn’t just good at automating human tasks (see AlphaZero and AlphaGo).

It can be used to recognize patterns and train to solve problems better than
humans. Though in this case since the market is people making predictions
about a prediction I’m not sure how much it’d help.

There’s probably something about humans using a rough estimate of what the
stock has been historically to inform where it could go though so it’s value
is probably non-zero.

~~~
fredliu
By DL, i meant generic DL techniques like CNN, LSTM (used in this article),
which relies on large amount of labeled data to train, and predict on similar
data, thus my comment about automation. Alpha Go (Zero) is very specialized
for the game of go, not sure how much of its specialized algorithm could
transfer to other generic use cases.

~~~
rl3
> _Alpha Go (Zero) is very specialized for the game of go, not sure how much
> of its specialized algorithm could transfer to other generic use cases._

AlphaZero is a generalized successor, and it does just that:

[https://en.m.wikipedia.org/wiki/AlphaZero](https://en.m.wikipedia.org/wiki/AlphaZero)

~~~
fredliu
Good to know it generalized, but seems it's only generalized on board games
like problems where you have a problem space to search through, is that
understanding correct? If so, it probably won't help in the use cases we are
talking about here, right?

~~~
fossuser
Yeah it's not an AGI, but I think the response was more that DL isn't
necessarily limited to automating tasks that are easy for humans but hard for
machines without DL (which I think was your original statement).

DL could potentially do a better job by recognizing patterns in the dataset
that lead towards winning (making more money) that humans might miss. Like how
AlphaZero can recognize moves in Go or Chess that humans don't understand are
the best moves to make.

It's not obvious to me how this would be done, but I think it's plausible that
some clever implementation could help.

------
docker_up
These predictions and models work until they don't. And when they don't, they
blow up in spectacular fashion.

Look at Long Term Capital Management. It had big names behind it and the
model/strategy made a lot of money, until I believe the Russian bond crisis
caused the markets to act irrationally. Then everything they had invested in
quickly went down the toilet, and it was big enough to almost cause a
financial crisis.

------
joshgel
I wonder if someone could comment on how AI is used at big investment banks? I
assume its much more than technical analysis (like this model). I imagine that
NLP might be helpful for quickly ingesting a news feed and then making
buy/sell decisions based on the news in less time than humans can read and
react?

~~~
AznHisoka
As you mentioned, one way is to quickly ingest earnings announcements from
press wires/company websites. As soon as the earnings report hits the wire,
they want to immediately known how big a miss/beat was it, and whether they
should immediately dump it or buy more.

Other examples include analyzing the words/sentiments in earning calls, to see
if certain words indicate a bearish or bullish signal. For example if a CEO
mentions the word "headwinds", does that historically lead to a much better
quarter next time?

Some other companies do analysis with public/private data. For example, you
can use a weather dataset to calculate the average temperature for all cities
where there are restaurants owned by Cheesecake Factory in a quarter, and see
if it will negatively or positively impact sales.

~~~
this_user
I don't think you need AI for that, though. There are news feeds that are
specifically designed to be machine readable. All you need to do is parse it
and compare the actual numbers to the expected ones, then calculate the delta
for your pricing model. All of this is pretty simple. The real challenge is
being faster with this than the competition.

~~~
AznHisoka
You're absolutely right. None of this is AI. Wall Street is notorious for
making stuff sound more impressive then it really is.

------
hmate9
I don’t know what’s more embarrassing. This article, or the fact that it’s #1
on HN right now.

------
ifdefdebug
Is this graph at the end a one-step-ahead prediction? Because if it is, then
it looks very much like a 50/50 success/failure ratio. Which is what I would
expect, because otherwise you could extract a signal from it.

------
jaupe
I cannot fault your enthusiam but I can fault your method :)

Read Tsay's Financial Time Series Analysis for a better idea how to forecast
financial data.

The biggest problem you will face is stationarity: the statistical properties
of the data is not constant over time. For example, the mean and std dev is
not constant over time. Using returns instead of raw prices helps to make
better financial forecasts.

Two methods to explore:

1\. You are better off predicting stock prices by predicting future returns
and then forecasting is the current price plus predicted future return.

2\. You could use your neural model to predict absolute size of returns using
realized volatility.

------
chunsj
No, this is a bad example of prediction; if you bet money on the direction of
prediction, you'll lose much. Drawing prices and making it looking as similar
is not correct prediction. At least you should show >50% correct direction
prediction if it is considered to be "well processed" machine learning.

------
ttlei
From my experience, LSTM or other recurrent neural network models only work
"well" at forecasting bounded and periodic or oscillating time series. Might
work for something like seasonal sale data, but would fail spectacularly with
unbounded and chaotic time series like stock prices.

------
eternalny1
I used to run a technical analysis site for exchange traded funds (ETFs).

Skipping to the "results" of this experiment, I am seeing nothing. It looks
like a slow, short period moving average (SMA)... which does nothing in terms
of predicting anything. It just averages out the chaos over time.

------
norswap
> From the plot we can see that the real stock price went up while our model
> also predicted that the price of the stock will go up. This clearly shows
> how powerful LSTMs are for analyzing time series and sequential data.

Is laughter or tears the right reaction here?

------
drngdds
I know nothing about ML and not very much about finance, but I find it
incredibly unlikely that some random guy on Medium has an algorithm that can
be used to accurately predict the market. Why isn't he hilariously rich?

------
gcmartinelli
I believe a simple moving average would achieve the same result and that
mixing training and test data for validation invalidates your model results...

On a positive note, nice showcase of how simple implementation with Keras is.

------
mino
Seriously, how on earth is this the news #1 of the frontpage of HN? Isn't this
community supposed to know better?

Before clicking I thought "I bet they do stacked LSTM to predict the _next_
time step" :|

------
i_phish_cats
Let's take it easy on the author... LSTM's _are_ cool especially when you
first grok them, and stock data is the most abundant time series data out
there.

~~~
ipsa
It's the webdev equivalent of creating a log-in form tutorial and putting the
password in a JavaScript variable.

------
myth_buster
Think the important thing is model performance and tuning hyper parameters
which is missing from this tutorial.

Without that you are just regurgitating some model that does something.

------
tjpaudio
Simple linear regression would work just fine for something rudimentary like
this. This using neural nets stuff for everything lately is getting
ridiculous.

~~~
vilaca
What about trainnig a nn for linear regression?

------
jamesonthecrow
Regardless of whether or not this would make anyone money, it's a really nice
introduction to forecasting time series using LSTMs. Thanks for the post!

------
mbrumlow
Shame on me for looking. But this is a account that post only from fritz.ai --
might be nonsense spam.

------
hiker512
Well, I'm only on mobile. But the problem with stock prices is often that you
don't have good data, unless you are working for the big four or whatever. And
even then, they are apparently still working on integrating external data in
real time.

Imagine having the whole Twitter stream, maybe categorized by some network in
an automatic way.

Also prequential evaluation.

------
aj7
Feed it 3 cycles of a perfect sine wave. What does it predict?

------
java-man
It's tough to make predictions, especially about the future.

\-- Yogi Berra

~~~
bobosha
Have heard this quote attributed to einstein.

~~~
java-man
Or Bohr:

[0]
[https://en.wikiquote.org/wiki/Niels_Bohr](https://en.wikiquote.org/wiki/Niels_Bohr)

[1] [https://quoteinvestigator.com/2013/10/20/no-
predict/](https://quoteinvestigator.com/2013/10/20/no-predict/)

------
ericb
Predicting stock prices with deep learning is the "Killing Baby Hitler" of
Time Travel. Even though they know it won't work out, everyone tries it their
first time.

------
ElBarto
Isn't the concensus these days that every bit of information is squeezed out
so that the stock price always reflects all that is predictable?

In addition, major factors are external and trying to predict further price
only based on past price is bound to disappoint.

------
gaius
Oblig XKCD [https://xkcd.com/1570/](https://xkcd.com/1570/)

Seriously tho', articles like this need to come with a big banner at the top
saying _warning: do not try this at home_.

------
gcb0
wonder how effective those models will become when everyone starts believing
in those models to do trades.

~~~
caymanjim
The vast, vast majority of trading is fully automated algorithmic already.

~~~
auganov
Of which almost all is HFT. And by definition HFT has to be fully automated.
I'm not sure if there's any data on the kind of algorithmic trading that the
article talks about.

------
throwaway487548
Why on earth there is supposed to be even a correlation between so-called past
and future (bullshitting as a business aside)?

It is like predicting Trump by Nickson or, even better - predicting a next
Trump.

------
rustcharm
We'll know if it works when he gets to be a trillionaire.

