
Can a Machine Learning Model Predict the SP500 by Looking at Candlesticks? - mariofilho
http://mariofilho.com/can-machine-learning-model-predict-the-sp500-by-looking-at-candlesticks/
======
module0000
No, not at all. Move along. Everyone tries this initially, and (quickly)
learns this lesson. I suppose it cost you a lot less to find out the answer
via posting to HN though.

------
jimrandomh
"The reason the stock market is hard to predict is because it is a
prediction." \--Andrew Critch

------
Nasrudith
I wonder how well a literal candlestick model would work - as in literally
acting like a diviner watching candles melt and trying to map something from
it. Given how much is up to chance and comingled variables (how the market
performs correlated to temperature for instance could make it technically
correct if epistemologically utter stupid) it may wind up ironically better
than random chance. Still not something to stake your life savings on.

------
MisterOctober
12:40am : restate my assumptions

~~~
sj4nz
:) For anyone else who doesn't recognize this:
[https://www.imdb.com/title/tt0138704/](https://www.imdb.com/title/tt0138704/)

------
kuhhk
The book “A random walk down Wall Street” concluded “NO”

~~~
mruts
I mean, that book was looking at technical analysis and charting. Also, he was
just looking at mutual funds (which can only go long and suffer from a host of
other problems). Hedge funds weren't really invented/popular back then and
quant investing was practically non-existent.

Economists love to talk about EMH, and there's a great joke that illustrates
the difference between economists and traders: The economist is asked what he
would do if he saw a $20 dollar on the street. He replies "well it wouldn't
ever happen, because someone would have already picked it up!"

The trader is the one picking up the $20, and the economist is the one who
never believes it can exist.

Back to the point, it seems clear to me that there are few quant hedge funds
that can have consistently outperformed the market that disproves the null
hypothesis (no out performance) with a P < 0.05. Names like RenTech,
Bridgewater, AQR, 2 Sigma.

~~~
astazangasta
>it seems clear to me that there are few quant hedge funds that can have
consistently outperformed the market that disproves the null hypothesis (no
out performance) with a P < 0.05

The meaning p < 0.05 is that by random chance you expect to find 5% of
companies doing this well relative to the rest with no actual underlying
cause. The existence of a few companies that manage this is proof of exactly
nothing about those companies.

~~~
mikeyouse
There's also a strong hint that the most prominent and successful Funds with
long records of market over-performance were actually just insider trading all
along and their continued success is an indictment of our ability to
effectively police white collar criminals:

[https://www.newyorker.com/magazine/2017/01/16/when-the-
feds-...](https://www.newyorker.com/magazine/2017/01/16/when-the-feds-went-
after-the-hedge-fund-legend-steven-a-cohen)

~~~
mruts
Comparing SAC/Point 72 with RenTech or Bridgewater is a joke.

SAC is/was run by stupid goons (I know a couple of them, they’re dumb).
Bridgewater and RenTech employ top down systematic quantitative strategies (I
also know a couple of them, they’re quite smart) that consistently generate
alpha.

It’s easy to discredit people/industries that you are not familiar with. But I
work in it, and awhile a lot of hedge funds are full of shit, there are a few
that are the real deal.

You can systematically outperform the market year over year, and just because
you might be more familiar with SV than Wall St doesn’t mean we are a bunch of
crooks.

I work day in, day out, trying to make money for our funds investors. And
honestly, I find it extremely insulting that you are calling all of us a bunch
of criminals.

Edit: I’m sorry I sounded s little hostile, but I take pride in my work and
believe I am helping the world. I think a lot of HNers are unfamiliar with the
tech people on Wall St. And that’s okay, but we are just like you: trying to
make money for the firm using technology and quantitative reasoning. The same
kind of work you might do at FAANG, we do on Wall St.

------
ohiovr
If a well trained ai at scale to read the sticks and only the sticks fails,
why would it be considered possible for humans to do it?

~~~
ivalm
They can't. Charting is garbage, everyone knows this.

~~~
ohiovr
[https://www.investopedia.com/university/charts/](https://www.investopedia.com/university/charts/)

seems some people do. Charting is probably bunk and it might be impossible to
get reliable trades. if that could be scientifically proven people wouldn’t
lose their money tryng to use it.

~~~
TuringNYC
>> seems some people do.

Yes, people do charting. No, they dont actually make money on it. There is a
_vast_ market to manage money very profitably for anyone who can demonstrate
consistent performance (mutual funds, etfs, hedge funds, etc.) There are also
many sites now that will audit your performance and prove you are performing
well by tracing outcomes. If indeed charting was profitable, there would be
proof of it and people trying to profit off it by managing money using it.

------
dbs
How to set yourself to failure by starting with "one of the most widely known
techniques". You can't outperform the average by using the same tools as
everybody else.

------
uptownfunk
I was always curious if you trained a model on a literal visual representation
(pixels/image) of the charts or candlesticks, would the model be able to “see”
something that we can’t.

~~~
IshKebab
Of course not. It's the same data but presented in a harder to process format.

~~~
uptownfunk
It may be the similar but as they say, a picture speaks a thousand words, the
visual features that a CNN might pick up could be something completely
different than the features someone could think of. It is all about data
representation. Hypothetically, the data representation shouldn't matter, but
I think it is like viewing the optimization surface from a different angle, it
is possible to get something different out of it.

------
HockeyPlayer
Using RMSE doesn't make sense to me, your losses are linear to how wrong you
are. Shouldn't the error just be how far off you were?

------
jeletonskelly
Isn't that what quantitative trading is?

~~~
mruts
Quantitative trading isn't trying to predict the S&P500. It's trying to
predict the distribution of returns, which is a (slightly) easier problem.
Quants are just trying to make money, not predict the future. And judging by
the returns of quant funds like Bridgewater or Renaissance, the answer is that
while they might not be able to predict the S&P, they can generate alpha year
over year.

------
lawlorino
No. Saved you a click.

------
notroot
Yes, no, maybe, I don't know

Can you repeat the question?

~~~
notroot
Remind me what the ROC curve looks like for `can` again.

