
Machine Learning Often a Complicated Way of Replicating Simple Forecasting - henning
https://medium.com/@mikeharrisNY/machine-learning-often-a-complicated-way-of-replicating-simple-forecasting-methods-in-financial-25c38db2f624
======
PLenz
Yes, in a toy space ML replicates simplistic measures, there's only so much
information to be had in such a space to start with. The power of these
techniques is when we apply them to high-complexity or even unbounded-
complexity areas.

~~~
candiodari
And that you apply them to domains where the knowledge of the domain is basic
-at best. It enables people who know one subject well -machine learning- to do
speech recognition and weather forecasting well.

Plus this article has a number of problems. It compares machine learning to
MA(3). That it works pretty well totally ignores the selection of MA(3) vs
MA(5). MA(100) is "common" indicator often talked up (why I do not know,
except perhaps it protects you from really big errors, but other than that
it's not very helpful).

If you actually try to use the prediction of MA(3) you will find it is often
very hard to use, because it's always just too late. I wonder if the machine
learning version had the same problem, or not.

~~~
longerthoughts
>If you actually try to use the prediction of MA(3) you will find it is often
very hard to use, because it's always just too late. I wonder if the machine
learning version had the same problem, or not.

Why does the sensitivity matter when we know that the accuracy was so similar?
The point on sensitivity does make me wonder how an exponential moving average
would've performed though.

~~~
candiodari
Because there are "chaotic cases" in statistics. Cases where there are so many
possible patterns that stating which pattern the data follows doesn't have any
less information than the data itself (or only has a little less information
than the data itself). Financial market data is quite notorious for being one
such case (if not there wouldn't be any poor mathematicians).

So MA(3) is not a general indicator. It's not universally useful, and it isn't
the only one this article considers. It was chosen from a great many possible
indicators (go to tradingview.com, find a graph, expand the indicators tab,
and look at the list. And keep in mind that ALL MA's are just one entry). So
there's tens of thousands of indicators that MA(3) was chosen from. So the
predictive value of MA(3) as an indicator in the general case is really only
1/10000th of what this person claims it to be (actually infinitely less, but
let's assume you use, say, MA(200) as the limit of what you're willing to
consider, which makes it a finite number).

Predictive information is this: let's say I tell you I know what the outcome
is of a football match. And at the end of the match, I pull the outcome out of
a stack of papers. The size of that stack of papers is inversely related to
how informative my prediction was. If I just put down one paper before the
match, that's great. If I put down 1000. Not so great. For this guy, the stack
of papers was pretty thick.

So the article is saying "I can, out of a great many indicators, find one that
performs similar to this machine learning algorithm". Unless the article also
gives a clear and definitive reasoning for why THAT indicator was chosen for
that dataset, it doesn't contain much information at all.

~~~
longerthoughts
You're presenting this from the perspective of the author's selection of
MA(3), but I think what you're really getting at is yes, MA(3) happened to be
comparable to the ML model in this case, but the beauty of the ML model was
that it _knew_ to nearly replicate MA(3) out of a great many common
indicators. Is that essentially what you're saying? If so, I'm curious how
things like longer period moving averages or an exponential moving average
perform relative to both MA(3) and the ML model in this case.

------
longerthoughts
Why are so many people reading this as an attack on ML? The author is simply
reminding people that ML isn't a universal remedy for uncertainty. "Stop
writing ML models solely for the sake of writing ML models" is a very
different message than "ML didn't improve upon MA(3) in this case so it's
universally worthless".

------
iotasigma
I understand that the general sentiment for this article seems to be
dismissive but I want to share this awesome University of Vermont lecture
series/course. It's related to forecasting mathematics and includes an
introduction to fractal geometry which I haven't seen much of elsewhere.

[https://youtu.be/Lr3qrB6dPCM](https://youtu.be/Lr3qrB6dPCM)

------
stanfordkid
Pretty banal article. TLDR: A very complicated model will fit simple data, as
will a simple model. Modern ML/AI has nothing to do with 1D timeseries
forecasting.

~~~
cs702
Was about to write the same thing.

What the heck is this doing on the front page?

~~~
henning
There have been thousands of papers and many books written on the subject of
using machine learning for time series forecasting and an unknown amount of
money traded/invested with it. Most of them cast trading as a time series
forecasting problem where performance is measured by mean square error. They
usually then assume that would lead to profitable trading.

You don't get to define what "modern machine learning" is any more than I do.

The application of complex models to higher-dimensional domains have problems
of their own. Both the successes and failures are worth writing and thinking
about.

The post is an important reminder about simplicity and complexity in modeling.
That's why I submitted it.

~~~
stanfordkid
I don't understand why I'm being downvoted for this opinion -- the topic of
overfitting gets beaten to death by the 3rd week of every intro
AI/statistics/ML class out there. I didn't find it very worthy of the
frontpage (or a blog post).

------
m3kw9
But the power comes when the model doesn’t overfit and can predict way more
than your simple models

------
TTPrograms
"Is advanced machine learning a very complicated way of reproducing the
results of trivial forecasting methods? In many cases it appears this is the
case."

If by "many cases" you mean "the one example I looked at which was carried out
by an undergrad" than sure. That sort of phrasing really gives away the
author's bias.

~~~
longerthoughts
The author isn't arguing that ML isn't useful, they're simply pointing out
that it isn't worthwhile for all problems and data sets.

