
Machine Learning Is the New Statistics - danielrm26
https://danielmiessler.com/blog/machine-learning-new-statistics/#gs.vG8Oa8A
======
markovbling
Machine learning is subset of statistics.

The standard text in ML, "The Elements of Statistical Learning" is authored by
statistics Professors.

Statistics is the new statistics. The rest is marketing bullshit.

~~~
danielrm26
Hmm. Maybe you're right.

But let me try to push back.

If we're trying to draw human-consumable wisdom about the state of the world
from data, simply capturing a snapshot and then applying some basic analysis
is one thing.

Creating self-improving mechanisms for doing this is another, even if the
latter use statistics in the process. Perhaps in a similar way that Statistics
uses Algebra yet is distinct in its description and capabilities.

I'm not convinced you're wrong here, just trying to talk through it.

~~~
wodenokoto
What the parent is saying is that machine learning is a buzzword for
statistics or at best another word for applied statistics.

Saying that applied physics will be the new physics because it changes the
real world, is nonsensical.

In earlier days Google called their translation algorithms statistical, then
they changed it to ML and now AI has come in favour again, so that is the word
being used.

------
oktavist
People that think ML/AI isn't statistics typically haven't studied statistics,
or have a marketing agenda. I can tolerate the latter as a fact of life. But
the former ... there is often a disturbing lack of statistical understanding
in "ML/AI" practitioners at the ground level, even though the vast majority of
their tooling is built on basic multivariate statistics. It's rather
inevitable give the sudden sex-appeal of the field, but will lead to an 'AI
winter' as those folks over promise and under-deliver. Computational
statistics, statistical learning, machine learning, pick your term certainly
continues to progress as computational horsepower improves. But as another
commenter noted, physicists/chemists still self-identify with quantum
mechanics even though the computational methods/approximations for molecular
dynamics continue to rapidly improve.

~~~
nojvek
Not sure about AI winter. I think a lot of advances are fuel by GPUs, ASICs
and colossal datasets. Also opensourcing the frameworks makes it easier for
new comers.

We can recognize objects, recognize speech, at almost human level accuracies.
That's a big milestone when you think about it.

Also technology improves exponentially when a ton of smart people funded by
crazy fuck you money work on pushing it forward.

------
minimaxir
While I'm not a fan of the modern abuse of the term "machine learning" as a
marketing buzzword, this article is tautological ("Machine Learning is the new
statistics because it is not statistics") and does not provide much insight
aside from invoking other buzzwords ("Reality Analysis Level 1"?).

~~~
danielrm26
The "insight" would be in the form of a falsifiable claim, i.e. that ML will
be bigger than the advent of statistics in terms of its ability to improve our
understanding of the world.

It's a claim, nothing more.

------
fonnesbeck
The statement "More wisdom potentially gets extracted when you apply
Statistics to more (and better) data, but the analysis itself doesn’t improve
with better data." simply isn't true. A hierarchical model, for example, is
increasingly able to model subgroups and additional levels of hierarchy as
more data are added. Penalized regression (or Bayesian regression) is another
example -- the model is structurally different as you change the quantity of
data.

The difference between ML and statistics is entirely semantic. Is logistic
regression a ML method or a statistical method? It is both!

~~~
danielrm26
Machine Learning is generically defined as a method of data analysis that
automates analytical model building.

That's the part that's going to have the impact---automated improvement of the
way Statistics is applied to data analysis.

Is that not major enough to be considered and discussed separately?

~~~
taeric
I think the kicker is that most machine learning is incrementing on a single
model. Typically one known from statistics before. The weak learner track of
combining models almost guess against this, but I think even then it is
usually the same shape of model.

So, I actually agree to an extent. Much as computers can be seen as the "next
logic". Only, it is such a "builds on" relationship that I think calling it
next is dubious.

------
charlescearl
While I am not statistically sophisticated to say whether the statement is
true or not, it seems that the mathematical machinery is strongly converging,
as witnessed by papers like [1], where some of the statistical machinery is
being developed on the ML side. It might be historically interesting to note
that 20-30 years ago, ML also more evenly spanned work that would be
considered AI based -- as including logic or knowledge representation see [2 -
4] as examples.

[1] Variational Inference: A Review for Statisticians,
[https://arxiv.org/abs/1601.00670](https://arxiv.org/abs/1601.00670)

[2] A Summary of Machine Learning Papers from IJCAI-85
[http://web.engr.oregonstate.edu/~tgd/publications/mlj-
ijcai8...](http://web.engr.oregonstate.edu/~tgd/publications/mlj-
ijcai85review.pdf)

[3] Chunking in soar: the anatomy of a general learning mechanism,
[http://repository.cmu.edu/cgi/viewcontent.cgi?article=2552&c...](http://repository.cmu.edu/cgi/viewcontent.cgi?article=2552&context=compsci)

[4] Explanation-Based Learning: An Alternative View,
[http://www.cs.utexas.edu/~ml/papers/ebl-
mlj-86.pdf](http://www.cs.utexas.edu/~ml/papers/ebl-mlj-86.pdf)

------
choxi
I think specifically Deep Learning is where all the magic is. The rest of
machine learning (SVMs, clustering, decision trees, etc.) are all old methods
that were invented in the 90's or earlier, recent lifts in data storage and
compute power have made them proportionally more powerful but they haven't
unlocked new technology as far as I know.

Deep Learning wasn't even possible until recently though, data and compute
power have made it possible as opposed to just proportionally better [1].
There have also been a lot of breakthroughs in Reinforcement Learning riding
on the wave of Deep Learning, and both of those (DL and RL) are more than
applied statistics.

1\. I think of this as a 0 to 1 innovation versus a 1 to n innovation if
you're familiar with Peter Thiel's terminology on that.

------
partycoder
Modeling is the next modeling.

